人工智能代理正在测试零信任安全的极限

This article is brought to you by Queen’s University Belfast . Several teams around the globe are leveraging AI by creating a task force of agents managed by an orchestrator and creating applications to support risk assessment, client profiling, match making, and information yield with a slight overview of decision making. One of the important components for these agents is to ensure that the logs are managed and historic relevance is not being used to avoid skewing towards same decision making. Thus, the system is expected to keep checkpoints in place where agents are prompted to delete such historic relevance. However, despite such methodology in place, a future prompt could make the agents bring forward the information that was supposed to be deleted and can be tricked into revealing more information, which is a classic case of context recovery attack. So, the question raised here is: How can we trust that our instructions have been followed? And where is the proof? There is an element of utilizing zero-trust architectures (ZTA) for orchestrating and managing agentic activity, which simply means all agentic interactions use the core principle of “never trust,” “always verify.” ZTA components include a policy enforcement point (PEP) that has to be trusted to maintain trust across the system, so the question that remains is where is the component of zero-trust, and shall we trust the policies enforced by PEP? Having a pro-agentic approach and utilizing agents could be helpful with the right guardrails in place. Opening our systems to random agents that are being trained on off-the-shelf models and blindly relying on them can pose significant cyber risks. For example, vibe coding by using an AI model to help develop software, giving away the entire codebase to an untrusted entity outside the organization can lead to vulnerability sweeping across multiple surfaces. Vibe coding by using an AI model to help develop software, giving away the entire codebase to an untrusted entity outside the organization can lead to vulnerability sweeping across multiple surfaces. This opens critical discussions towards how vibe coding could interfere with the positioning and implementation of trusted execution of the agentic queries. Consider an autonomous web of agents that relies on agents’ inputs to drive the operations. When the system is operating under low tolerance of failure and with non-tracking based agentic workflows, it could be computationally expensive to trust and verify each query in real-time that leads to the common trade-offs of memory vs execution. Another risk to consider is the zero-day attack. When it happens, how will the system react? Would such an event be manageable without a total infrastructure breakdown? The scale of impact can be observed from over reliance on agentic ecosystems with only one trusted layer. This could mean that when failure occurs, the entire ecosystem could be exposed, just like there will be no autonomy if the backbone network is not available, which essentially means no layer to place trust. How quickly can de-authentication attacks occur? An in-house experiment done at Queen’s University Belfast (QUB) provided an initial understanding of how the scale of agentic attacks could mean a lower window of vulnerability. The team focused on very popular de-authentication attacks and with agentic orchestration, the swift rate to do de-authentication attacks could go as low as 2 to 3 seconds. This gives little to no time to react and decide on patching the system. There is another aspect of the large-scale agentic system, which establishes trust under asynchronous autonomy. Trust serves as a critical foundation for traceability. How would you ensure that all the agents operating towards a common goal are aligned and understand the delays associated with their goals? Our team worked around this space, which at first involved demonstrating that the problem exists, and its impact and scale can be huge when you must prove trust under boundaries of asynchronous autonomy. Solutions towards such problems can be simplified by keeping things like AI-bill of material or leveraging distributed ledger technology that can provide underlying infrastructure of trust via traceability and immutability and keep the agentic activity and associated trust vectors agnostic to the platform. Within trust, another factor that is not well explored but needs a thorough investigation is the concept of horizontal anomalies. It defines how agents interact differently with various sources under different settings. As agents behave correctly and align themselves with the architectural expectations for one specific task or application, that does not necessarily mean these can be trusted for their operations across all associated components. How to identify such anomalies and what methods can be employed without complying with such agents remain open questions and topics for further exploration. The critical aspect is that utility of agents with semi to low trust governance offer short-term high benefits, but the cost of recovery when the system fails is yet to be explored, and this will not be a contained issue. We are looking into the system where agents are already sweeping into different entities and may not allow complete recovery. For example, a mere supply chain issue because an agent going rogue is now not a containable small task but needs sweeping across all the touch points of those affected agents. This essentially means despite the number of agents and their ability to trigger proxy-agents, which may not be directly linked to the wider ecosystem, there is a need to ensure that computationally expensive trackers are in place for all the agents. Other relevant PhD works carried out in our research group are towards sovereignty of AI, exploring aspects of trust and accountability. Our hypothesis is that sovereignty and trust need to be governed together as onus of provenance is high on the host of agents. Thus, it needs understanding of laws governing trust-based executions of AI. The important challenge here is that there is a vital difference between missed information and missed instructions, and both have different impacts when it comes to trusted accountability. Does true zero-trust architecture exist? The real-world issue with these systems lies in the failure to adopt approaches that historically have relied on state-based modeling when governing trust. In stateless systems where agentic autonomy is high, the convergence and reliability towards trust-provenance decreases significantly. Thus, it brings us back to the original question – does a true zero-trust even exist? Can policies be trusted? If so, how should these be managed without falling into the trap of policy-based accessibility and control issues? Despite the argument that PEP can help manage access between the agents, placing a single entity and expecting it to scale towards all agentic interactions is underestimating the problem space. Formal verification of trust is another part of a trust-based agentic ecosystem. When dealing with AI models, understanding the robustness helps to put trust in place for these models. To build our case and settle around the challenges of trust embeddings in Agentic AI, within Cyber AI hub , we are exploring opportunities to help companies scale up as well as build their pipeline of agentic applications for commercial viability. However, the element that governs our regular work and decision making are scale of problem and number of agentic deployments, anomaly detection across the agentic workflows, continuous evaluation of the attack surfaces and vulnerability assessment, exploring differences in the pure agentic AI and task-driven multi-agent system. We are working with industries where trust is a pre-requisite, and these organizations put trust as a guardrail managed via internal policies with a pure focus on securing the systems through policies rather than trusting the third-party agentic service providers. Additionally, trust in services is presented by ensuring constraints around their service models where agentic activities are tracked and managed via logs including management of their token revocation list to avoid replay attacks depending on the type of application. Formal verification of trust is another part of a trust-based agentic ecosystem. When dealing with AI models, understanding the robustness helps to put trust in place for these models. Things like balancing the adversarial accuracy along with certified adversarial robustness can help to determine how much the model can be trusted. But the question is, can we settle for anything less than foolproof when it comes to deployment of agents in the real world? And if so, how much impact can be controlled, and who is responsible for the remaining component? References Wickremasinghe G, Sandjaja FR, Rafferty K, Sharma V. State-verification in asynchronous blockchain-driven agentic AI systems. 40th International Conference on Information Networking (ICOIN) 2026 Jan 14 (pp. 139-144). IEEE. Sandjaja, F. R., Abdul Majeed, A., Abdullah, A., Wickremasinghe, G., Rafferty, K., & Sharma, V. (Accepted/In press). Policy design in zero-trust distributed networks: challenges and solutions. In Industrial Networks and Intelligent Systems: 12th EAI International Conference, INISCOM 2026: (to appear in proceedings). Melis M, Del Rincon JM, Sharma V. Improving Certified Robustness via Adversarial Distillation. arXiv preprint arXiv:2606.31653. 2026 Jun 30 (Accepted to appear in BMVC’26).