Agentic AI Governance Requirements in the EU AI Act

Compliance is not a hypothetical concern. For agentic AI, it is hard to tie legal duties to systems that choose tools, keep memory, and act across several steps. Begin by mapping each obligation to the part of the architecture that must enforce it and produce evidence.

What makes agentic systems structurally harder to govern than conventional AI

An agent can break a rule by taking a series of actions that are each allowed on their own. An agent preparing a quarterly report might read a CRM, access financial databases, fetch external data, generate visualizations, and email the result. The agent may be allowed to read the database and to send email. Together, they can become a data exfiltration event.

Kaptein et al. (2026) describe this as a path-dependent governance problem. Controls built for deterministic, statically specified systems cannot set out all the rules that agents need. Memory makes the problem harder still. A wrong or compromised decision can carry over into later sessions, and when agents delegate to one another, it is harder to trace the original authority chain. Syed et al. (2026) identify related failure modes including indirect prompt injection, backdoor triggers, goal misgeneralization, memory contamination, and cross-session data leakage.

The Act’s application to “agentic” systems therefore starts with classification. An organization must establish the system’s intended purpose, functionality, application domain, risk classification, and provider and deployer roles. High-risk systems face requirements covering risk management, data governance, documentation, record-keeping, transparency, human oversight, accuracy, robustness, cybersecurity, conformity assessment, and EU database registration. These duties extend beyond the underlying model. Article 9 requires risk management to continue with evidence to back it up, so governance must follow the agent for as long as it is in use.

How conformity assessment and technical documentation translate to agent architecture

Conformity assessment and technical documentation start before the system is deployed. For an agent, documentation must describe the intended purpose, functionality, and risk classification through the full range of tools it can call and the decision logic governing their use. Documentation must supplement model weights and training procedures with details of which enterprise systems the agent can reach and which actions those connections permit. The documented system must include those capabilities.

That record also needs a version history. Agent behavior is non-deterministic and path-dependent, so a static snapshot cannot show all that an agent can do as its capabilities change. Syed et al. propose the Trustworthy Agent Development Lifecycle, or TADL, with six phases covering specification, design, training, evaluation, deployment, and monitoring. Its phase-specific evidence artefacts and risk-based decision gates support the same lifecycle approach that conformity assessment demands.

EU database registration is a separate obligation, but it depends on an equally basic internal record. Each deployed agent needs a stable, unique identity tied to its capabilities and granted permissions. An internal MCP registry gives a tool-calling deployment a governed catalog of servers that have been vetted, with access control, support for more than one environment, and audit trails. It connects the agent inventory to the tools agents can use. Treating those servers as governed artefacts makes their origins, changes, and deployed versions traceable across development, CI/CD, and production, giving technical documentation a maintainable foundation.

Where data governance obligations land in a tool-calling agent architecture

For a tool-calling agent, data governance reaches the point where it connects to enterprise sources. MCP provides transport. The protocol alone does not establish authority, enforce the full access policy, or audit the reasoning behind a request. Those controls depend on the implementation. A catalog of approved tools is therefore only part of the answer, because each connection must also respect the rules governing the data it exposes.

The June 2025 MCP spec revision formalized servers as OAuth Resource Servers and mandated Resource Indicators under RFC 8707. This prevents a token issued for one server from being reused to access another when correctly implemented. Existing implementations still differ. Some use per-user OAuth authentication, while others rely on bearer tokens or API keys shared across an organization. The revised specification does not fix those deployments on its own or settle data sovereignty concerns when requests pass through infrastructure in another jurisdiction.

Inherited permissions make an agent’s access reflect the querying user’s existing entitlements. The supporting controls include OAuth 2.0 authentication with credentials outside AI context, per-operation role-based and attribute-based authorization, path and scope controls, rate limiting, and sensitivity label evaluation. These controls belong at the connection. Attribution is equally important because a record of a tool call alone does not establish which user authorized it, what data it retrieved, or what happened to that data.

What human oversight actually requires in an agent's execution path

Human oversight requires a person to be able to intervene before an agent’s action becomes irreversible. A retrospective review cannot provide that opportunity. Oversight must therefore sit inside the execution path, where a proposed action can be assessed in context before it runs. This is a different requirement from displaying an approval option somewhere in the interface. The intervention point must correspond to the action that creates the risk.

Kaptein et al. explain why prompts and static access controls cannot do this job alone. Prompts can reduce the probability of a bad sequence, but they do not strictly enforce a boundary. Static access controls can forbid a category of action, but cannot decide differently because of what happened earlier. The proposed runtime policy instead evaluates agent identity, the partial execution path, the proposed next action, and organizational state to produce a policy violation probability. With this policy, the earlier database read counts when the agent later proposes an external email.

The 2026 Singapore Consensus on Global AI Safety Research Priorities treats Interruptibility and Human Oversight as distinct agentic risk management principles. Interruptibility is the architectural property that allows an agent to be stopped mid-sequence without corrupting state. Shadow MCP servers installed outside IT oversight skip this governed execution path, so provisioning teams through a governed registry is a control built into the structure, as well as a way to make administration easier. Administrators set which tools each team can use, so the tools stay within the scope of oversight.

What a compliant audit log for an agentic system must contain

A record of what an agent did is not proof that it was authorized to do it. The authorization decision needs its own evidence. For a high-risk agentic system, the audit trail must capture what the governance system decided about an action, including the policy version in force at the time. Standard MCP connection logging captures execution without necessarily capturing the reasoning, policy, or authority behind it.

Attribution-level logging supplies the link between an action and its authorization. It records which user authorized the request, what specific data was retrieved, and what action was taken with it. The policy version and the point in the execution path then establish the context for the governance decision. If an agent keeps information, logs from separate sessions cannot by themselves show whether it still follows the stated policy in later sessions.

The Singapore Consensus separates Auditability from Legibility, which requires an agent’s reasoning and actions to be interpretable by a human reviewer after the fact. More raw text does not always make that easier to understand. Cryptographic signing linked to an immutable hash chain is one technical approach to making action records tamper-evident, but integrity alone does not supply missing attribution. A sufficient record brings these elements together in a detailed, centralized, encrypted system of record. It connects the agent and authorizing user to the retrieved data, subsequent action, execution-path context, policy version, and governance decision, so a reviewer can assess authority as well as execution.

How provider and deployer roles under the Act divide compliance responsibility across an agentic stack

The Act divides responsibility according to provider and deployer roles. The provider places the system on the market or puts it into service and holds the primary conformity obligations. The deployer uses the system under the provider’s authority and holds human oversight and monitoring obligations. An agentic stack complicates that division. Different parties may hold these roles at different layers of its foundation model, orchestration layer, MCP servers, and enterprise data sources.

The practical requirement is to map each obligation to the party controlling the relevant layer. Vendor documentation becomes a compliance input because an enterprise cannot complete its own assessment work from a description of only the components it controls. Internal ownership needs the same precision. The team closest to the business context can own the relevant AI skills and data connections, aligning operational oversight with the deployment’s domain. That ownership supports the organization’s duties rather than making responsibility disappear into the stack.

Team-scoped MCP configurations give that ownership a concrete boundary. A sales team might receive CRM read/write and Slack tools, an engineering team GitHub and Jira, and a finance team accounting read tools. A governed registry adds version control, ownership records, and access scoping, making it possible to establish who approved a capability and when. By contrast, teams independently rebuilding the same prompt can create untracked deployments with different risk characteristics and no corresponding documentation, turning duplication into a governance failure as well as an efficiency problem.

The practical starting point for building an EU AI Act-compliant agentic architecture

For an enterprise turning these obligations into an implementation plan, the first step is the agentic asset inventory. Every agent in use needs a unique identity and a record of its intended purpose, what it can do, and what it has permission to do. This comes before assessment and registration work. Without that inventory, the organization lacks a stable account of what it is assessing. The inventory also gives the organization a way to assign ownership and track later changes in what an agent can do.

The second step is inherited and synced permissions derived from users’ existing entitlements in source systems. Access should not be redefined through a broader service account for each new AI integration. The third step is attribution-level audit logging. Those records need to connect the action to the authorizing user, the policy version, and the point in the execution path. Establishing that evidence makes subsequent human oversight demonstrable.

The fourth step is a governed registry with version control, using a private sub-registry of vetted MCP servers. Its structured vetting pipeline includes automated security review, dependency analysis, SBOM generation, and compliance checks. This makes documentation maintainable as tools change. The fifth step is runtime path-dependent policy enforcement, which checks proposed actions before they run based on the path the agent has taken. It closes the sequence-level gap left by prompts and static access controls.

The order reflects dependencies among the five projects. An inventory anchors documentation, permissions bound data access, and attributed records make policy decisions reviewable. A governed registry then keeps capabilities and ownership traceable as deployment expands, while runtime enforcement applies policy to actual behavior. These are also the controls that let teams share agentic skills without rebuilding permissions and oversight for every new use. Governance infrastructure and scaling infrastructure are the same architecture.