How AI Agents Are Governed Under the EU AI Act

“2025 is the year of AI agents.” Speaking at this year’s World Economic Forum in Davos, OpenAI’s Kevin Weil drew attention to a major new frontier for artificial intelligence. 

What are AI agents? An emerging class of AI applications, agents are designed to automate complex real-world tasks at great speed and with less need for human involvement. 

This means that AI agents will be able to take actions independently. As of now, agents remain focused on specific tasks like managing inboxes and booking flights. But if they live up to their promise, they may soon become highly-capable digital coworkers or personal assistants. 

The prospect marks a fundamental shift: AI is directly influencing real-world environments. And this growing level of autonomy introduces significant risks. For the impact of autonomous AI systems to be more helpful than harmful, governance will be key.

Our report Ahead of the Curve: Governing AI Agents under the EU AI Act is the first comprehensive analysis of how AI agents are regulated in the European Union. This builds on The Future Society’s efforts over the past decade to support AI governance in Europe and beyond, including most recently through participation in the process to define the EU AI Act’s General-Purpose AI Code of Practice.

Although the AI Act was not originally designed with AI agents in mind, we find that the world’s most comprehensive regulatory framework for governing AI does in fact apply to agents. But gaps remain, which require additional guidelines from the European Commission and an update to the technical standards that turn the legal text into concrete governance processes.

As AI agents evolve and risk pathways emerge, governance frameworks must keep pace. Our analysis identifies open questions that present clear opportunities for the European Union to act on, while other jurisdictions should also take note of the EU’s approach.

Our three primary findings on AI agent governance under the EU AI Act: 

1. Agent risks are governed by the Act’s provisions for both general-purpose AI models and high-risk systems. Since most current agents rely on GPAI models with systemic risk (GPAISR), model providers must assess and mitigate systemic risks from AI agents. However, agents can also be high-risk systems, depending on their specific use case. Moreover, agents that are intended for multiple purposes can be assumed to be high-risk, unless the provider takes sufficient precautions.
2. Managing agent risks effectively requires governance along the entire value chain. This must account for the various asymmetries between actors, such as the superior resources and expertise of model providers and the context-specific information available to agent deployers. In general, model providers must build the fundamental infrastructure for AI agents, system providers must adapt these tools to their specific contexts, and deployers must adhere to and apply these rules during operation.
3. The AI Act governs AI agents through four primary pillars: risk assessment, transparency tools, technical deployment controls, and human oversight design. Within these pillars, we identify ten measures to propose specific requirements for GPAISR providers, agent providers, and agent deployers–supported by relevant articles in the Act. Particularly for high-risk systems, the technical standards under development will likely fail to fully address risks from agents; our work offers a starting point for future updates to these standards.

Decision tree presenting the logic for assessing the applicability of the EU AI Act to AI agents.

Read the full report for more insights.