Determining the State of the Art in General-Purpose AI Risk Management: From Code to Practice

The EU’s General-Purpose AI Code of Practice Safety and Security Chapter (‘the Code’) requires providers of the most advanced AI models to meet the ‘state of the art’ (SOTA) for several of its key requirements. Yet, both the AI Act itself and the Code leave essential questions unaddressed:

• What makes a safety and security practice SOTA?
• Who must be involved in advancing and determining SOTA?
• How should competing claims be adjudicated?

We address these questions in a new policy memo co-authored with the Oxford Martin AI Governance Initiative. An early version of the memo was discussed at a workshop held on 22 October 2025, which brought together a range of experts on AI and other high-risk domains, including contributors who are listed as co-authors on this memo

Dynamic References

For many of its requirements, the Code relies on dynamic references, such as ‘best practices,’ ‘state of the art,’ and ‘more innovative processes’. These are meant to future-proof the Code by allowing compliance expectations to evolve alongside technological developments, without requiring continuous formal revision. Among these, SOTA plays a special role, as it is the only dynamic reference expressly required for some of the Code’s most central methods, such as model evaluations (Measure 3.2; Appendix 3) or risk modelling (Measure 3.3).

The Expert Ecosystem Determines SOTA

The Code defines SOTA as a dynamic reference to the ‘forefront of relevant research, governance and technology.’ Most importantly, we argue that SOTA is best understood as a process-driven concept: for SOTA to go ‘beyond’ best practices, the basis of validation must move from provider acceptance to the broader expert ecosystem. Any actor can advance the forefront, so long as their claims are epistemically robust, for instance, validated through peer review, independent reproduction, or expert elicitation.

Operationalising SOTA: Availability, Proportionality and Verifiability

To make SOTA assessments structured and comparable, we propose three criteria, reflecting relevant precedent in EU law:

1. Availability. A measure must be technically feasible for providers to adopt, not purely aspirational. In practice, this may not require full public disclosure: where sharing a method raises security concerns or might undermine its effectiveness, providers may satisfy the requirement through partial disclosure.
2. Proportionality. A measure should sit at the optimal frontier of effectiveness and burden. Where risks are uncertain or elevated, the precautionary principle justifies more demanding measures even at greater cost.
3. Verifiability. Claims must be open to independent scrutiny, not validated by providers alone. This means transparent documentation of methods and, where empirical testing is involved, reproducible results.

An Institutional Process

Because the entire expert ecosystem can advance SOTA, it requires an appropriate institutional process to function effectively. Our proposed process consists of three steps:

1. Monitoring identifies gaps in existing risk management measures, areas where current practices may be inadequate or absent, in light of new developments, such as the emergence of novel risks.
2. Innovation and evidence sharing, through which any actor, from providers to academia, civil society, and independent experts, addresses identified gaps through research, sharing the resulting measures with the AI Office, and if appropriate, the wider ecosystem.
3. Assessment and communication by the AI Office, supported where necessary by the Scientific Panel, which evaluates whether submitted measures meet the criteria for becoming the new SOTA, and communicates the SOTA update.

Five Priority Recommendations for the EU AI Office

To make this process operational, we offer five priority recommendations for the EU AI Office:

1. Issue guidance to operationalise the SOTA assessment, for example specifying how the effectiveness of a risk management measure should be evaluated.
2. Create secure evidence-submission and sharing channels, introducing structured, secure submission routes for SOTA claims.
3. Formalise the Scientific Panel’s supporting role, empowering the Panel to provide ad hoc scientific opinions on contested or high-impact SOTA claims.
4. Leverage existing expert networks, including leading researchers, AI Safety/Security Institutes, and the Network of Evaluators, to regularly collect structured insights on emerging SOTA developments.

Establish and maintain a SOTA Register, providing an overview over current SOTA measures, with tiered access for sensitive details. While near-term mechanisms like the Signatory Taskforce communicate SOTA to providers, the register would also orient the broader ecosystem.

For more insights, research, and expert analysis, follow The Future Society on Linkedin.