Systemic Risk Designation

The primary quantitative trigger for systemic risk designation. A general-purpose AI model is presumed to have high impact capabilities when the cumulative amount of computation used for its training exceeds 10^25 floating-point operations (FLOP). This metric serves as an objective, measurable proxy for model capability and potential societal-scale harm. Providers must notify the Commission within two weeks of meeting this threshold and prepare for additional obligations.

The European Commission holds the authority to designate a model as posing systemic risk ex officio or based on a qualified alert from the scientific panel, even if it falls below the compute threshold. This captures models with equivalent impact based on criteria such as:

• Number of business users and reach
• Performance benchmarks and novel capabilities
• Potential for harmful manipulation or autonomous replication Providers have the right to present reasoned arguments against such a designation.

Providers of designated models must perform and document state-of-the-art adversarial testing to identify and mitigate systemic risks. This includes red-teaming for:

• Chemical, biological, radiological, and nuclear (CBRN) risks
• Offensive cyber capabilities enabling large-scale attacks
• Loss of control and autonomous replication
• Large-scale discrimination or disinformation Results must be reported to the AI Office without delay.

A mandatory obligation to document, report, and track serious incidents arising from the model's operation. An incident is considered serious if it directly or indirectly leads to:

• Death of a person or serious damage to health or property
• Major disruption to critical infrastructure
• Widespread violations of fundamental rights A detailed incident register must be maintained and made available to the AI Office and national authorities upon request.

Systemic risk designation mandates a cybersecurity protection level commensurate with the risks. This goes beyond standard IT security to include:

• Model-specific threat modeling for weight theft and unauthorized access
• Protection against model leakage and exfiltration
• Securing the model architecture and training infrastructure
• Preventing unauthorized fine-tuning or downstream misuse The security posture must be documented in the technical file and continuously updated.

Systemic risk obligations cascade to downstream providers integrating the designated model. A downstream provider modifying or fine-tuning a systemic-risk model inherits the full set of obligations unless they can demonstrate the modifications do not retain or amplify the systemic risk profile. This creates a chain of accountability ensuring that risk mitigation is not circumvented through corporate structuring or technical partitioning.

The specific registration obligations imposed on providers of foundation models that can serve a variety of downstream tasks. Unlike narrow high-risk system registration, this process requires disclosing training compute budgets, energy consumption, and evaluation results against standardized benchmarks. Providers must demonstrate robust adversarial testing and incident reporting mechanisms before market placement.

An AI system classified under the EU AI Act as posing significant risk to health, safety, or fundamental rights. Key characteristics include:

• Mandatory conformity assessment before deployment
• Strict human oversight requirements
• Continuous post-market monitoring obligations
• Registration in the EU database before market placement

Systemic risk designation imposes additional obligations beyond this baseline classification.

Testing model resilience against malicious inputs designed to bypass safety guardrails. For systemic risk models, this includes:

• Red-teaming for chemical, biological, radiological, and nuclear (CBRN) capabilities
• Jailbreak resistance testing against prompt injection attacks
• Data poisoning recovery assessments
• Model inversion and extraction defense validation

Results must be documented in the technical file submitted during registration.

The technical mechanism connecting a registered AI system's unique ID to a mandatory incident reporting portal. For systemic risk models, providers must:

• Report serious incidents within 72 hours of discovery
• Document near-misses where safety systems prevented harm
• Maintain immutable audit trails of all reported events
• Link incident data to specific model versions and deployment contexts

The process of filing a structured transparency artifact detailing a model's evaluation results, limitations, and intended use. For systemic risk designation, model cards must include:

• Capability benchmarks against known dangerous tasks
• Alignment evaluation scores and failure modes
• Training data provenance with copyright compliance attestation
• Compute resource disclosure including total FLOPs and hardware configuration

The continuous, systematic process by which providers collect and analyze data on real-world AI system performance. Systemic risk models require:

• Real-time monitoring for emergent capabilities
• Automated detection of misuse patterns across deployments
• Periodic re-evaluation against updated safety benchmarks
• Supply chain transparency for downstream fine-tuning and adaptation

This ensures ongoing compliance after initial registration.