Post-Market Surveillance

The immutable recording of every AI-driven decision, including its inputs, outputs, and confidence scores. This creates the evidentiary foundation for post-market surveillance, enabling auditors to reconstruct the system's reasoning for any specific transaction. Without comprehensive logging, demonstrating compliance with the right to explanation is technically impossible. Logs must capture model version, feature vectors, and any human override events to establish a complete chain of custody.

A primary trigger for post-market intervention. Concept drift occurs when the statistical relationship between model inputs and the target variable changes in the real world, rendering the model's learned patterns obsolete. Unlike data drift, which is a shift in input distributions, concept drift means the same input now yields a different correct output. For example, a fraud detection model may suffer concept drift as criminals adapt their tactics, requiring continuous drift detection and model retraining.

The predefined protocol activated when post-market surveillance detects a critical failure or safety violation. This is not a generic IT process; it must account for AI-specific failures like hallucination cascades, biased output clusters, or reward hacking. A robust plan includes:

• Model rollback to a last-known-safe version
• Kill switch activation for immediate containment
• Forensic snapshot capture for root cause analysis
• Regulatory notification timelines per EU AI Act Article 73

The automated, real-time verification that a deployed AI system remains within its conformity assessment boundaries. This shifts compliance from a point-in-time gate to a persistent operational state. Monitoring engines ingest decision logs and telemetry, comparing live behavior against policy-as-code rules derived from regulations like the EU AI Act. A breach—such as exceeding a defined disparate impact ratio—generates an immediate alert, triggering the incident response workflow.

The automated statistical process of monitoring for shifts in the distribution of input features compared to the training baseline. This is a leading indicator of potential model degradation. Common detection methods include the Kullback-Leibler divergence and Population Stability Index (PSI). A detected drift does not always require immediate action, but it must trigger an alert for investigation. Unaddressed data drift is a leading cause of silent model failure in production.

A predefined, often automated, operational protocol for reverting a production AI system to a previous stable version. This is the primary remediation action during a post-market incident. The procedure must be atomic and tested regularly via chaos engineering. Key requirements include:

• Backward-compatible model artifacts stored in a registry
• Canary deployment capability to validate the rollback
• State reconciliation to handle decisions made by the faulty model
• Clear RTO (Recovery Time Objective) defined in the SLA