Post-Market Monitoring

The automated capture of all AI system inputs, inferences, and outcomes in a production environment. This is distinct from pre-deployment evaluation data and forms the basis for detecting concept drift and model degradation. Logs must capture the raw input, the model's output, any confidence scores, and the version of the model artifact that served the request. This data serves as the primary evidence for an audit trail.

A statistical subsystem that continuously compares the distribution of live production data against the baseline training data. It triggers an alert when a significant divergence is detected. Key metrics include:

• Data Drift: A change in the input feature distribution (e.g., a sudden shift in user demographics).
• Concept Drift: A change in the relationship between the inputs and the target variable (e.g., a fraud pattern that evolves to evade the model).
• Prediction Drift: A change in the model's output distribution, often a symptom of the first two.

A formal, documented procedure for reporting serious incidents or malfunctions to market surveillance authorities. Under the EU AI Act, a 'serious incident' is any event that directly or indirectly leads to death, serious harm to health or property, or a serious disruption of critical infrastructure. The pipeline must define clear escalation paths, root-cause analysis protocols, and strict reporting timelines. This is a core component of the AI Incident Response framework.

A technical mechanism that captures end-user feedback and operational outcomes to create a virtuous cycle of improvement. This includes:

• Explicit Feedback: A 'thumbs down' or 'report' button that captures a user's direct dissatisfaction.
• Implicit Feedback: Observing that a user immediately copied and pasted an AI-generated text into a different tool, signaling a failure.
• Outcome Feedback: A downstream system confirming the AI's prediction was correct (e.g., a predicted fraudulent transaction was confirmed as fraud by a human analyst).

The ongoing verification that a high-risk AI system continues to meet its original compliance requirements after any modification or retraining. A substantial modification—such as a change to the model's intended purpose or a parameter-efficient fine-tuning update—may trigger a new conformity assessment. This process relies on automated policy-as-code checks to ensure that updated models do not violate pre-defined fairness or robustness thresholds before deployment.

The generation and secure storage of tamper-proof logs that provide a complete, chronological record of the monitoring system's own operation. This is a meta-requirement proving that monitoring was active and effective. Techniques include cryptographic hashing of log entries and append-only storage architectures. These records are critical for demonstrating compliance to notified bodies and for fulfilling the right to explanation under GDPR.

The level of risk that remains after all planned risk mitigation measures have been implemented in an AI system. Post-market monitoring is the primary control for managing residual risk, as it detects harms that pre-deployment testing failed to eliminate.

• Must be explicitly disclosed in EU AI Act conformity assessments
• Informs the frequency and intensity of monitoring activities
• If residual risk is deemed unacceptable, the system may require human-in-the-loop oversight indefinitely