Concept Drift

A change in the statistical properties of the input features (P(X)) without a change in the relationship to the target.

• Example: A hospital system updates its EHR, changing the default template for discharge summaries. The vocabulary and structure of the notes shift.
• The definition of an adverse event remains the same, but the model sees unfamiliar linguistic patterns.
• Often a precursor to concept drift; a model retrained on the new feature distribution may recover performance.

A change in the posterior probability P(Y|X) — the fundamental relationship between the input and the target variable.

• Example: Post-market surveillance reveals a new, previously unknown adverse event. The clinical definition of what constitutes a 'serious cardiac event' for the drug is updated by regulators.
• The model's learned mapping from text to event codes is now objectively wrong.
• This requires relabeling of ground truth data and full model retraining; simple recalibration is insufficient.

An abrupt, discontinuous shift in the data-generating process, often triggered by an external regulatory or organizational event.

• Example: The FDA issues a new Drug Safety Communication that changes the expectedness criteria for a specific adverse event. Reporting patterns shift overnight.
• Example: A merger between two large hospital networks forces the unification of two different coding standards (e.g., ICD-9 to ICD-10 mapping).
• Detection relies on real-time monitoring dashboards with tight statistical process control thresholds.

A gradual, continuous evolution of the target concept over an extended period, often imperceptible in short time windows.

• Example: Clinical documentation practices slowly evolve as new medical literature influences physician language for describing a condition.
• Example: The demographic profile of a patient population shifts slowly, altering the baseline prevalence of certain comorbidities.
• Mitigation requires online learning or periodic retraining cycles with a sliding window of recent data to forget obsolete patterns.

Cyclical or seasonal patterns where the concept changes predictably and reverts to a previous state.

• Example: During flu season, the co-occurrence of respiratory symptoms with a target drug increases, temporarily altering the signal-to-noise ratio for respiratory adverse events.
• Example: Annual regulatory reporting deadlines cause a spike in ICSR submissions, temporarily changing the statistical properties of the FAERS data stream.
• Can be addressed with time-aware models that incorporate seasonal features or by maintaining separate models for distinct temporal regimes.

A self-reinforcing form of drift where the model's own predictions influence the future labels it is trying to predict.

• Example: An AI-driven triage system flags certain case narratives for expedited human review. Reviewers, influenced by the AI's flag, apply a lower threshold for causality assessment on those cases.
• The model learns from this biased human feedback, amplifying its initial propensity over time.
• Requires causal monitoring and the injection of randomized control cases to break the feedback cycle and measure true model performance.

Data drift (covariate shift) occurs when the input distribution P(X) changes—for example, a hospital system updates its EHR software, altering the linguistic style of clinical notes. Concept drift is a change in the conditional distribution P(Y|X)—the same note now means something different. A report of 'fatigue' might shift from a mild symptom to a critical immune-related adverse event as medical knowledge evolves. Monitoring only input features misses this critical distinction.

Drift manifests in distinct temporal patterns:

• Sudden drift: An abrupt change, such as a regulatory body issuing new diagnostic criteria for a syndrome overnight, instantly reclassifying historical cases.
• Gradual drift: A slow evolution, like a drug's safety profile maturing over years as rare, long-latency events emerge.
• Recurring drift: Cyclical patterns tied to seasonal illness reporting or periodic literature publication cycles. Detection strategies must be tuned to the expected drift velocity.

A symmetric metric quantifying the divergence between a baseline (development) distribution and a production distribution. In pharmacovigilance, PSI is calculated across binned model prediction scores or input feature categories. A PSI < 0.1 indicates no significant shift; 0.1–0.25 suggests moderate drift requiring investigation; > 0.25 signals a major distributional break. PSI is a primary early-warning trigger for automated model retraining pipelines.

A practical technique to directly test for drift. A classifier is trained to distinguish between samples from the training set and the current production data. If the classifier achieves high accuracy (AUC > 0.7), significant feature-space drift is present. The most discriminative features identified by this model pinpoint the exact clinical variables—such as new drug names or lab codes—that have shifted, guiding targeted feature engineering.

A mitigation strategy where the model incrementally updates its parameters with each new labeled ICSR or clinical note. This is distinct from batch retraining. Key challenges include setting an appropriate learning rate decay to prevent catastrophic forgetting of rare, critical safety signals, and implementing a rollback mechanism to revert model weights if a sudden drift event is later identified as a data pipeline error rather than a true semantic shift.

A pharmacovigilance-specific drift trigger. When a drug's prescribing label or Investigator's Brochure is updated, the definition of expectedness changes. An event previously classified as 'unexpected' and reportable may become 'expected' and non-reportable. This is a pure form of concept drift where P(Y|X) changes instantaneously by regulatory fiat, requiring an immediate model update to align with the new RSI.