Change Point Detection

A fundamental algorithm that continuously evaluates the likelihood ratio between two hypotheses—null (no change) and alternative (change occurred)—as each new data point arrives.

• Minimizes expected sample size to reach a decision
• Operates with Type I and Type II error guarantees
• Forms the statistical backbone of many modern drift detectors

Example: Detecting a shift in user authentication failure rates by comparing current failure probability against a historical baseline of 0.1%.

A sequential analysis technique that accumulates deviations from a target mean over time, triggering an alert when the cumulative sum exceeds a predefined threshold.

• Highly sensitive to small, persistent shifts in the mean
• Uses a drift parameter to tune sensitivity
• Computationally lightweight for real-time streaming

Example: Monitoring API latency where a gradual 50ms increase over 10 minutes triggers a compliance alert before breaching SLA thresholds.

A probabilistic framework that maintains a distribution over run lengths—the time since the last change point—and updates beliefs recursively using Bayes' theorem.

• Provides uncertainty quantification around detected changes
• Naturally handles multiple change points
• Supports model selection for different segment types

Example: Detecting regime changes in financial transaction volumes where both the timing and magnitude of shifts carry compliance implications.

An optimal offline algorithm that finds the exact set of change points minimizing a cost function with a linear penalty term, using dynamic programming with pruning.

• Guarantees global optimum under certain cost functions
• Scales linearly with data size through pruning
• Suitable for retrospective audit analysis

Example: Post-hoc analysis of model prediction errors across a 12-month period to identify exact dates where concept drift degraded compliance with fairness thresholds.

A non-parametric method that maps data into a reproducing kernel Hilbert space (RKHS) and detects distributional changes using maximum mean discrepancy (MMD).

• Detects changes in any distributional property, not just mean/variance
• Works on structured data like graphs and text embeddings
• No parametric assumptions about data distribution

Example: Monitoring semantic drift in LLM outputs where the distribution of embedding vectors shifts, indicating potential prompt injection or model hallucination patterns.

Extension of univariate techniques to high-dimensional data using sparse mean shift assumptions or subspace monitoring to isolate which features changed.

• Uses L1 regularization to identify sparse change vectors
• Reduces false alarms from noisy dimensions
• Critical for monitoring multi-metric compliance dashboards

Example: Simultaneously monitoring 200 model features for fairness drift, where only 3 features actually shifted—sparse detection isolates the root cause without drowning operators in alerts.

The Cumulative Sum control chart sequentially accumulates deviations from a target mean to detect small, persistent shifts in a process. Unlike Shewhart charts that only use the last observation, CUSUM integrates historical data, making it highly sensitive to subtle drift.

• One-sided CUSUM: Detects increases or decreases independently
• Decision Interval (H): The threshold that triggers an alert when cumulative sum exceeds it
• Reference Value (K): The allowable slack, typically set to half the shift magnitude you want to detect

Commonly applied to monitor model prediction residuals for early signs of concept drift.

A statistical hypothesis testing framework developed by Abraham Wald that evaluates data as it arrives rather than waiting for a fixed sample size. SPRT continuously computes a likelihood ratio comparing two competing hypotheses and terminates when sufficient evidence accumulates.

• Type I Error (α): Probability of false alarm, detecting a change when none exists
• Type II Error (β): Probability of missed detection, failing to identify a real shift
• Boundaries A and B: Derived from α and β to define the continue-sampling region

SPRT minimizes the average sample number required to reach a decision, making it optimal for real-time compliance monitoring where latency matters.

A probabilistic approach that models the run length — the time elapsed since the most recent change point — as a latent variable updated recursively via Bayes' theorem. This method produces a full posterior distribution over possible change point locations rather than a single point estimate.

• Hazard Function: Encodes prior belief about how frequently changes occur
• Predictive Distribution: Evaluates how well new data fits the current model parameters
• Message-Passing Algorithm: Efficiently propagates run-length probabilities forward in time

Particularly valuable when you need uncertainty quantification around detected shifts for audit documentation.

An optimal partitioning algorithm that finds the exact set of change points minimizing a cost function with a linear penalty term. PELT achieves O(n) computational complexity through dynamic programming with pruning, making it feasible for long time series.

• Cost Function: Typically negative log-likelihood, measuring fit quality within each segment
• Penalty Constant (β): Controls the trade-off between sensitivity and false positives
• Pruning Rule: Discards candidate change points that can never be optimal, dramatically reducing search space

Ideal for offline retrospective analysis of model performance logs to identify all significant regime changes over a historical period.

Operational thresholds that automatically initiate remediation workflows when change point detection signals cross predefined boundaries. These triggers bridge statistical detection with automated governance actions.

• Soft Trigger: Generates an alert and creates a Jira ticket for manual investigation
• Hard Trigger: Activates a circuit breaker, halting inference and rolling back to a shadow model
• Evidence Capture: Automatically snapshots the data window surrounding the detected change point for audit trail immutability

Integrates with Policy-as-Code engines to enforce pre-approved response playbooks without human intervention for known failure modes.

A symmetric metric quantifying the divergence between two distributions by binning data and comparing proportions. While traditionally used for data drift detection, PSI can serve as a univariate change point signal when monitored over time.

• PSI < 0.1: Insignificant shift, no action required
• 0.1 ≤ PSI < 0.25: Moderate shift, warrants investigation
• PSI ≥ 0.25: Significant shift, likely indicates a change point

Commonly paired with Kullback-Leibler Divergence and Jensen-Shannon Distance as complementary distribution comparison methods in continuous compliance dashboards.