Control Chart

The Center Line represents the process's average or expected performance over time. It is typically calculated as the mean of the plotted statistic (e.g., sample mean, proportion, count).

• For an X-bar chart, the CL is the grand mean of all subgroup means.
• For a P chart, the CL is the average proportion of nonconforming items.
• The CL serves as the baseline against which process shifts are measured.

Control Limits are the statistically calculated boundaries that define the expected range of common cause variation. They are not specification limits or customer tolerances.

• Typically set at ±3 standard deviations from the center line.
• Calculated using formulas specific to the chart type (e.g., UCL = CL + A2 * R-bar for X-bar).
• A point outside these limits is strong statistical evidence of special cause variation, indicating the process is 'out of control'.

This is the quality characteristic being monitored, plotted in time order. The statistic is chosen based on the data type and monitoring goal.

• Variables Data: Measurable quantities (e.g., latency, temperature). Uses charts like X-bar (means) and R (ranges).
• Attributes Data: Counts or classifications (e.g., defect counts, error rates). Uses charts like P (proportion defective) and C (defect count).
• Each point represents a rational subgroup of data collected at a specific time.

Rational Subgrouping is the strategic method for grouping individual measurements into samples. Its purpose is to maximize the chance of detecting variation between subgroups while minimizing variation within subgroups.

• Proper subgrouping isolates special causes to appear between subgroups (affecting the center line) while common cause appears within subgroups (affecting control limits).
• Poor subgrouping can mask special causes, making the process appear more stable than it is.

These are supplementary pattern-detection rules that identify non-random behavior within the control limits, signaling a process shift. The control chart is divided into zones (each 1 sigma wide). Common rules include:

• Rule 1: A single point beyond the 3-sigma control limits.
• Rule 2: Nine points in a row on the same side of the center line.
• Rule 3: Six points in a row, all increasing or all decreasing (a trend).
• Rule 4: Fourteen points in a row, alternating up and down (systematic variation).

The practical utility of a control chart depends on rich contextual metadata. This includes:

• Time Axis: Clearly labeled to correlate points with operational events.
• Annotations: Notes marking known interventions, system changes, or external events (e.g., 'deployment at 14:00', 'data source changed').
• Specification Limits: Often shown as horizontal lines (dashed) to contrast statistical control with customer requirements. A process can be in control but not capable if its natural spread exceeds spec limits.

Control charts are deployed to track key model performance metrics (e.g., accuracy, F1-score, AUC-ROC) over time in production. A sustained shift beyond the control limits signals model degradation or concept drift, triggering retraining or investigation.

• Example: Plotting daily inference accuracy for a fraud detection model.
• Detection: A run of 7 points below the centerline indicates a statistically significant performance drop.

Applied to feature serving data, control charts monitor the statistical properties of input features. Charts for the mean (X-bar chart) and variance (R or S chart) of numerical features can detect covariate shift.

• Example: Monitoring the average transaction amount fed to a credit scoring model.
• Rational Subgrouping: Data is sampled in hourly batches to maximize between-batch variation from drift.

Control charts instrument data pipeline observability, tracking metrics like record counts, null rates, and latency. This applies Statistical Process Control principles to data engineering.

• Key Charts:
  • P Chart: For monitoring the proportion of failed records.
  • Individuals Chart (I-MR): For tracking daily data freshness (latency in hours).
• Goal: Distinguish common cause variation (normal pipeline noise) from special cause variation (a broken extractor).

Control charts provide a statistically rigorous baseline for anomaly detection in operational and IoT data streams. The Upper Control Limit (UCL) and Lower Control Limit (LCL) define the expected range of normal variation.

• Method: Exponentially Weighted Moving Average (EWMA) charts are sensitive to small, persistent shifts in sensor telemetry.
• Advantage: Reduces false positives compared to simple threshold-based alerts.

Used to verify process stability before and after deploying a new model version or feature. Establishing a stable baseline with a control chart ensures any observed performance difference is attributable to the change, not inherent process noise.

• Process Capability Analysis: Determines if the current model process is capable of meeting performance specification limits (e.g., >95% accuracy).
• Application: Confirming a new recommendation algorithm's click-through rate remains in control.

In LLM Operations, control charts can track metrics related to output quality and stability, such as sentiment scores, factual consistency scores, or hallucination rates from automated evaluators.

• Attribute Control Charts: C Charts can monitor the count of flagged hallucinations per 1000 generated responses.
• Benefit: Provides an objective, statistical method for detecting regressions in prompt engineering or model fine-tuning.

Control limits are statistically calculated boundaries on a control chart, typically set at ±3 standard deviations from the process mean. They are derived from the process's own historical data and are used to distinguish between common cause variation (inherent, random noise) and special cause variation (assignable, non-random signals). Unlike specification limits, which are set by customer requirements, control limits indicate whether the process itself is stable and predictable.

Understanding variation is the core purpose of a control chart.

• Common Cause Variation: Inherent, random variation present in any stable process. It is predictable within statistical limits and represents the natural "noise" of the system. Management must change the underlying process to reduce it.
• Special Cause Variation: Non-random, assignable variation that signals a specific, identifiable change in the system (e.g., a machine fault, new raw material batch). It appears as points outside control limits or non-random patterns. This variation should be investigated and eliminated at the operational level.

Process capability analysis measures a process's ability to produce output within specification limits. It compares the natural spread of the process (6σ) to the width of the specification tolerance.

• Cp (Process Capability): Measures the potential capability if the process is perfectly centered. Formula: Cp = (USL - LSL) / (6σ).
• Cpk (Process Capability Index): Measures actual capability, accounting for how centered the process is within the specs. It is the minimum of: (USL - Mean)/(3σ) or (Mean - LSL)/(3σ). A Cpk ≥ 1.33 is generally considered capable.

An Individuals and Moving Range (I-MR) Chart is a pair of control charts used when data is collected as individual observations rather than in subgroups. It is commonly applied in data observability for metrics like pipeline latency or daily record counts.

• I Chart (Individuals): Monitors the process mean over time by plotting each individual measurement.
• MR Chart (Moving Range): Monitors process variability by plotting the absolute difference between consecutive individual measurements. This chart pair is essential for detecting shifts and trends in continuous, real-time data streams.

These are control charts for attribute data (counted, non-numeric data), crucial for monitoring data quality metrics.

• P Chart (Proportion Chart): Tracks the proportion of nonconforming units in a sample of varying size. Example: monitoring the daily percentage of database records that fail a validation rule.
• C Chart (Count Chart): Tracks the count of nonconformities (defects) in a sample of constant size. Example: monitoring the number of schema violations detected per 10,000 records ingested.

The Western Electric Rules are a set of heuristic decision rules for detecting out-of-control conditions on a control chart beyond a single point outside the control limits. They identify non-random patterns that indicate special cause variation. Common rules include:

• Rule 1: A single point outside the 3σ control limits.
• Rule 2: Two out of three consecutive points beyond the 2σ warning limits.
• Rule 3: Four out of five consecutive points beyond the 1σ limits.
• Rule 4: Eight consecutive points on one side of the center line. Automated data observability platforms implement these rules to trigger alerts for anomalous process behavior.