Causal Impact

The core mechanism involves building a synthetic control—a weighted combination of predictor time series that were not affected by the intervention. The model uses a Bayesian structural time-series framework to capture trend, seasonality, and covariate relationships during the pre-intervention period. This synthetic baseline represents what would have happened absent the intervention, enabling pointwise causal effect estimation as the difference between observed and predicted values.

Unlike frequentist approaches that yield point estimates, Causal Impact generates full posterior distributions over the counterfactual. This provides credible intervals for the causal effect at every time point. Key outputs include:

• Pointwise effect: The difference between actual and predicted at each time t
• Cumulative effect: The summed impact over the entire post-intervention window
• Posterior tail-area probability: A Bayesian analog to the p-value, indicating the likelihood of observing the effect by chance

The validity of the synthetic control hinges on a critical assumption: the predictor time series must not be causally affected by the intervention itself. If the intervention influences both the target metric and the covariates used to build the counterfactual, the model will absorb the treatment effect into the baseline, yielding biased estimates. Common valid predictors include:

• Macroeconomic indicators
• Untreated geographic regions
• Competitor performance metrics
• Seasonally correlated but causally independent series

The underlying model decomposes the time series into interpretable structural components:

• Local linear trend: Captures evolving level and slope over time
• Seasonality: Handles weekly, monthly, or custom periodic patterns using Fourier terms
• Regression component: Incorporates covariate time series with static or time-varying coefficients via spike-and-slab priors for automatic variable selection This decomposition allows practitioners to diagnose whether an apparent effect is genuinely causal or an artifact of unmodeled seasonality.

In production ML systems, Causal Impact is particularly valuable when a global model update is deployed to all users simultaneously, making a randomized holdout group infeasible. By using untreated business metrics or external market data as predictors, teams can estimate the incremental impact of a new recommendation algorithm or pricing model on revenue, conversion, or engagement without maintaining a long-term control group.

Practitioners must validate model adequacy through posterior predictive checks:

• One-step-ahead prediction errors: Should exhibit no autocorrelation in the pre-period
• Cumulative absolute error: Should remain stable before the intervention; a pre-trend indicates poor fit
• Sensitivity to predictor set: Results can shift with covariate selection; robustness checks with alternative predictor combinations are essential The method also assumes the relationship between predictors and the target remains stationary across the pre- and post-intervention periods.

The underlying state-space model used by Causal Impact to flexibly decompose a time series into components of interest. It models the observed data as a combination of trend, seasonality, and regression components with spike-and-slab priors for variable selection.

• Handles missing data natively through Kalman filtering
• Spike-and-slab priors automatically select relevant control series
• Provides full posterior distributions over the causal effect, not just point estimates

The process of predicting what would have occurred in the absence of an intervention. In Causal Impact, this is achieved by training a model on pre-intervention data where the outcome and control series are observed together, then forecasting the outcome into the post-intervention period using only the controls.

• The causal effect is the difference between actual and counterfactual
• Assumes the relationship between outcome and controls remains stable
• Critically relies on controls not being affected by the intervention themselves

A classical causal inference technique that compares the change in outcome over time for a treated group against the change for an untreated control group. While simpler than Causal Impact, it relies on the strong parallel trends assumption—that the two groups would have followed similar trajectories absent treatment.

• Estimates the Average Treatment Effect on the Treated (ATT)
• Requires at least one pre-intervention and one post-intervention observation
• Less flexible than synthetic controls when multiple untreated units are available

A quasi-experimental design that analyzes a single time series before and after an intervention to detect a change in level (immediate shift) or slope (trend change). Unlike Causal Impact, it does not use a control group, making it vulnerable to confounding from co-occurring events.

• Models the outcome as a function of time and the intervention indicator
• Autocorrelation must be accounted for using ARIMA or Newey-West standard errors
• Weaker than methods incorporating control series for causal attribution

A technique for creating a synthetic control group in observational studies by matching treated units to untreated units with similar propensity scores—the estimated probability of receiving treatment given observed covariates. Used to reduce selection bias when randomization is impossible.

• Balances observed confounders between treatment and control groups
• Does not account for unobserved confounders
• Often paired with Difference-in-Differences for stronger causal claims