Difference-in-Differences (DiD)

The foundational identification assumption in DiD. It posits that in the absence of treatment, the average outcome trajectories for the treated and control groups would have followed parallel paths over time.

• Key Implication: Any divergence in outcomes post-treatment is attributable to the intervention.
• Validation: Researchers often examine pre-treatment periods visually and statistically to assess plausibility.
• Violation: If trends are not parallel, the DiD estimator will be biased, potentially overstating or understating the true treatment effect.

The simplest DiD setup involves two groups (treated and control) observed over two time periods (pre-treatment and post-treatment). The causal effect is calculated as a double difference:

1. Calculate the change in outcome for the treated group: (Post_T - Pre_T)
2. Calculate the change in outcome for the control group: (Post_C - Pre_C)
3. The DiD estimator is the difference between these changes: (Post_T - Pre_T) - (Post_C - Pre_C)

This differencing removes bias from time-invariant unobserved confounders and common time trends.

The modern implementation of DiD typically uses a two-way fixed effects (TWFE) regression model. The specification is:

Y_it = α_i + γ_t + δ * D_it + ε_it

• α_i: Unit fixed effects, controlling for all time-invariant characteristics of each individual or group.
• γ_t: Time fixed effects, controlling for temporal shocks common to all units.
• D_it: The treatment indicator (1 if unit i is treated in period t, 0 otherwise).
• δ: The coefficient of interest, representing the Average Treatment Effect on the Treated (ATT).

A dynamic DiD specification that estimates treatment effects for each period relative to the treatment event. This is the standard for testing parallel trends and visualizing treatment effect dynamics.

The model includes leads and lags of treatment: Y_it = α_i + γ_t + Σ β_k * D^k_it + ε_it

• Leads (k < 0): Pre-treatment coefficients should be statistically indistinguishable from zero, validating the parallel trends assumption.
• Lags (k ≥ 0): Post-treatment coefficients trace out the dynamic causal effect over time, revealing whether effects are immediate, delayed, or growing.

DiD is a powerful tool for causal inference in biomedical research when randomized controlled trials are infeasible. Common applications include:

• Health Policy Evaluation: Estimating the effect of a new drug formulary restriction by comparing a state implementing the policy to a neighboring state that did not.
• Hospital Intervention Studies: Assessing the impact of a new surgical protocol by comparing patient outcomes in a hospital that adopted it versus a control hospital, before and after adoption.
• Natural Experiments: Leveraging exogenous shocks, like a sudden regulatory change for a specific biomarker test, to identify causal effects on diagnostic accuracy or patient survival.

DiD is the gold-standard quasi-experimental design for evaluating the causal impact of health policy changes on population-level outcomes.

• Medicaid expansion: Compare changes in uninsured rates between states that expanded Medicaid (treated) and those that did not (control), before and after the policy change.
• Smoking bans: Estimate the effect of indoor smoking prohibitions on hospital admissions for acute myocardial infarction by comparing jurisdictions with and without bans.
• Vaccine mandates: Assess the impact of school-entry vaccination requirements on disease incidence using states without mandates as controls.

The key identifying assumption is parallel trends: in the absence of the policy, the treated and control groups would have followed similar outcome trajectories.

When randomized controlled trials are unethical or impractical, DiD provides a robust framework for estimating drug effectiveness using observational data.

• New drug adoption: Compare changes in HbA1c levels among patients switched to a new diabetes medication (treated) versus those remaining on standard therapy (control), before and after the switch.
• Concurrent control: The control group accounts for secular trends in disease progression that would otherwise confound pre-post comparisons.
• Staggered adoption: Modern DiD estimators handle settings where patients initiate treatment at different times, avoiding biases from two-way fixed effects models.

DiD isolates the treatment effect by differencing out both time-invariant unobserved confounding and common time trends affecting both groups.

DiD is ideally suited for evaluating clinical workflow interventions implemented at the hospital or clinic level, where individual randomization is logistically impossible.

• Electronic health record (EHR) alerts: Compare changes in guideline-concordant prescribing between clinics that deployed a new clinical decision support alert (treated) and those that did not (control).
• Telemedicine adoption: Estimate the effect of telehealth implementation on no-show rates by comparing practices that adopted telemedicine platforms to those maintaining in-person-only visits.
• Nurse staffing ratios: Assess the impact of mandated minimum nurse-to-patient ratios on patient mortality using hospitals in states without mandates as controls.

The unit of analysis is typically the clinic, hospital, or geographic region, with patient-level outcomes aggregated accordingly.

The event study extension of DiD is critical for biomedical applications, allowing researchers to test the parallel trends assumption and examine dynamic treatment effects over time.

• Pre-trend testing: Estimate leads of the treatment indicator to verify that treated and control groups followed parallel outcome trajectories before the intervention.
• Dynamic effects: Estimate lags to assess whether treatment effects grow, diminish, or stabilize over time—essential for understanding drug tolerance or policy phase-in.
• Reference period: One pre-treatment period is omitted as the reference category; coefficients on pre-periods should be statistically indistinguishable from zero.

Violation of parallel pre-trends suggests time-varying confounding and invalidates the standard DiD estimator.

When a single control group is insufficient, synthetic control methods extend DiD by constructing a weighted combination of multiple untreated units that best approximates the treated unit's pre-intervention trajectory.

• Single treated unit: Ideal for evaluating a policy implemented in one state or a drug approved in one country, where no single untreated unit provides a valid counterfactual.
• Weight optimization: Weights are chosen to minimize the pre-intervention root mean squared prediction error between the treated unit and the synthetic control.
• Placebo tests: Apply the synthetic control method to untreated units to assess whether the estimated effect for the treated unit is unusually large relative to the placebo distribution.

This method is increasingly used in pharmacoepidemiology and health services research for comparative effectiveness studies.

Triple differences (difference-in-difference-in-differences) adds a third dimension to address confounding when parallel trends may not hold in the standard DiD framework.

• Additional control group: Introduce a third difference—such as a population subgroup unaffected by the policy within both treated and untreated regions—to net out subgroup-specific time trends.
• Example: To estimate the effect of a cancer screening program, compare changes in screening rates (before vs. after) between regions with and without the program, and between age groups eligible and ineligible for screening.
• Bias reduction: DDD removes both region-specific time trends and demographic-specific time trends that could bias the standard DiD estimator.

DDD is particularly valuable in biomedical contexts where treatment eligibility is determined by multiple intersecting criteria.

A statistical technique for estimating causal effects when controlled experiments are infeasible. It uses a third variable—the instrument—to account for unobserved confounding.

• Core conditions: The instrument must be associated with the treatment, affect the outcome only through the treatment (exclusion restriction), and share no common causes with the outcome.
• Biomedical application: Mendelian Randomization is the canonical example, using genetic variants as instruments to assess if a biomarker causes a disease.
• Key contrast with DiD: IV analysis leverages a variable to induce exogenous variation, while DiD leverages temporal variation and a parallel trends assumption.

An extension of DiD that constructs a data-driven counterfactual by weighting untreated units to create a synthetic control that closely tracks the treated unit's pre-intervention trajectory.

• Advantage over DiD: Does not rely on a simple average of controls; instead optimizes weights to minimize pre-treatment prediction error.
• Biomedical use case: Evaluating the impact of a regional health policy when only one state or hospital system implements it, using a weighted combination of other regions as the counterfactual.
• Formalization: Minimizes the root mean squared prediction error (RMSPE) in the pre-treatment period.

A quasi-experimental design that exploits a cutoff or threshold in a continuous running variable to assign treatment, comparing outcomes just above and just below the cutoff.

• Sharp RDD: Treatment is deterministically assigned at the cutoff (e.g., patients with BMI ≥ 30 receive an intervention).
• Fuzzy RDD: The probability of treatment jumps at the cutoff but is not deterministic, requiring an instrumental variable approach.
• Key assumption: Units cannot precisely manipulate the running variable to sort themselves around the cutoff. Local randomization is assumed within a narrow bandwidth.

A framework for designing observational studies by explicitly specifying the protocol of a hypothetical randomized trial that would answer the causal question, then emulating it with observational data.

• Steps: Define eligibility criteria, treatment strategies, assignment procedures, follow-up period, outcomes, causal contrasts, and analysis plan—all before looking at the data.
• Biomedical relevance: Increasingly used in pharmacoepidemiology to estimate drug effectiveness using electronic health records and claims data.
• Relationship to DiD: DiD can serve as the analytical engine within a target trial emulation when a suitable control group and pre-post data structure exist.