Meta-Learners

The S-Learner fits a single supervised model where the treatment indicator is included as a feature alongside covariates.

• Mechanism: Estimates (\mu(x, w) = E[Y | X=x, W=w]) using one model. The CATE is (\hat{\tau}(x) = \hat{\mu}(x, 1) - \hat{\mu}(x, 0)).
• Strength: Simple to implement with any regression algorithm.
• Weakness: The model may regularize away the treatment effect entirely if the treatment variable is not given sufficient weight, especially in high-dimensional settings.
• Use Case: Baseline estimation when treatment is a dominant signal.

The T-Learner partitions the data by treatment status and fits two separate conditional mean functions.

• Mechanism: Estimates (\mu_1(x) = E[Y(1) | X=x]) on treated units and (\mu_0(x) = E[Y(0) | X=x]) on control units. CATE is (\hat{\tau}(x) = \hat{\mu}_1(x) - \hat{\mu}_0(x)).
• Strength: No risk of the treatment variable being ignored; each model specializes.
• Weakness: Inefficient when one treatment group is small, leading to high variance in the corresponding model.
• Use Case: Randomized controlled trials with balanced groups or when treatment is a strong effect modifier.

The X-Learner leverages information from both treatment groups to impute counterfactuals, making it robust to imbalanced designs.

• Mechanism: Fits (\hat{\mu}_1) and (\hat{\mu}_0) as in the T-Learner. Then imputes treatment effects: (\tilde{D}_i^1 = Y_i(1) - \hat{\mu}_0(x_i)) for treated, and (\tilde{D}_i^0 = \hat{\mu}_1(x_i) - Y_i(0)) for controls. Fits two models (\hat{\tau}_1(x)) and (\hat{\tau}_0(x)) on these imputed effects. Final CATE is a weighted average: (\hat{\tau}(x) = g(x)\hat{\tau}_0(x) + (1-g(x))\hat{\tau}_1(x)).
• Strength: Excels when one group is much larger than the other.
• Weakness: Involves four regression stages, increasing computational cost.
• Use Case: Observational studies with severe treatment imbalance.

The R-Learner frames CATE estimation as a residual-on-residual regression derived from Robinson's partialling-out transformation.

• Mechanism: Defines the outcome residual (Y - \hat{m}(x)) and treatment residual (W - \hat{e}(x)), where (\hat{m}(x) = E[Y|X=x]) and (\hat{e}(x) = P(W=1|X=x)) is the propensity score. The CATE (\hat{\tau}(x)) is estimated by regressing outcome residuals on treatment residuals, weighted by the squared treatment residual.
• Strength: Neyman-orthogonal; the target parameter is insensitive to errors in the nuisance functions (\hat{m}) and (\hat{e}).
• Weakness: Requires accurate propensity score estimation.
• Use Case: Foundation for Double Machine Learning (DML) with high-dimensional covariates.

The DR-Learner combines outcome regression and propensity score weighting to provide a consistent estimator if at least one nuisance model is correctly specified.

• Mechanism: Constructs the doubly robust pseudo-outcome: (\psi = \hat{\mu}_1(x) - \hat{\mu}_0(x) + \frac{W(Y - \hat{\mu}_1(x))}{\hat{e}(x)} - \frac{(1-W)(Y - \hat{\mu}_0(x))}{1-\hat{e}(x)}). A final regression of (\psi) on (X) yields (\hat{\tau}(x)).
• Strength: Double robustness—consistent if either the outcome models (\hat{\mu}_w) or the propensity model (\hat{e}) is correct.
• Weakness: Can have high variance if propensity scores are near 0 or 1.
• Use Case: High-stakes causal inference where model misspecification risk must be minimized.

Causal Forest adapts the random forest algorithm to estimate heterogeneous treatment effects using honest splitting to avoid overfitting.

• Mechanism: Grows trees by partitioning the covariate space to maximize heterogeneity in treatment effects. Each tree uses a splitting sample to determine partitions and an estimation sample to compute leaf-wise treatment effects. The forest averages across trees.
• Strength: Automatically discovers complex, non-linear treatment effect heterogeneity without parametric assumptions.
• Weakness: Requires large sample sizes for stable leaf estimates.
• Use Case: Market segmentation for personalized pricing or policy targeting where subgroups with distinct causal responses exist.

The Single-Learner approach uses one model where the treatment indicator is included as a feature alongside covariates. The model estimates E[Y|T, X], and the treatment effect is calculated as the difference in predictions when T is set to 1 versus 0.

• Strength: Simple to implement with any base learner
• Weakness: The model may ignore the treatment variable entirely if its signal is weak relative to covariates
• Best for: Scenarios with strong treatment effects and low-dimensional data

The Two-Learner approach fits two separate models: one for the treated group and one for the control group. The treatment effect is the difference between the two model predictions for a given set of covariates.

• Strength: Allows different functional forms for treated and control responses
• Weakness: Splits the data, reducing effective sample size per model
• Best for: Large datasets where treatment and control groups have fundamentally different outcome distributions

The Cross-Learner extends the T-Learner by leveraging information from both groups. It imputes counterfactual outcomes by predicting the treatment effect for the control group using the treated model, and vice versa, then combines these with a propensity score-weighted final model.

• Strength: Excels when one group (usually control) is much larger than the other
• Weakness: More complex pipeline with multiple modeling stages
• Best for: Imbalanced treatment assignments common in observational studies

The propensity score e(x) = P(T=1|X=x) is the probability of receiving treatment given observed covariates. It is a central component in meta-learners for inverse probability weighting (IPW) and doubly robust methods.

• Purpose: Reduces the dimensionality of the conditioning set for bias removal
• Key property: Conditioning on the propensity score is sufficient to remove confounding bias under the ignorability assumption
• Estimation: Typically modeled via logistic regression or gradient boosting

A doubly robust estimator combines an outcome regression model (like an S-Learner) with a propensity score model. It provides a consistent estimate of the Average Treatment Effect (ATE) if at least one of the two models is correctly specified.

• Formula: Combines direct outcome modeling with inverse probability weighting
• Advantage: Provides two chances to get the specification right
• Application: Often used as the final stage in X-Learners and R-Learners to improve robustness to model misspecification

An adaptation of random forests designed to estimate heterogeneous treatment effects. It uses a splitting criterion that maximizes the difference in treatment effects between child nodes rather than the difference in outcomes.

• Honest estimation: Uses one subsample to build the tree structure and another to estimate leaf effects
• Output: Produces individual-level treatment effect predictions with confidence intervals
• Relation to meta-learners: Can be viewed as a non-parametric, tree-based alternative that implicitly performs the decomposition meta-learners do explicitly