Inverse Propensity Scoring (IPS)

IPS corrects for selection bias by re-weighting each observed outcome by the inverse of its probability under the logging policy. If a logging policy chose action a with probability p, the outcome is weighted by 1/p. This creates an unbiased estimator of the target policy's value, as rare actions receive higher weights to compensate for their under-representation in the data. The fundamental insight is that the expected value of the IPS estimator equals the true value of the evaluation policy.

IPS critically depends on knowing the exact probability that the logging policy assigned to each action. This is known as the propensity score. Key requirements:

• The logging policy must be fully probabilistic, not deterministic
• Propensities must be logged alongside every decision
• The common support assumption must hold: any action the target policy might take must have a non-zero probability under the logging policy
• Violations of common support lead to biased estimates and undefined weights

The primary weakness of standard IPS is extremely high variance, especially when propensity scores are small. If a logging policy assigns a probability of 0.001 to an action, the weight becomes 1000x, causing individual samples to dominate the estimate. This leads to:

• Unstable and unreliable estimates from limited data
• Sensitivity to outliers in the weighted distribution
• Poor convergence rates in practice Mitigation strategies include weight clipping, self-normalization, and doubly robust estimation.

A practical variant that divides the weighted sum of rewards by the sum of the weights themselves, rather than by the total number of samples. This self-normalization introduces a small bias but dramatically reduces variance, often making it the preferred estimator in practice. The SNIPS estimator is equivalent to a weighted average of observed outcomes, which bounds the estimate within the observed reward range and prevents the extreme outliers that plague the standard IPS formulation.

Weight clipping caps the maximum importance weight at a threshold M, replacing any weight exceeding M with M. This trades a small amount of bias for a significant reduction in variance. Common practices:

• Clip weights at values like 10, 50, or 100
• The bias introduced is bounded and quantifiable
• Clipping is often combined with self-normalization
• The optimal threshold can be selected via cross-validation on a held-out logging dataset

IPS forms one half of the Doubly Robust (DR) estimator, which combines IPS with a direct outcome model. The DR estimator uses the direct model as a baseline and applies IPS only to the residual error. This provides a crucial guarantee: the estimator remains unbiased if either the propensity model OR the outcome model is correctly specified. This double protection makes DR significantly more reliable than pure IPS in real-world applications where perfect propensity logging or outcome modeling is rarely achievable.

Before deploying a new deep learning recommender, IPS estimates its potential Click-Through Rate (CTR) using historical logs from the current production model. By re-weighting clicks by the inverse probability of the logging policy showing that item, data scientists can rank-order candidate models and discard underperformers without exposing a single real user to a bad experience. This is the standard gatekeeper for modern Netflix-style recommendation pipelines.

E-commerce platforms use IPS to evaluate a new surge pricing or discount targeting policy against historical transaction logs. The logging policy (e.g., fixed pricing) had a low probability of offering the deep discount the new policy proposes. IPS correctly up-weights those rare historical events to provide an unbiased estimate of revenue lift, preventing the deployment of strategies that would inadvertently cannibalize margins on price-insensitive customers.

In clinical decision support, running a randomized trial on a new treatment recommendation algorithm can be unethical or impractical. IPS evaluates the new policy using Electronic Health Records (EHR) generated by doctors' historical decisions. By correcting for the confounding by indication—sicker patients were more likely to get the treatment—IPS provides a statistically sound estimate of the new algorithm's potential impact on patient outcomes before any clinical deployment.