Interference Effect

Interference occurs when the potential outcome for one unit depends on the treatment assignment of other units, breaking the independence assumption. This creates a spillover effect where control group users are indirectly exposed to treatment conditions.

• Direct effect: Impact of treatment on the treated unit
• Indirect effect: Impact of one unit's treatment on another unit's outcome
• Total effect: Sum of direct and indirect effects across the network

In ride-sharing platforms, surge pricing applied to one geographic zone inevitably affects driver supply and rider demand in adjacent zones.

When interference is present, traditional randomization at the user level fails. Cluster-based randomization assigns entire network neighborhoods to the same treatment arm to contain spillover effects within clusters.

• Ego-network clustering: Group users by their immediate social connections
• Graph partitioning algorithms: Use community detection to identify naturally isolated subgraphs
• Geographic clustering: Define treatment regions using spatial boundaries

LinkedIn often randomizes at the company or geographic level when testing social features to prevent cross-contamination between connected users.

Marketplaces connecting buyers and sellers are inherently susceptible to interference because both sides compete for the same finite resources. A treatment improving search ranking for one seller directly reduces visibility for others.

• Demand-side interference: Promotions to one buyer segment affect inventory available to others
• Supply-side interference: Incentivizing one set of drivers or hosts alters availability for all users
• Equilibrium effects: Short-term experimental gains may vanish as the marketplace rebalances

Uber's experimentation framework explicitly models these cross-unit dependencies when testing driver incentive programs.

Standard estimators like the difference-in-means become biased when interference is present. Specialized causal frameworks are required to recover valid treatment effect estimates.

• Exposure mapping functions: Define how a unit's outcome depends on the full treatment assignment vector
• Horvitz-Thompson estimators: Weight outcomes by the probability of specific exposure conditions
• Interference graphs: Explicitly model the known pathways through which spillover propagates

These methods allow experimenters to estimate both direct and spillover effects rather than treating interference as mere noise.

Before applying complex interference corrections, experimenters must first detect whether interference is actually present. Several diagnostic approaches exist:

• Varying cluster sizes: Randomize at different aggregation levels and test for divergence in treatment effect estimates
• Distance-based analysis: Check if treatment effects diminish as network or spatial distance from treated units increases
• Placebo tests: Assign sham treatments to control units and verify no outcome changes occur

Failure to detect interference before analysis can lead to overestimated precision and inflated false positive rates.

For time-series settings where units cannot be isolated, switchback experiments alternate treatment assignment over time within the same unit, using each unit as its own control.

• Temporal randomization: Assign treatment in randomized time blocks rather than across units
• Carryover adjustment: Model and subtract the lagged effect of previous treatment periods
• Applicable domains: Dynamic pricing, inventory algorithms, and content recommendation feeds

DoorDash uses switchback designs to test dispatch algorithms, as individual drivers and orders cannot be isolated from each other in real-time.

The foundational causal inference assumption that the Interference Effect directly violates. SUTVA has two components: no interference (one unit's treatment doesn't affect another's outcome) and consistency (the treatment is applied uniformly). In two-sided marketplaces like ride-sharing, a dynamic pricing model applied to one rider can affect the availability and price for a control rider, breaking SUTVA. Validating SUTVA is the first diagnostic step before trusting any A/B test results.

A primary mitigation strategy for interference where the unit of randomization is shifted from the individual to a group. Instead of assigning users randomly, entire clusters—such as geographic regions, corporate accounts, or social network subgraphs—are assigned to treatment or control. This isolates the treatment effect within clusters, preventing spillover. The trade-off is a significant reduction in statistical power, requiring a larger sample size and a longer experiment duration to detect the same effect size.

The economic structure most susceptible to interference, where the platform mediates between distinct user sides (e.g., riders and drivers, buyers and sellers). A treatment that increases demand on one side (e.g., a buyer discount) can cannibalize supply from the control group, creating a negative spillover. Conversely, a treatment that improves supply efficiency can benefit the control group, causing a positive dilution of the measured effect. Understanding these cross-side network effects is critical for experimental design.

Advanced econometric techniques used to model and correct for interference post-hoc when cluster randomization is infeasible. Methods include exposure mapping, which models a unit's exposure to the treatment based on the treatment status of its neighbors, and graph neural network-based estimators that learn the interference structure directly from the network topology. These approaches attempt to estimate what the control group's outcome would have been in the absence of spillover.

A diagnostic technique to detect interference in advertising experiments. A ghost ad is a simulated auction bid that is logged but never actually served. By comparing the conversion behavior of users who organically would have seen a treatment ad against a pure control group, experimenters can quantify the direct causal effect without the confounding influence of auction dynamics. A persistent divergence between ghost ad estimates and standard A/B results is a strong indicator of interference.

A specialized randomization technique for social network experiments where users are clustered based on their social graph connectivity. Algorithms like normalized cut or label propagation are used to partition the graph into isolated buckets with minimal cross-edges. Treatment is then applied at the bucket level. This directly addresses the interference caused by social contagion, where a user's behavior is influenced by the treatment status of their friends, ensuring a clean causal measurement.