Position Bias

The foundational model for understanding position bias. It posits that a user clicking on a document requires two independent events: the user examining the document's position and the document being relevant to the user's query. A click occurs only if both conditions are met. If a document is not examined, it cannot be clicked, regardless of its perfect relevance. This hypothesis mathematically separates the probability of a click into P(Click = 1 | Position = p, Relevance = r) = P(Examination = 1 | Position = p) * P(Relevance = r).

A specific and widely-used instantiation of the examination hypothesis. This model assumes users scan results in a strict top-down linear order. A user starts at position 1 and examines each result sequentially. They click on the first relevant document they encounter and then abandon the search session entirely. This model implies that the probability of examination for a position is the product of the probabilities that all preceding items were non-relevant. It starkly illustrates how a highly relevant item at position 10 might never be seen if a moderately relevant item is at position 1.

A core debiasing technique that counters position bias by assigning an inverse propensity weight to each observed click. The propensity is the probability of a user examining a given position. A click on a low-propensity item (e.g., at position 20) provides a much stronger relevance signal than a click on a high-propensity item (e.g., at position 1). The model's loss function is re-weighted as:

• Loss = sum( (1 / Propensity(p)) * L(click, prediction) ) This process effectively down-weights clicks from prominent positions and up-weights clicks from obscure ones, creating an unbiased learning objective.

The gold-standard method for empirically measuring a system's position bias curve. This involves conducting a controlled experiment where the search engine's standard ranking algorithm is intentionally bypassed for a small fraction of traffic. Results are shuffled and displayed in a random order. By observing click-through rates for identical items placed at different positions, one can directly estimate the examination probability for each rank without the confounding factor of relevance. This generates a clean propensity curve, typically showing a sharp exponential decay from position 1.

A dangerous dynamic where position bias creates a self-fulfilling prophecy. An item that gets a slightly higher initial rank due to noise will receive more clicks. An unbiased model interprets these clicks as a signal of higher relevance, promoting the item further. This creates a positive feedback loop where a potentially mediocre item dominates a top position, while a truly superior item languishes unseen on page two. This effect entrenches popularity and stifles discovery, requiring explicit countermeasures like exploration strategies or propensity-weighting to break the cycle.

A complex, compounding bias where users are more likely to click on top-ranked items not just because they are examined first, but because they trust the system's judgment. The user assumes the ranking algorithm is correct and therefore the top result must be the most authoritative. This is distinct from pure position bias (which is about visual scanning) as it involves a cognitive judgment. Debiasing models must disentangle the effect of the rank itself from the user's learned trust in the system's competence, often requiring models that separate presentation effect from content relevance.

A counterfactual evaluation technique used to debias click data by re-weighting observed interactions based on their probability of being seen. In position bias, IPS assigns higher importance to clicks on items in lower positions, as they were less likely to occur by chance. This creates an unbiased estimator for ranking quality, though it suffers from high variance when propensity scores are small.

The foundational causal model stating that a user clicking on a result requires two independent events: relevance of the item and examination of the position. Formally, P(Click=1 | Position=k, Relevance=r) = P(Examination=1 | Position=k) × P(Relevance=r). This assumes users only click on an item if they both see it and find it relevant, enabling the separation of position effects from true relevance.

A user behavior model where a user scans results sequentially from top to bottom and makes a decision at each position. The user clicks with probability determined by relevance, and abandons the session with a certain probability after each result. This model directly explains position bias: top items are always examined, while lower items are only examined if preceding items are skipped.

The process of estimating the probability of examination for each position in a ranking. Common methods include:

• Randomization tests: Swapping top and bottom results to measure pure position effect
• Result randomization: Placing random items in specific positions to observe click-through rates
• EM-based inference: Using expectation-maximization on click logs to jointly estimate relevance and examination probabilities

A self-reinforcing cycle where a ranking model trained on biased click data learns to place items with historically high CTR at the top. These items continue receiving high clicks due to position, not relevance, and the model incorrectly reinforces their top placement. Breaking this loop requires explicit position bias modeling or counterfactual learning during training.

A related cognitive distortion where users exhibit higher click-through rates on items from sources they perceive as authoritative, regardless of the specific result's relevance. Unlike pure position bias, trust bias is driven by brand perception and domain credibility. It often compounds with position bias, as trusted sources frequently occupy top organic positions.