Fair Ranking

Measures and calibrates the visibility each item receives in a ranked list. Unlike pointwise accuracy metrics, exposure fairness accounts for position bias—the empirical reality that users click items at higher ranks far more often, regardless of relevance.

• Discounted Cumulative Gain (DCG) variants weight exposure by position
• Ensures items from protected groups receive visibility proportional to their merit
• Addresses the "rich-get-richer" feedback loop in recommendation systems

A multi-stakeholder framework that simultaneously optimizes for consumer utility (relevance, satisfaction) and producer equity (fair exposure, opportunity). This is critical in marketplace platforms where maximizing only one side harms the ecosystem.

• Balances ranking quality loss against fairness gain via Pareto-optimal trade-offs
• Uses constrained optimization: maximize relevance subject to fairness constraints
• Applies to gig economy matching, app store rankings, and product search

A post-processing intervention that re-orders an initial relevance-ranked list to satisfy fairness criteria without ignoring item quality. The system starts with a utility-maximizing ranking and applies controlled perturbations.

• Fairness-aware re-ranking algorithms like FA*IR guarantee minimum exposure for protected groups
• Uses integer linear programming or greedy heuristics for real-time constraints
• Preserves the principle that more relevant items should generally outrank less relevant ones

Two competing philosophical approaches to defining equitable rankings:

• Group fairness: Ensures statistical parity across demographic groups (e.g., equal exposure for items from male and female sellers). Measured via metrics like Skew@k.
• Individual fairness: Requires that similar items receive similar treatment. A ranking is individually fair if swapping two items with nearly identical relevance scores does not significantly change their positions.
• Modern systems often blend both, applying group constraints with individual smoothing

A temporal fairness paradigm that ensures equity over a sliding window of queries rather than enforcing strict constraints on every single ranking. This allows short-term flexibility while guaranteeing long-term non-discrimination.

• Tracks cumulative exposure debt for each item or group
• When a protected group falls below its fair share, the system compensates in subsequent rankings
• Reduces the utility cost of fairness by smoothing constraints over time
• Critical for high-velocity systems like news feeds and dynamic product listings

A framework for multi-stakeholder platforms that simultaneously optimizes for equitable outcomes on both sides of a marketplace. For ranking, this means balancing fair exposure for producers (sellers, creators) with fair, relevant recommendations for consumers.

• Producer-side: ensuring new sellers get visibility
• Consumer-side: maintaining recommendation quality
• Requires explicit multi-objective optimization

A fairness criterion requiring a classifier to have equal true positive and false positive rates across different protected groups. Applied to ranking, it ensures that the system's errors—showing irrelevant items or hiding relevant ones—are evenly distributed.

• Focuses on error parity, not outcome parity
• More nuanced than demographic parity
• Can be enforced as a post-processing constraint on ranked lists

The ability to provide a clear, actionable path for individuals to reverse an unfavorable algorithmic decision. In fair ranking, this means telling a content creator or seller exactly what changes would improve their position.

• Example: "Increase your shipping speed to 2-day to qualify for top-10 placement"
• Shifts focus from passive fairness to empowerment
• Requires interpretable ranking features

An ethical framework focused on the fair allocation of outcomes, benefits, and resources among different members of a population. In ranking, this translates to how exposure, clicks, and revenue are distributed across item providers.

• Draws from Rawlsian and utilitarian philosophy
• Asks: Who benefits from this ranking?
• Informs exposure-aware loss functions

A phenomenon where a biased ranking model's predictions influence future user behavior, generating new training data that reinforces and amplifies the original bias. This creates a self-perpetuating cycle of inequity.

• Popular items get more clicks → more data → better ranking → more clicks
• Niche items starve for data
• Breaking the loop requires explicit exploration or fairness interventions