False Discovery Rate (FDR)

The False Discovery Rate (FDR) is formally defined as the expected proportion of false positives among all rejected null hypotheses. In the context of pathway enrichment analysis, it estimates the fraction of declared 'significant' pathways that are actually null.

• E(V/R | R > 0): The mathematical expectation, where V is false positives and R is total rejections.
• Contrasts with Family-Wise Error Rate (FWER), which controls the probability of any false positive.
• FDR offers greater statistical power than FWER when testing thousands of gene sets, making it the standard for genomic studies.

The Benjamini-Hochberg (BH) procedure is the most widely implemented method for controlling FDR at a desired level α (typically 0.05 or 0.10). It is a step-up procedure that operates on ordered p-values.

• Algorithm Steps:
  1. Order m p-values: P(1) ≤ P(2) ≤ ... ≤ P(m)
  2. Find the largest k such that P(k) ≤ (k/m) * α
  3. Reject all null hypotheses for i = 1, ..., k
• Assumes independence or positive regression dependency among test statistics.
• The Benjamini-Yekutieli variant relaxes the dependency assumption for arbitrary correlation structures.

The q-value, introduced by John Storey, is the minimum FDR at which a particular test may be called significant. It provides a measure of significance in terms of the FDR for each individual hypothesis.

• A q-value of 0.05 implies that 5% of significant results with this score or lower are expected to be false positives.
• Unlike the BH procedure's global cutoff, q-values allow feature-level significance ranking.
• Calculated using an estimate of π₀, the proportion of true null hypotheses among all tests, which captures the overall signal in the dataset.

In Over-Representation Analysis (ORA) and Gene Set Enrichment Analysis (GSEA), FDR correction is applied to the raw enrichment p-values generated for hundreds or thousands of tested gene sets.

• ORA: Corrects p-values from the hypergeometric distribution or Fisher's exact test.
• GSEA: Corrects nominal p-values derived from phenotype permutation testing.
• A typical threshold of FDR < 0.25 is often accepted in exploratory GSEA, while FDR < 0.05 is standard for ORA.
• Failure to apply FDR correction leads to unacceptably high numbers of spurious pathway associations.

The positive FDR (pFDR) is defined as E(V/R | R > 0), conditioning on at least one rejection occurring. Its estimation relies critically on π₀, the proportion of true null hypotheses.

• Storey's method estimates π₀ from the flat tail of the p-value histogram, where p-values are uniformly distributed under the null.
• A low π₀ (e.g., 0.4) indicates a high signal dataset with many true effects, while π₀ near 1.0 suggests sparse signal.
• This estimation provides a more adaptive and powerful FDR control than the fixed BH procedure.

The choice between FDR and Family-Wise Error Rate (FWER) control represents a fundamental trade-off between discovery and certainty.

• FWER (e.g., Bonferroni correction): Controls the probability of one or more false positives. Extremely stringent; ideal when a single false positive is catastrophic.
• FDR: Controls the proportion of false positives. Tolerates a few errors to gain substantial power; ideal for hypothesis generation and screening.
• In biomarker identification, FDR is preferred because follow-up validation experiments will filter initial screening hits, making a controlled proportion of false leads acceptable.

The probability of making one or more Type I errors across an entire family of hypothesis tests. Controls the strictest criterion—ensuring even a single false positive is unlikely. The Bonferroni correction is the classic FWER-controlling procedure: it divides the significance threshold α by the number of tests m, setting the per-test cutoff to α/m. While robust, FWER control is often too conservative for genomics, where thousands of tests are performed and some false positives are tolerable in exchange for greater statistical power to detect true signals.

The foundational step-up algorithm for controlling FDR at a desired level q (typically 0.05). The procedure:

• Ranks all m p-values from smallest to largest: p₁ ≤ p₂ ≤ ... ≤ pₘ
• Finds the largest rank k where pₖ ≤ (k/m) × q
• Rejects all null hypotheses with rank ≤ k This adaptive method is less conservative than FWER approaches and is the default correction in tools like DESeq2, edgeR, and GSEA for differential expression and enrichment analyses.

The minimum FDR at which a given test would be called significant. While a p-value measures the probability of observing data as extreme under the null, the q-value directly estimates the proportion of false positives incurred when calling that feature significant. Calculated using the Storey-Tibshirani method, q-values provide a more interpretable measure of statistical confidence for each individual feature. A feature with q = 0.03 means that among all features with q ≤ 0.03, approximately 3% are expected to be false discoveries.

The umbrella category of statistical adjustments applied when performing simultaneous inference on thousands of gene sets or genomic features. Without correction, testing 10,000 genes at α = 0.05 would yield ~500 false positives by chance alone. Correction strategies fall into two families:

• FWER methods (Bonferroni, Holm): Control probability of any false positive
• FDR methods (Benjamini-Hochberg, Benjamini-Yekutieli): Control the expected proportion of false positives FDR is preferred in exploratory biomarker discovery; FWER is reserved for confirmatory clinical trials.

An extension of the Benjamini-Hochberg procedure that controls FDR under arbitrary dependency structures among test statistics. The standard BH procedure assumes independence or positive regression dependency—an assumption often violated in pathway enrichment where gene sets overlap substantially. The BY procedure multiplies the BH critical value by a harmonic sum constant, making it more conservative but guaranteeing FDR control regardless of correlation patterns. Essential when analyzing highly redundant gene set collections like GO terms.

Two fundamentally different null hypotheses in gene set testing:

• Self-contained tests: Null hypothesis states no genes in the set are differentially expressed. Tests only within-set signal. Examples: GSEA, Global Test.
• Competitive tests: Null hypothesis states genes in the set are no more differentially expressed than genes outside the set. Compares against a background complement. Examples: ORA, camera. FDR correction applies differently—competitive tests compare gene sets against each other, while self-contained tests evaluate each set independently against its own null.