Multiple Hypothesis Testing Correction

Controls the probability of making one or more false discoveries across the entire family of hypothesis tests. The Bonferroni correction is the simplest FWER method, dividing the significance threshold α by the number of tests performed.

• Mechanism: Adjusts individual p-values upward by multiplying by the number of tests
• Strength: Extremely stringent; virtually guarantees no false positives
• Weakness: Dramatically reduces statistical power, especially with correlated tests
• Use case: Confirmatory analyses where a single false positive is unacceptable

Controls the False Discovery Rate (FDR) — the expected proportion of false positives among all rejected null hypotheses. This is the default correction in most pathway enrichment tools.

• Mechanism: Ranks p-values, then applies a linearly increasing threshold: p(k) ≤ (k/m) × q
• Strength: Substantially more power than FWER methods
• Adaptive nature: The threshold depends on the distribution of observed p-values
• Standard threshold: q = 0.05 means 5% of significant results are expected to be false discoveries

An extension of the Benjamini-Hochberg method that controls FDR under arbitrary dependence structures among test statistics. Pathway tests are often correlated due to overlapping gene membership, making BY more theoretically appropriate.

• Mechanism: Applies an additional harmonic correction factor to the BH threshold
• When to use: Gene sets with substantial gene overlap or correlated expression data
• Trade-off: More conservative than standard BH; may miss true enrichments
• Implementation: Available in R's p.adjust(method="BY")

Estimates the positive false discovery rate (pFDR) by modeling the distribution of p-values and estimating π₀ — the proportion of true null hypotheses. The q-value is the minimum FDR at which a test is called significant.

• Key insight: Not all null hypotheses are true; estimating π₀ increases power
• Output: Each test receives a q-value analogous to a p-value but on an FDR scale
• Advantage: More powerful than BH when many true positives exist
• Tool: Implemented in the qvalue R/Bioconductor package

Uses phenotype permutation to empirically estimate the null distribution of enrichment scores without parametric assumptions. This is the default approach in GSEA software.

• Mechanism: Shuffles sample labels thousands of times, recomputes enrichment scores
• Output: Normalized Enrichment Score (NES) and permutation-based FDR
• Advantage: Preserves gene-gene correlation structure inherent to the data
• Consideration: Computationally intensive; requires sufficient sample size for meaningful permutations

A sequentially rejective FWER method that is uniformly more powerful than the simple Bonferroni correction while maintaining strong control of the family-wise error rate.

• Mechanism: Orders p-values, then tests the smallest against α/n, the next against α/(n−1), stopping at the first non-rejection
• Advantage: Always more powerful than Bonferroni; no additional assumptions
• Limitation: Still an FWER method; overly conservative for exploratory enrichment analysis
• Relevance: Useful when a small number of pathways must be identified with high confidence

The probability of making one or more Type I errors across an entire family of hypothesis tests. FWER control is stringent and appropriate when even a single false positive is unacceptable.

• Bonferroni Correction: The simplest FWER method, dividing the significance threshold α by the number of tests m (α/m). Highly conservative with correlated data.
• Holm-Bonferroni Method: A uniformly more powerful step-down procedure that still controls FWER.
• Use Case: Genome-wide association studies (GWAS) where the standard threshold is p < 5 × 10⁻⁸.

A statistical method that identifies pathways over-represented in a list of differentially expressed genes using the hypergeometric distribution or Fisher's exact test. ORA requires an arbitrary significance cutoff to define the input gene list.

• Hypergeometric Test: Models the probability of observing k or more pathway genes in a list of n DEGs, given a background of N total genes.
• Background Selection: The choice of background gene universe critically impacts results; using the whole genome vs. only expressed genes changes null expectations.
• Limitation: Loses information by dichotomizing continuous expression data into 'significant' and 'non-significant' bins.

An empirical approach to multiple testing correction that builds a null distribution by randomly shuffling data labels and recalculating test statistics thousands of times. This preserves the inherent correlation structure of the data.

• Phenotype Permutation: Shuffles sample class labels; used in GSEA to assess gene set significance.
• Gene Permutation: Shuffles gene labels; used in ORA to assess pathway over-representation.
• Advantage: Does not assume independence of tests, providing accurate adjusted p-values for correlated genomic data where theoretical distributions fail.