Multiple Testing Correction

Controls the probability of making at least one Type I error across the entire family of hypotheses. The Bonferroni correction is the most conservative FWER method, dividing the significance threshold (α) by the number of tests (m). If you test 20,000 genes, the new threshold becomes α/20,000. This guarantees strong control but severely reduces statistical power, often missing true biological signals in high-dimensional genomics.

Controls the expected proportion of false positives among all rejected hypotheses, offering a more practical balance for exploratory research than FWER. The Benjamini-Hochberg procedure ranks p-values and compares each to a linearly increasing threshold. If you accept a 5% FDR, you expect 5% of your 'significant' findings to be null. This is the standard for GWAS and differential expression analysis.

When testing a single hypothesis at α=0.05, the chance of a false positive is 5%. Running 10,000 independent tests inflates the probability of at least one false positive to nearly 100%. This phenomenon occurs frequently in federated GWAS where millions of SNPs are tested for association with a phenotype. Without correction, spurious genotype-phenotype links would overwhelm valid biological signals.

Bonferroni divides α by the total number of tests, ensuring zero tolerance for any false positive. Benjamini-Hochberg sorts p-values and rejects hypotheses where p ≤ (i/m)Q, where Q is the desired FDR. The key trade-off: Bonferroni is appropriate for confirmatory trials where a single false positive is catastrophic; Benjamini-Hochberg is preferred for hypothesis generation in federated cohort discovery where follow-up studies will validate findings.

The q-value is the FDR analogue of the p-value, representing the minimum FDR at which a particular test would be deemed significant. Calculated using the Storey-Tibshirani method, q-values estimate the proportion of true null hypotheses (π₀) from the empirical distribution of p-values. This provides a direct measure of significance for each individual feature rather than a binary reject/fail-to-reject decision.

A non-parametric alternative that preserves the correlation structure between tests, unlike Bonferroni which assumes independence. The procedure randomly shuffles phenotype labels thousands of times, recalculates the test statistic for each permutation, and builds an empirical null distribution. The Westfall-Young method uses this to control FWER while accounting for linkage disequilibrium in genetic data, providing more power than Bonferroni for correlated SNPs.

The most conservative single-step method for controlling the family-wise error rate (FWER). It divides the significance threshold (α) by the number of tests (m) performed.

• Mechanism: Reject null hypothesis only if p ≤ α/m
• Strengths: Simple, strict control of any false positive
• Weakness: Dramatically reduces statistical power in high-dimensional genomics
• Use Case: Appropriate when the cost of a single false positive is catastrophic

A step-up method that controls the False Discovery Rate (FDR) rather than the FWER, offering a practical balance between discovery and error control.

• Mechanism: Ranks p-values, finds the largest k where p(k) ≤ (k/m) * q
• q-value: The FDR analogue of the p-value
• Advantage: Much greater power than Bonferroni in GWAS with thousands of markers
• Default FDR: Typically set at 5% or 10% for exploratory genomics

A scatter plot that visualizes the negative logarithm of p-values against chromosomal position, serving as the primary diagnostic tool for GWAS results.

• Y-axis: -log10(p-value); higher points indicate stronger association
• Threshold lines: Genome-wide significance (p < 5e-8) and suggestive significance (p < 1e-5)
• Peaks: Clusters of significant SNPs indicating potential causal loci
• Interpretation: Points above the Bonferroni-adjusted line survive multiple testing correction

The probability of making one or more Type I errors across an entire family of hypothesis tests. Controlled by methods like Bonferroni.

• Calculation: FWER = 1 - (1 - α)^m for independent tests
• Type I Error: Incorrectly rejecting a true null hypothesis (false positive)
• Strict Control: Ensures <5% chance of any false positive in the study
• Criticism: Overly stringent for genomic studies where some false positives are tolerable

The expected proportion of false positives among all rejected hypotheses. A more pragmatic error metric for exploratory genomic research.

• Definition: FDR = E[V/R] where V is false positives and R is total discoveries
• Benjamini-Hochberg: The standard procedure to control FDR at level q
• Interpretation: An FDR of 5% means ~5% of your significant hits are expected to be false
• Advantage: Scales gracefully to millions of tests without catastrophic power loss

A computationally intensive, non-parametric approach to estimate the empirical null distribution of test statistics by randomly shuffling phenotype labels.

• Process: Repeat analysis thousands of times with scrambled outcomes
• Adjusted p-value: Proportion of permuted statistics exceeding the observed statistic
• Advantage: Accounts for unknown correlation structures between tests
• Limitation: Computationally prohibitive for biobank-scale federated GWAS