Cramér-Rao Lower Bound (CRLB)

The CRLB is mathematically defined as the inverse of the Fisher Information Matrix (FIM). For any unbiased estimator of a parameter vector θ, the covariance matrix satisfies:

• Scalar case: var(θ̂) ≥ 1/I(θ), where I(θ) is the Fisher information
• Vector case: cov(θ̂) ≥ F⁻¹, where F is the Fisher Information Matrix
• The FIM quantifies the curvature of the log-likelihood function—sharper peaks yield more information and lower bounds
• This relationship directly links signal structure to estimation precision limits

The standard CRLB applies exclusively to unbiased estimators—those whose expected value equals the true parameter. Key implications:

• An estimator achieving the CRLB is called efficient
• Biased estimators can sometimes beat the CRLB by trading variance for bias
• The biased CRLB variant incorporates the bias gradient into the bound calculation
• In modulation classification, maximum likelihood estimators asymptotically approach the CRLB as sample size increases
• Practical classifiers often operate above the bound due to finite samples and model mismatch

The CRLB derivation requires specific regularity conditions on the probability density function. These ensure mathematical validity:

• The support of the density must not depend on the parameter being estimated
• The density must be twice differentiable with respect to the parameter
• The order of integration and differentiation can be interchanged
• Violations occur in practical scenarios like uniform distributions or parameter-dependent boundaries
• For modulation signals with unknown carrier phase, these conditions typically hold for AWGN channels

Under AWGN channels, the CRLB takes a particularly elegant form for signal parameter estimation:

• The Fisher information scales with the signal-to-noise ratio (SNR)
• For carrier frequency estimation: CRLB ∝ 1/(SNR × N³), where N is sample count
• For phase estimation: CRLB ∝ 1/(SNR × N)
• Higher-order modulations (64-QAM vs QPSK) exhibit different Fisher information due to constellation geometry
• This provides a theoretical SNR wall—below which reliable classification becomes fundamentally impossible

The CRLB serves as the gold standard for evaluating modulation classifier designs:

• Any classifier's empirical variance can be compared against the CRLB to quantify efficiency
• A classifier operating within 1-2 dB of the CRLB is considered near-optimal
• The bound reveals which signal parameters are inherently difficult to estimate for specific modulation types
• In composite hypothesis testing, the CRLB on nuisance parameters determines the penalty for unknown channel state
• Used to justify stopping criteria in iterative estimation-maximization algorithms

Maximum Likelihood Estimators (MLEs) possess a special asymptotic relationship with the CRLB:

• MLEs are asymptotically efficient—they achieve the CRLB as sample size approaches infinity
• The asymptotic covariance of the MLE equals the inverse Fisher Information Matrix
• For finite samples, MLEs may exhibit bias and variance above the bound
• In modulation classification, the Average Likelihood Ratio Test (ALRT) inherits this asymptotic optimality
• Practical implementations using the Expectation-Maximization algorithm converge toward the CRLB with sufficient iterations

A non-symmetric measure of how one probability distribution diverges from a reference distribution. In modulation classification, KL divergence quantifies the discriminability between competing signal hypotheses. Larger KL divergence between modulation candidates implies easier classification and a lower probability of error. The CRLB and KL divergence are linked through the Fisher Information: the curvature of KL divergence at the true parameter equals the Fisher Information, connecting information geometry to estimation bounds.

An estimator is Cramér-Rao efficient if its variance achieves the CRLB for all possible parameter values. In practice, maximum likelihood estimators are asymptotically efficient—they approach the bound as sample size grows large. For modulation classification:

• Asymptotic efficiency: ML estimators achieve the CRLB as N → ∞
• Finite-sample bias: Real-world estimators may be biased, requiring the biased CRLB variant
• Super-efficiency: Rare cases where estimators beat the CRLB at isolated parameter points (disallowed by regularity conditions)

An extension of the classical CRLB that incorporates prior information about the parameter being estimated. Unlike the standard bound which assumes deterministic unknown parameters, the Bayesian CRLB averages over the prior distribution. This is directly relevant to the Maximum A Posteriori (MAP) Classifier and Average Likelihood Ratio Test (ALRT), where unknown signal parameters are treated as random variables. The Bayesian bound is always tighter than or equal to the classical bound when informative priors are available.

The process of estimating unknown variables that are not of primary interest but must be accounted for to accurately evaluate the likelihood function. In modulation classification, nuisance parameters include:

• Carrier phase offset
• Symbol timing error
• Channel gain and noise variance The CRLB can be partitioned to isolate the bound for parameters of interest while treating others as nuisances, using the Schur complement of the Fisher Information Matrix to compute the marginal bound.