Fisher Information Matrix (FIM)

The CRLB establishes the minimum variance achievable by any unbiased estimator, defined as the inverse of the FIM: Var(θ̂) ≥ [I(θ)]⁻¹. In modulation classification, this bound quantifies the irreducible error floor due to noise. For a single parameter, the variance cannot fall below 1/I(θ). The bound is asymptotically tight—maximum likelihood estimators achieve it as sample size grows large. This provides a benchmark for evaluating practical classifier performance against theoretical optimality.

For N independent and identically distributed (i.i.d.) observations, the total Fisher Information scales linearly: I_N(θ) = N · I₁(θ). This implies that doubling the observation interval doubles the information about the parameter. In signal classification, this property justifies the trade-off between observation time and estimation accuracy—longer sensing windows directly improve modulation identification reliability. The standard error of estimation decreases proportionally to 1/√N.

The FIM transforms predictably under smooth, invertible reparameterizations. If η = g(θ) is a one-to-one transformation, then I(η) = Jᵀ · I(θ) · J, where J is the Jacobian matrix of the inverse transformation. This property ensures that information content is a geometric quantity intrinsic to the statistical manifold, not an artifact of the chosen coordinate system. In practice, it allows classifier designers to work in the most convenient parameter space—such as polar vs. Cartesian signal representations—without distorting the underlying information geometry.

The FIM emerges as the second-order Taylor expansion of the Kullback-Leibler divergence between distributions separated by a small parameter perturbation. For a small displacement dθ, the KL divergence approximates to D_KL(p(x|θ) || p(x|θ+dθ)) ≈ ½ dθᵀ · I(θ) · dθ. This geometric interpretation reveals the FIM as the Riemannian metric tensor on the statistical manifold. In modulation classification, this quantifies how distinguishable two nearby signal constellations are in the presence of noise.

The FIM obeys a data processing inequality: applying any transformation to the data cannot increase the Fisher Information. If T(X) is a statistic of the raw data X, then I_T(θ) ≤ I_X(θ), with equality if and only if T(X) is a sufficient statistic. This formalizes the intuitive notion that preprocessing raw IQ samples—through filtering, decimation, or feature extraction—can only preserve or lose information, never create it. Sufficient statistics achieve perfect information compression without loss.

A singular FIM indicates that not all parameters are independently identifiable from the observed data. This occurs when parameters are structurally confounded—for example, when carrier phase offset and channel delay produce indistinguishable effects on the received signal. In modulation classification, singularity signals that the chosen signal model is overparameterized. Regularization techniques or model reduction must be applied to restore identifiability before reliable estimation or classification can proceed.

Two distinct forms of the FIM exist for practical computation. The expected Fisher information is a theoretical quantity derived from the statistical model, while the observed Fisher information is calculated directly from the realized data sample.

• Expected FIM: I(θ) = -E[∂²ℓ/∂θ²], used for experimental design and asymptotic analysis.
• Observed FIM: J(θ) = -∂²ℓ/∂θ² evaluated at the MLE, used for practical confidence interval construction.
• Distinction: In correctly specified models, they converge asymptotically, but the observed FIM is preferred for finite-sample inference in signal processing.

A singular Fisher Information Matrix indicates a fundamental identifiability problem in the estimation task. When the FIM is not invertible, the CRLB is undefined, and the parameters cannot be uniquely estimated from the observed data.

• Cause: Occurs when two modulation parameters are perfectly correlated or redundant in the likelihood function.
• Example: Joint estimation of carrier phase and timing offset may become singular if the observation window is too short.
• Remedy: Requires reparameterization, additional constraints, or the injection of pilot symbols to break the degeneracy.

In composite hypothesis testing, the FIM must be evaluated under each candidate modulation hypothesis to compute the corresponding CRLB. This reveals the intrinsic discriminability between signal types.

• Modulation-Dependent Bounds: Each hypothesis Hᵢ has its own FIM Iᵢ(θ), reflecting how precisely nuisance parameters can be estimated under that modulation assumption.
• KL Divergence Link: The FIM appears in the second-order Taylor expansion of the Kullback-Leibler divergence between neighboring parameter values, connecting estimation accuracy to classification separability.
• Design Insight: Comparing FIMs across hypotheses guides feature selection by identifying parameters with the highest Fisher information ratios.

The FIM has been adapted as the Natural Gradient in neural network optimization, where it re-scales the gradient by the inverse FIM to account for the geometry of the parameter space.

• Natural Gradient Descent: Δθ = -η · F⁻¹ · ∇L, where F is the empirical Fisher matrix.
• Fisher Information Matrix in Pruning: Diagonal elements of the FIM indicate the sensitivity of the loss to each weight, enabling importance-based parameter pruning without significant accuracy loss.
• Relevance to AMC: In deep modulation classifiers, the FIM can identify which network layers carry the most discriminative information about the modulation format.

The log-likelihood function is the direct precursor to the FIM. The FIM is defined as the negative expected Hessian of this function, measuring its average curvature at the true parameter value.

• Definition: ℓ(θ|x) = log p(x|θ)
• Sharpness Interpretation: A sharply peaked log-likelihood (high curvature, large FIM) implies that small deviations from the true parameter cause large drops in likelihood, enabling precise estimation.
• Additivity: For independent observations, the log-likelihood and FIM are additive, making them natural tools for sequential and distributed signal processing.