Log-Likelihood Function

In likelihood-based classifiers, computing the joint probability of a long sequence of IQ samples requires multiplying thousands of conditional densities. This product rapidly underflows standard floating-point precision. The log-likelihood function replaces this product with a sum: log L(θ|x) = Σ log p(x_i|θ). This transformation keeps values within a manageable dynamic range, preventing the vanishing probability problem that plagues naive implementations. For example, processing 10,000 samples with individual likelihoods of 0.1 would yield a product of 10^-10000—a value indistinguishable from zero in double precision—while the log-sum remains a tractable -23025.

Most signal models under Additive White Gaussian Noise (AWGN) belong to the exponential family of distributions. Taking the logarithm collapses the exponential and the Gaussian exponent into a simple quadratic form. The log-likelihood for a received signal r given transmitted symbol s becomes proportional to -|r - s|^2 / (2σ^2). This transforms the Maximum Likelihood (ML) estimation problem into a minimum Euclidean distance problem, enabling closed-form solutions and efficient grid searches over the constellation. Without the log transformation, deriving gradient updates or solving for critical points would be algebraically intractable.

The natural logarithm is a strictly monotonically increasing function. This guarantees that the ordering of likelihood values is preserved: if L(θ_1) > L(θ_2), then log L(θ_1) > log L(θ_2). Consequently, the Maximum Likelihood Sequence Estimator (MLSE) and Maximum A Posteriori (MAP) decision rules yield identical classification outcomes whether operating on the raw likelihood or the log-likelihood. The decision boundaries in the hypothesis space remain invariant. This property is critical—it means the log transform is a purely computational convenience with zero impact on the optimality of the classifier.

For many modulation schemes, the log-likelihood function factorizes into a sum of terms, each depending on a distinct sufficient statistic. In phase-shift keying (PSK), the log-likelihood decomposes into a term involving the received phase and another involving the received amplitude. This decomposition reveals which signal features carry all relevant information for classification, enabling dimensionality reduction. The Fisher Information Matrix (FIM) is derived directly from the second derivative of the log-likelihood, quantifying the curvature and thus the precision with which parameters can be estimated from the observed data.

Model selection metrics like the Akaike Information Criterion (AIC) and Bayesian Information Criterion (BIC) are built directly on the log-likelihood. AIC = -2 log L + 2k and BIC = -2 log L + k log n, where k is the number of free parameters and n is the sample size. These criteria penalize the maximized log-likelihood to prevent overfitting when comparing modulation hypotheses of varying complexity. The log-likelihood thus serves as the raw goodness-of-fit measure that must be balanced against model parsimony in composite hypothesis testing.

When closed-form ML estimates are unavailable, iterative methods like Expectation-Maximization (EM) or gradient ascent are required. The log-likelihood's additive structure yields clean expressions for the score function (gradient) and observed Fisher information (negative Hessian). For a Gaussian noise model, the gradient is a sum of weighted residuals, and the Hessian has a block-diagonal structure that accelerates Newton-Raphson convergence. These properties are exploited in the Generalized Likelihood Ratio Test (GLRT) to efficiently estimate unknown nuisance parameters before forming the classification statistic.