Expectation-Maximization (EM) Algorithm

The EM algorithm alternates between two distinct steps until convergence:

• E-Step (Expectation): Computes the conditional expectation of the complete-data log-likelihood given the observed data and the current estimate of the parameters. This step infers the sufficient statistics of the hidden variables.
• M-Step (Maximization): Updates the parameter estimates by maximizing the expected log-likelihood function formed in the E-step. This is often a closed-form, simpler optimization. This decoupling transforms an intractable direct maximization into a sequence of tractable subproblems.

A fundamental property of the EM algorithm is that the observed-data likelihood is guaranteed to increase or stay the same with each iteration. It never decreases.

• The algorithm constructs a lower bound (Q-function) on the log-likelihood in the E-step.
• The M-step maximizes this lower bound, pushing the likelihood upward.
• While it converges to a local maximum or saddle point, it does not guarantee finding the global maximum, making initialization critical.

In Automatic Modulation Classification, the transmitted symbols and channel state are often hidden variables. The EM algorithm excels here by treating them as missing data.

• Unknown Symbols: The algorithm iteratively estimates the transmitted symbols (E-step) and then updates the channel estimate (M-step) in a data-aided fashion without a pilot sequence.
• Nuisance Parameters: It jointly estimates carrier phase offset, timing error, and channel coefficients while classifying the modulation.
• This enables quasi-coherent classification performance without requiring full synchronization beforehand.

The EM algorithm works by optimizing a surrogate function, Q(θ|θ^(t)), rather than the intractable observed-data likelihood directly.

• Definition: Q(θ|θ^(t)) = E[log P(X, Z | θ) | X, θ^(t)], where X is observed data and Z is hidden data.
• This surrogate is a lower bound that touches the true log-likelihood at the current parameter estimate.
• The M-step finds the peak of this surrogate, which guarantees an improvement in the actual likelihood unless a stationary point has been reached.

The algorithm stops when progress becomes negligible. Common stopping rules include:

• Parameter Stability: Stop when the change in parameter estimates ||θ^(t+1) - θ^(t)|| falls below a small threshold.
• Log-Likelihood Plateau: Stop when the relative increase in the observed-data log-likelihood is below a tolerance.
• Iteration Limit: A hard cap on iterations prevents infinite loops in flat regions. Linear convergence is typical, but acceleration techniques like Aitken's method can speed up the process significantly.

The K-Means algorithm is a classic, hard-assignment special case of the EM algorithm applied to a Gaussian Mixture Model (GMM) with specific constraints.

• Hard vs. Soft: K-Means assigns each data point to exactly one cluster (hard), while EM for GMMs assigns a posterior probability of belonging to each cluster (soft).
• Variance: K-Means implicitly assumes all clusters have identical, spherical covariance matrices. EM relaxes this, learning full covariance structures.
• Understanding this connection provides intuition for how EM iteratively refines both assignments and model parameters.

An optimal detection technique that selects the most probable transmitted symbol sequence by maximizing the likelihood function over the entire sequence. MLSE is commonly implemented via the Viterbi algorithm, which efficiently searches the trellis of possible state transitions. In modulation classification, MLSE provides the theoretical upper bound on classification accuracy when all signal parameters are known, serving as the benchmark against which EM-based classifiers are compared.

The natural logarithm of the likelihood function, used to transform products of conditional densities into sums for numerical stability. In the E-step of the EM algorithm, the log-likelihood of the complete data is computed given current parameter estimates. This transformation is essential because raw likelihoods for long signal sequences become vanishingly small, causing numerical underflow in practical implementations of modulation classifiers.

A statistical model representing a system with unobserved states that emit observable symbols. The EM algorithm, specifically the Baum-Welch variant, is the standard method for training HMMs when state sequences are unknown. In modulation classification, HMMs model the sequential structure of coded transmissions where the underlying modulation state evolves according to Markovian dynamics, and EM jointly estimates both the state sequence and channel parameters.

A non-symmetric measure of how one probability distribution diverges from a reference distribution. The EM algorithm can be interpreted as alternating KL divergence minimization steps: the E-step minimizes the KL divergence between the posterior of latent variables and a variational distribution, while the M-step maximizes the expected complete-data log-likelihood. This information-theoretic view provides convergence guarantees for modulation classifier training.

A fundamental lower bound on the variance of any unbiased estimator, expressed as the inverse of the Fisher Information Matrix (FIM). For EM-based modulation classifiers, the CRLB provides the theoretical benchmark for parameter estimation accuracy. The observed Fisher information, computable via Louis' method within the EM framework, quantifies the uncertainty in estimated channel parameters and modulation indices after convergence.

A model selection criterion that balances goodness-of-fit with model complexity by penalizing the log-likelihood with a term proportional to the logarithm of the sample size. In modulation classification, BIC is used alongside EM to select the correct modulation order—for example, distinguishing between QPSK, 16-QAM, and 64-QAM—by evaluating the penalized likelihood after EM converges for each candidate hypothesis.