Bayesian Information Criterion (BIC)

BIC is derived as an asymptotic approximation to the Bayes factor between competing models. It approximates the marginal likelihood (model evidence) by integrating over the parameter space. The formula is:

BIC = -2 · ln(L̂) + k · ln(n)

• L̂: The maximized value of the likelihood function.
• k: The number of free parameters to be estimated.
• n: The number of observations (sample size).

Selecting the model with the lowest BIC is equivalent to choosing the model with the highest approximate posterior probability, assuming uniform priors over models.

Unlike AIC, which penalizes complexity with a factor of 2k, BIC's penalty is k · ln(n). This distinction is critical:

• For n ≥ 8, ln(n) > 2, meaning BIC imposes a heavier penalty than AIC.
• As the sample size grows, the penalty increases, aggressively discouraging overfitting.
• This property makes BIC asymptotically consistent: as n → ∞, the probability of selecting the true model (if it is in the candidate set) approaches 1.
• In modulation classification, this favors simpler constellation geometries when sample evidence is strong.

In likelihood-based modulation classification, BIC is used to select between candidate modulation hypotheses (e.g., BPSK vs. QPSK vs. 16-QAM) when the number of unknown nuisance parameters varies.

• Nested Models: BIC effectively penalizes higher-order modulations that require more parameters (e.g., phase offset, timing error, channel coefficients).
• GLRT Integration: BIC can be computed using the maximized likelihood from a Generalized Likelihood Ratio Test (GLRT) framework.
• Practical Use: For a received IQ sample block, compute the maximized log-likelihood under each modulation hypothesis, apply the BIC formula, and select the hypothesis with the minimum score.

While both criteria balance fit and complexity, their philosophical goals differ:

• AIC aims to minimize the Kullback-Leibler divergence from the true data-generating process, optimizing for predictive accuracy. It is efficient but not consistent.
• BIC aims to identify the true model (if it exists), maximizing the posterior probability. It is consistent but not efficient.
• Practical Impact: AIC tends to select more complex models, especially with large samples. BIC selects simpler models. In signal classification, BIC is preferred when the goal is to identify the exact modulation format rather than just achieve low error rates.

BIC is mathematically equivalent to the Minimum Description Length (MDL) principle under a specific coding scheme. Both select the model that provides the shortest encoding of the data:

• Data Encoding Cost: -ln(L̂), the negative log-likelihood, measures the bits needed to encode the data given the model.
• Model Encoding Cost: (k/2) · ln(n), the bits needed to encode the model parameters.
• Total Description Length: BIC/2 approximates the total code length.

This information-theoretic interpretation reinforces BIC's role in selecting parsimonious models that compress the observed signal efficiently.

BIC relies on several assumptions that must be verified in practice:

• Laplace Approximation: The derivation uses a quadratic expansion around the MLE, valid only for large samples and regular models.
• Well-Specified Model: BIC assumes the true model is in the candidate set. If all models are misspecified, its theoretical guarantees weaken.
• Independent Observations: The standard formula assumes i.i.d. data. For correlated signal samples (e.g., with channel memory), modified criteria are required.
• Parameter Count: The penalty depends on an accurate count of free parameters. In modulation classification, nuisance parameters like channel taps must be carefully counted.
• Prior Sensitivity: The approximation assumes a unit information prior. Alternative priors lead to variants like the Generalized BIC.