Bayesian Active Learning

The core of Bayesian Active Learning is a probabilistic model, such as a Bayesian Neural Network (BNN) or a model with a Gaussian Process prior. Instead of point estimates for parameters, the model maintains a posterior distribution over parameters or functions. This distribution encodes all model uncertainty given the observed data. The ability to make predictive distributions—not just point predictions—is the source of all subsequent utility measures used for query selection.

A key output is the explicit quantification of predictive uncertainty, which decomposes into:

• Aleatoric Uncertainty: Inherent noise or randomness in the observation (e.g., label ambiguity). This is data uncertainty.
• Epistemic Uncertainty: Model uncertainty due to a lack of knowledge, which can be reduced with more data. This is the primary target for active learning queries.

Methods like Monte Carlo Dropout or deep ensembles provide practical approximations of this Bayesian posterior for complex models.

Query selection is driven by an acquisition function that scores unlabeled points based on the expected utility of their label. Bayesian frameworks enable principled functions derived from information theory:

• Maximum Entropy: Query the point with the highest predictive entropy (total uncertainty).
• Bayesian Active Learning by Disagreement (BALD): Seeks points that maximize the mutual information between the model parameters and the predicted label. It targets points where the model is uncertain about its parameters.
• Expected Model Change: Measures the expected gradient magnitude if the point were labeled and added to the training set.

The framework naturally balances exploration (querying uncertain regions to improve the global model) and exploitation (querying near decision boundaries to refine them). Information-theoretic acquisition functions like BALD inherently manage this trade-off by seeking information that maximally reduces global model uncertainty, rather than just local predictive confusion. This leads to more diverse and globally informative query batches compared to simple uncertainty sampling.

Bayesian Active Learning formalizes the query process as sequential decision-making. After each query and label acquisition, the model's posterior distribution is updated (via Bayesian inference). This updated posterior then informs the next query decision. This closed-loop, adaptive process is optimal for minimizing future expected loss or maximizing information gain over a sequence of decisions, making it sample-efficient.

The framework is a direct application of Bayesian Optimal Experimental Design (BOED). The goal is to choose data points (experiments) that are expected to be most informative about the model parameters. The acquisition function is often the expected reduction in entropy of the posterior distribution over parameters. This provides a rigorous, decision-theoretic justification for the query strategy, grounded in statistics rather than heuristic.

An Acquisition Function is a mathematical criterion that scores unlabeled data points based on their expected utility if labeled, guiding the query strategy. It formalizes the exploration-exploitation trade-off.

• Purpose: Quantifies the value of querying a specific point.
• Common Examples: Expected Improvement, Probability of Improvement, Upper Confidence Bound (UCB), and Entropy-based measures.
• In Bayesian AL: Often derived from the posterior predictive distribution, such as Bayesian Active Learning by Disagreement (BALD) which selects points where the model parameters are most uncertain about the prediction.

Uncertainty Sampling is the most common active learning query strategy, selecting instances where the model's current predictive uncertainty is highest.

• Core Idea: Labeling uncertain points is most informative for refining decision boundaries.
• Uncertainty Measures:
  • Least Confident: 1 - max(P(y|x)).
  • Margin Sampling: Difference between top two predicted probabilities.
  • Entropy: ∑ -P(y|x) log P(y|x).
• Bayesian Context: Naturally implemented using posterior predictive entropy or mutual information between predictions and model parameters.

Bayesian Neural Networks are neural networks with probability distributions over their weights, providing a principled framework for aleatoric and epistemic uncertainty quantification.

• Contrast to Standard NNs: Outputs a predictive distribution, not a point estimate.
• Key for Bayesian AL: The posterior over weights enables computation of information-theoretic acquisition functions.
• Practical Approximations: Exact inference is intractable; methods include:
  • Variational Inference (Bayes by Backprop)
  • Monte Carlo Dropout
  • Deep Ensembles (a frequentist approximation).

Expected Model Change is an acquisition function that selects the data point expected to induce the largest change to the model's parameters if its label were known and incorporated.

• Intuition: Seeks points that would most impact the model's internal state.
• Common Metric: The expected gradient magnitude of the loss with respect to the model parameters.
• Bayesian Formulation: Can be computed by averaging over the posterior distribution of possible labels for the candidate point.
• Use Case: Particularly effective for models where parameter change correlates strongly with learning, such as logistic regression or neural networks.

Query-By-Committee is an ensemble-based active learning strategy that maintains a committee of models and queries points where committee members disagree the most.

• Committee Disagreement: Measured by vote entropy or Kullback-Leibler (KL) divergence between member predictions.
• Bayesian Interpretation: The committee approximates the posterior distribution over hypotheses. Disagreement approximates version space reduction.
• Methods to Form Committee:
  • Different model initializations.
  • Bootstrapped datasets.
  • Sampling from a Bayesian posterior (e.g., via MCMC).

Monte Carlo Dropout is a practical and scalable technique to approximate Bayesian inference in deep neural networks, enabling uncertainty estimation for active learning.

• Mechanism: Perform T forward passes at inference with dropout enabled. The variance across predictions estimates model (epistemic) uncertainty.
• Theoretical Basis: Shown to approximate deep Gaussian processes.
• Active Learning Application: The predictive entropy or BALD score can be computed directly from the T sampled softmax outputs. This provides a computationally feasible way to implement Bayesian active learning for large models without full BNN training.