Credible Interval

Unlike a frequentist confidence interval, a 95% credible interval allows you to state: 'There is a 95% probability that the true parameter value lies within this range, given the observed data.' This is the interpretation most non-statisticians intuitively expect. It is derived directly from the posterior distribution, which combines the prior distribution with the likelihood of the data via Bayes' theorem.

The interval's location and width are explicitly influenced by the chosen prior distribution. A strong, informative prior can significantly shrink or shift the credible interval. This is a feature, not a bug, as it allows for the formal incorporation of domain expertise. Sensitivity analysis is often performed by comparing intervals computed under different priors, such as a non-informative uniform prior versus a skeptical conservative prior.

The most common type of credible interval is the Highest Posterior Density Interval (HPDI). It is the narrowest interval containing the specified probability mass, ensuring that every point inside the interval has a higher probability density than any point outside it. This contrasts with an equal-tailed interval, which simply excludes α/2 from each tail but may include low-density regions.

Credible intervals are typically computed using posterior samples generated by Markov Chain Monte Carlo (MCMC) algorithms. Once a sufficient number of samples from the posterior are obtained, the interval is constructed empirically by finding the quantiles of the sample distribution. For complex hierarchical models, this sampling-based approach is the only tractable way to quantify uncertainty.

A 95% confidence interval does not mean there is a 95% chance the true parameter is in the interval. It means that if the experiment were repeated many times, 95% of the computed intervals would capture the true fixed parameter. The credible interval answers the direct question about parameter uncertainty, while the confidence interval makes a statement about the long-run performance of the procedure.

Credible intervals integrate seamlessly into Bayesian decision theory. A decision-maker can directly use the posterior probability that a parameter exceeds a critical threshold to calculate expected utility. For example, if the 95% credible interval for a drug's efficacy is entirely above the clinically significant margin, one can state the probability that the drug is effective is >97.5%.

The updated belief about a parameter's value after combining the prior distribution with observed data via Bayes' theorem. A 95% credible interval is computed directly from this distribution by finding the region containing 95% of the posterior probability mass. Unlike a confidence interval, the posterior allows direct probability statements: 'There is a 95% probability the true parameter lies in this range.'

The initial belief about a parameter's value before observing any data, encoded as a probability distribution. The choice of prior—whether informative (based on domain expertise), weakly informative (regularizing), or non-informative (flat)—directly shapes the resulting credible interval. The prior is the defining feature that distinguishes Bayesian credible intervals from frequentist confidence intervals.

The narrowest credible interval that contains a specified probability mass, where every point inside the interval has higher probability density than any point outside. Unlike an equal-tailed interval, the HDI always includes the posterior mode and is the optimal summary when the posterior is skewed or multimodal. It answers: 'What is the most compact range where the parameter likely falls?'

A confidence interval is a frequentist construct: if the experiment were repeated infinitely, 95% of computed intervals would contain the true fixed parameter. It cannot say 'the parameter has a 95% probability of being in this interval.' A credible interval directly states that probability given the observed data. This distinction is critical for risk communication in high-stakes domains like clinical trials.

A family of algorithms—including Metropolis-Hastings and Hamiltonian Monte Carlo—used to draw samples from posterior distributions when analytical solutions are intractable. Credible intervals are constructed empirically from these samples by computing quantiles. Modern probabilistic programming languages like Stan and PyMC automate this sampling process.

The foundational mathematical rule that updates prior beliefs with observed evidence: Posterior ∝ Likelihood × Prior. The credible interval is a direct summary of the resulting posterior. The theorem formalizes how rational agents should update uncertainty in light of new data, making credible intervals the natural expression of post-experiment knowledge.