ARIMA

Models the dependency between an observation and a specified number of lagged observations (its own past values). The parameter p denotes the order, or number of lags included. An AR(1) model, for instance, predicts today's sales using yesterday's sales. This component captures the momentum or mean-reverting behavior in the data. The mathematical formulation is a linear regression of the current value against its p previous values, making it effective for series where past values are strongly predictive.

Applies differencing to make the time series stationary. The parameter d represents the number of times the raw observations are differenced. For example, if d=1, the model works on the change between consecutive periods rather than the absolute values. This is critical because AR and MA components require a stable mean and variance. Stationarity is achieved when the differenced series has no visible trend or seasonality, a condition verified by statistical tests like the Augmented Dickey-Fuller test.

Models the dependency between an observation and the residual errors from a moving average applied to lagged observations. The parameter q specifies the order, or the number of lagged forecast errors incorporated. Unlike the AR component which uses past values, the MA component uses past shocks or innovations. An MA(1) model adjusts the forecast based on the error from the previous period, effectively smoothing out unpredictable, short-term fluctuations in the series.

The Autocorrelation Function (ACF) and Partial Autocorrelation Function (PACF) plots are the primary diagnostic tools for determining the p and q orders. The ACF plot shows the correlation between the series and its lags, including indirect effects. The PACF plot shows the direct correlation at each lag, removing intermediary influences. A sharp cut-off in the PACF suggests an AR(p) model, while a sharp cut-off in the ACF suggests an MA(q) model. These visual heuristics guide the initial model specification before iterative refinement.

For data with a repeating seasonal pattern, the Seasonal ARIMA (SARIMA) model adds a second set of seasonal parameters: (P, D, Q, m). Here, P is the seasonal autoregressive order, D is the seasonal differencing order, Q is the seasonal moving average order, and m is the number of periods per season (e.g., 12 for monthly data with a yearly cycle). This allows the model to capture both non-seasonal dependencies and the relationship between an observation and its value from one full seasonal cycle ago.

ARIMA models are built using the iterative Box-Jenkins methodology, a three-stage process. Stage 1: Identification—use plots and statistical tests to determine if the series is stationary and propose initial p, d, q values. Stage 2: Estimation—use maximum likelihood estimation to fit the parameters. Stage 3: Diagnostic Checking—analyze the residuals to ensure they resemble white noise (no remaining autocorrelation), confirming the model has captured all signal. The cycle repeats if diagnostics fail.