The autoregressive model of a time series
The AR model is the simplest way to state that the present depends on the past. This article explains the meaning of its parameter, how quickly a series forgets a shock, how the model forecasts, and where the distribution of the shocks matters.
Some processes have no memory: successive tosses of a fair coin are independent.
Most quantities measured over time are different. Interest rates, temperatures, inflation, the latency of a network link: today's value is usually close to yesterday's, plus something new. A hot day tends to be followed by another, and a slow server tends to remain slow for some time.
Daily temperature shows the idea clearly. Suppose today is 5 °C warmer than the seasonal normal and each day keeps 70% of the previous day's departure. Tomorrow's best forecast is then 3.5 °C above normal, the day after 2.45 °C, and a week later about 0.4 °C: the warm spell is gradually forgotten, while new weather adds fresh departures each day.
The autoregressive (AR) model is the simplest formal description of this, and nearly every more elaborate time-series model is built on it. This article addresses three questions: what its single parameter means, how quickly a series forgets a shock, and where the distribution of the shocks matters.
The autoregressive scheme goes back to Yule (1927), who used it for sunspots, as described in Yule's sunspot model. Walker (1931) generalised it to many lags, Mann and Wald (1943) established its estimation theory, and Box and Jenkins (1970) made it the core of a practical forecasting method; the current edition of their book (Box, Jenkins, Reinsel and Ljung, 2015) remains the standard reference. Hyndman and Athanasopoulos (2021) give a modern and freely available treatment.
The first-order model
The AR(1) model says each observation is a fraction of the previous one plus a new shock:
where the shocks are independent with mean zero and variance . The single parameter (written in the figure below) carries all the memory.
If the process is stationary: shocks fade and the series fluctuates around a fixed mean.
| Quantity | Value |
|---|---|
| Mean | |
| Variance | |
| Autocorrelation at lag |
The last line is the most useful. A shock loses half its effect after periods: about 6.6 periods for , one period for . In the temperature example, gives a half-life of about two days. At shocks never fade and the process becomes a random walk, which needs different tools altogether. The variance expression shows a less obvious effect: persistence amplifies variability. With the series varies about five times as much as the shocks that drive it.
More lags
The AR() model lets the present depend on past values,
and is stationary when all roots of lie outside the unit circle. The order is usually chosen with the partial autocorrelation function, which cuts off after lag , and an information criterion such as AIC.
Forecasting
For AR(1), the forecast steps ahead pulls the last observation back towards the mean:
As the horizon grows, the forecast approaches the mean and its uncertainty approaches the unconditional variance. Far enough ahead, the model knows nothing beyond the long-run average. For the temperature example, the forecast for a day two weeks ahead is practically the seasonal normal, with the full range of ordinary weather around it. Short-term interest rates behave in the same way over years rather than days, which is the basis of the Vasicek model.
In R the complete procedure takes four lines:
set.seed(1)
x <- arima.sim(model = list(ar = 0.8), n = 500) # simulate AR(1), phi = 0.8
fit <- arima(x, order = c(1, 0, 0)) # estimate phi and the mean
predict(fit, n.ahead = 5) # forecasts and standard errors
The distribution of the shocks
The point forecast only uses the mean of the shocks. Three other things depend on their full distribution:
- Prediction intervals. standard errors assumes normal shocks. With heavy-tailed shocks the interval is inaccurate in the tails, where risk is measured.
- Efficiency. Least squares is maximum likelihood only under normal errors; with heavier tails, a likelihood built on the right distribution estimates more precisely.
- Risk measures. Value at risk and expected shortfall are statements about tails, not centres.
Families such as the γ-order generalized normal, introduced in the γ-order generalized normal distribution, make the shape of the shocks something to estimate rather than assume.
The simulated series below illustrates three results: the autocorrelation at lag is ; a shock loses half its effect in steps; and memory inflates the variance by .
- Autocorrelation. At the measured autocorrelations (bars) follow the theoretical curve (dashed). When the slider is moved, the bars follow the new curve.
- A single shock. Press "Apply a shock"; the resulting spike decays. At it should halve in about 3 steps; at , in about 13.5.
- No persistence. Set : the series becomes pure noise and every bar beyond lag 0 sits near zero.
- Negative coefficient. Set . Now a high value is followed by a low one, and the autocorrelations alternate in sign, exactly as does.
- Near the unit root. Increase towards 0.99. The series shows long excursions, and the variance factor rises towards 50. At it would be infinite: a random walk has no stable variance at all.
Derivation of . Substituting the model into itself times gives . Those later shocks are independent of , so . In a stationary process both variances are equal, so the correlation is exactly . The half-life follows by solving .
The single parameter has three interpretations: how strongly today follows yesterday, how quickly a shock is forgotten, and how much the series swings compared with the shocks that drive it. The AR(1) model is simple enough to analyse by hand and still describes a considerable amount of real data. Its limitation is the distribution of the shocks, which does not affect the point forecasts but determines their uncertainty.
References
- G. U. Yule, "On a method of investigating periodicities in disturbed series, with special reference to Wolfer's sunspot numbers", Philosophical Transactions of the Royal Society of London A, 226, 267-298, 1927.
- H. B. Mann and A. Wald, "On the statistical treatment of linear stochastic difference equations", Econometrica, 11(3/4), 173-220, 1943.
- G. E. P. Box, G. M. Jenkins, G. C. Reinsel and G. M. Ljung, Time Series Analysis: Forecasting and Control, 5th ed., Wiley, 2015.
- R. J. Hyndman and G. Athanasopoulos, Forecasting: Principles and Practice, 3rd ed., OTexts, 2021. otexts.com/fpp3