Bayesian inference for the autoregressive coefficient near a unit root
Near a unit root, the classical sampling distribution of the AR coefficient is skewed and requires special tables. The Bayesian posterior, computed from the same simulations, is not. Sims and Uhlig's simulation illustrates the difference.
Suppose that the coefficient of an AR(1) model, estimated from 100 observations, is . The question is whether the series is a random walk.
The question has policy consequences. If the unemployment rate returns to a stable level after a recession, a temporary rise calls for temporary measures; if it behaves like a random walk, each recession leaves a lasting mark, an effect economists call hysteresis. An estimate of 0.95 from 100 quarterly observations, twenty-five years of data, is a realistic starting point for that discussion.
A classical analysis uses the Dickey-Fuller tables, because near the least-squares estimate has a skewed distribution with a downward bias, as described in least-squares estimation of the AR(1) coefficient in three regimes. The test then asks how surprising would be if the truth were exactly 1. A Bayesian asks a different question: given , how plausible is each value of ? The second question has a simple, almost symmetric answer that requires no special tables.
In 1991 Christopher Sims and Harald Uhlig made this point with a well-known illustration. They simulated the AR(1) model for a grid of true values, estimated each series, and plotted the joint distribution of (true , estimated ) as a surface, which they described as a helicopter tour. A slice in one direction gives the classical sampling distribution; a slice in the other direction gives the Bayesian posterior.
Zellner (1971) set out the Bayesian analysis of regression and autoregressive models, including the result that with a flat prior and normal errors, the posterior for the AR coefficient is centred on the least-squares estimate. Sims (1988) argued that classical unit-root testing gives misleading conclusions from a Bayesian point of view, and Sims and Uhlig (1991) illustrated why with this simulation. Phillips (1991) argued that the flat prior is not neutral near a unit root and proposed an objective (Jeffreys-type) prior that puts more weight on large values of ; the debate occupied a special issue of the Journal of Applied Econometrics. Both sides agree on the simulation; they disagree about the prior.
Sampling distribution and posterior
Simulate many series of length for each true on a fine grid, and estimate each one. The result is a large table of pairs : a joint distribution, provided the true values are spread evenly over the grid. Spreading them evenly corresponds to a flat prior.
- Fix the true and look at the spread of : a vertical slice. This is the sampling distribution, the classical object. Near it is skewed to the left, with a long tail of low estimates.
- Fix the estimate and look at which true values produced it: a horizontal slice. Normalised, this is the posterior distribution of given the data, under the flat prior. It is close to a normal centred near , whatever is.
The skewness of the vertical slices does not carry over to the horizontal ones. The surface is a ridge along the diagonal, and its cross-sections in the two directions have different shapes.
Symmetry of the posterior
The likelihood of an AR(1) with normal errors, viewed as a function of for fixed data, is
It is exactly a normal shape centred on , with precision , whatever the true value of was. The unit root affects the distribution of the data before they are observed; it does not affect the shape of the likelihood once they are observed. With a flat prior the posterior is the likelihood, so it is symmetric too.
The role of the prior
That symmetry depends on the flat prior. Phillips argued that a flat prior on is not neutral: series with near 1 carry much more information (their is larger), and a Jeffreys-type prior reflecting that puts more weight on the unit root, which brings back conclusions closer to the classical ones. In summary, near a unit root the classical answer depends on which tables are used, and the Bayesian answer depends on which prior is used. The second dependence is at least explicit. For the unemployment question, the Bayesian result reads directly as a probability that recessions leave a permanent mark, about 10% for with a flat prior, as the figure shows; a different prior changes that probability in a way that can be stated and examined.
The figure below reproduces the simulation.
The heatmap counts simulated pairs, 600 series of length 100 for each of 31 true values; the blue line is the selected vertical slice, the orange line the horizontal one.
Suggested settings:
- The skewed slice. Set the true to 1.00. The blue curve is the sampling distribution: skewed, with a tail to the left, and its mean is below its median. The box plots show the same for four true values: the boxes at 0.90 and 0.95 are fairly symmetric, the one at 1.00 has a long lower whisker.
- The symmetric slice. Set the observed to 0.95. The orange posterior is roughly symmetric around 0.95, with a 90% interval of about 0.89 to 1.00, and puts around 10% probability on .
- Along the ridge. Move from 0.85 to 1.05. The posterior keeps the same shape and slides along; nothing special happens as it crosses 1. Moving the true across 1, the sampling distribution changes shape and then becomes very narrow in the explosive region, showing the three regimes.
- Repeated simulation. Press "Re-run the 18,600 simulations". The landscape changes in detail but not in shape: the asymmetry belongs to the vertical slices, not the horizontal ones.
Derivation of the symmetry. Expand . Completing the square in with gives plus a term that does not involve . So, conditional on the observed series and with known, the likelihood in is proportional to a density. Multiplying by a flat prior leaves it unchanged. No step of the argument uses the true value of , which is why the posterior's shape is the same on both sides of the unit root, while the sampling distribution, an average over data sets, is not.
The simulation guards against two opposite errors: thinking that the unit-root problem is only a technical nuisance, and thinking that Bayesian methods make it disappear. The two approaches answer different questions. Asking "how surprising is this estimate if ?" needs special tables; asking "how plausible is given this estimate?" needs a prior. Both answers are informative, and both should be reported.
References
- A. Zellner, An Introduction to Bayesian Inference in Econometrics, Wiley, 1971.
- C. A. Sims, "Bayesian skepticism on unit root econometrics", Journal of Economic Dynamics and Control, 12(2-3), 463-474, 1988.
- C. A. Sims and H. Uhlig, "Understanding unit rooters: a helicopter tour", Econometrica, 59(6), 1591-1599, 1991.
- P. C. B. Phillips, "To criticize the critics: an objective Bayesian analysis of stochastic trends", Journal of Applied Econometrics, 6(4), 333-364, 1991.
- D. A. Dickey and W. A. Fuller, "Distribution of the estimators for autoregressive time series with a unit root", Journal of the American Statistical Association, 74(366), 427-431, 1979.