Recessions as a hidden state: Hamilton's regime-switching model
Whether an economy is in recession is not observed directly and is officially dated months later. Hamilton's Markov-switching model treats expansion and recession as a hidden state, estimates how long each lasts, and computes the probability of recession from the growth figures as they arrive.
Whether an economy is in recession matters to almost everyone: to a firm deciding whether to hire, to a household deciding whether to take on a mortgage, and to a central bank deciding whether to cut rates. Yet a recession is not observed directly. In the United States, the committee of the National Bureau of Economic Research that dates business cycles announced on 1 December 2008 that a recession had begun in December 2007, a year after the fact. In the euro area a similar committee of economists dates cycles with comparable delays. In the meantime, the public relies on rules of thumb, the best known of which declares a recession after two consecutive quarters of falling output.
The statistical question behind the rule is how to infer an unobserved state from noisy data. In 1989 James Hamilton proposed treating expansion and recession as two hidden states of a Markov chain, with output growing at a different average rate in each. The model estimates how long expansions and recessions typically last, and it computes, quarter by quarter, the probability that the economy is in recession. This article presents the model and compares it with the two-quarter rule.
The relevant literature is as follows.
- Burns and Mitchell (1946) defined business cycles empirically as recurrent sequences of expansions and contractions, the basis of the NBER's dating practice.
- Baum, Petrie, Soules and Weiss (1970) developed the estimation methods for hidden Markov models, and Rabiner (1989) made them widely known through their use in speech recognition.
- Hamilton (1989) applied a two-state Markov-switching model to US output growth and showed that its inferred recessions closely matched the NBER's dates.
- Kim and Nelson (1999) extended the approach to state-space models with regime switching.
- Chauvet and Piger (2008) tested Markov-switching models on the data that were available in real time and found that they identified US recession turning points with few false signals, generally well before the official announcements.
The model
Let be the state of the economy in quarter : expansion () or recession (). The state follows a Markov chain: an expansion continues into the next quarter with probability and a recession with probability . Measured growth is
with a higher mean in expansions and a lower, often negative, mean in recessions. Hamilton's model also allows the deviations from the mean to follow an autoregression; that refinement is omitted here.
The transition probabilities determine the length of each phase. The number of quarters a phase lasts is geometric, with expected duration for an expansion and for a recession, and the long-run share of quarters spent in recession is . For US output growth from the early 1950s to 1984, Hamilton estimated growth of about 1.2% per quarter in expansions and about −0.4% in recessions, with and . Those values imply expansions of about ten quarters and recessions of about four on average.
Inferring the state
The state is hidden, so the model works with its probability. Hamilton's filter updates that probability each quarter in two steps, in the same way as the Kalman filter does for a continuous state.
Predict. Before the new figure is known, the probability of recession is carried forward by the Markov chain: , where is last quarter's probability.
Update. When the growth figure arrives, Bayes' rule weighs the prediction by how likely that figure is under each state:
with the standard normal density. A single weak quarter during a long expansion raises the probability of recession only moderately, because expansions are persistent; a second weak quarter raises it much further. The two-quarter rule is a crude version of the same reasoning, with fixed thresholds instead of probabilities.
Using later data as well, a smoothed probability can be computed for each past quarter. That is the model's counterpart of the committee's retrospective dating, while the filtered probability corresponds to what can be known at the time.
Real-time signals and the two-quarter rule
The filter and the rule make different errors. The rule ignores the size of the decline: a quarter of −0.1% counts as much as one of −2%. It also misses recessions in which output falls sharply in one quarter and barely in the next, and it signals late, since it needs two quarters of data. The filter uses the size of each figure and the known persistence of the two phases, and it reports its uncertainty as a probability, which a central bank or a firm can weigh against the cost of acting too early or too late.
The figure below simulates forty years of quarterly growth and runs both methods on the same data. With Hamilton's estimates and measurement noise of 0.8 points, the filtered probability classifies about 90% of quarters correctly against about 82% for the two-quarter rule, and it signals a recession about half a quarter after it begins on average, against about one and a half quarters for the rule, which also misses many short recessions.
Suggested settings:
- Hamilton's estimates. With the defaults, recessions are shaded in both charts. The filtered probability rises above 50% within a quarter of most recession starts; the two-quarter rule reacts later, and some short recessions never trigger it.
- Noisier data. Raise the noise to 1.2. Both methods make more errors, and the filter's delay rises to about one quarter, because a single bad figure is no longer strong evidence on its own.
- Clearer data. Lower the noise to 0.5. The filter's classification rises to about 96% and its delay falls close to zero, while the rule improves only a little: its weakness is the fixed threshold, not the noise.
- More persistent phases. Set p = 0.95 and q = 0.85. Expansions last about twenty quarters and recessions about seven on average. The filter waits for more evidence before switching, because each phase is less likely to end in any given quarter.
Expected durations. In each quarter of an expansion, the expansion ends with probability , independently of how long it has lasted. The duration is therefore geometric: for , with mean . The long-run share of recession quarters solves the balance condition : in the long run, as many quarters move from recession to expansion as from expansion to recession. Solving gives .
The regime-switching model replaces a fixed rule with an explicit probability, and it separates two questions that are often confused: how long phases typically last, which the transition probabilities answer, and whether the economy is in recession now, which the filter answers with the data available. For anyone who must act before the official dating, the second question is the relevant one.
References
- A. F. Burns and W. C. Mitchell, Measuring Business Cycles, National Bureau of Economic Research, 1946.
- L. E. Baum, T. Petrie, G. Soules and N. Weiss, "A maximization technique occurring in the statistical analysis of probabilistic functions of Markov chains", Annals of Mathematical Statistics, 41(1), 164-171, 1970.
- J. D. Hamilton, "A new approach to the economic analysis of nonstationary time series and the business cycle", Econometrica, 57(2), 357-384, 1989.
- L. R. Rabiner, "A tutorial on hidden Markov models and selected applications in speech recognition", Proceedings of the IEEE, 77(2), 257-286, 1989.
- C.-J. Kim and C. R. Nelson, State-Space Models with Regime Switching, MIT Press, 1999.
- M. Chauvet and J. Piger, "A comparison of the real-time performance of business cycle dating methods", Journal of Business & Economic Statistics, 26(1), 42-49, 2008.