Jorge Quitério Search
Mathematics

The power spectrum and its estimation

A stationary series can be described by the distribution of its variance over frequencies. Estimating this distribution is difficult: the periodogram does not converge, however much data is available. Welch averaging and the spectrogram address this, as an AR(2) simulation shows.

A time series can be decomposed into slow, medium and fast fluctuations, each carrying a share of the total variance. The distribution of that variance over frequency is the power spectrum.

The idea is in daily use, often unnoticed. A guitar-tuning application estimates the pitch of a string from the spectrum of the microphone signal. Maintenance engineers monitor the vibration spectrum of pumps, gearboxes and wind-turbine bearings, because a developing fault adds energy at a frequency set by the geometry of the damaged part, often weeks before the machine fails.

The concept is old, and so is its main difficulty. The natural estimator of a spectrum, the periodogram, becomes more detailed as more data are added, but its variance does not decrease. This note explains why, how Welch's method addresses the problem, and how a spectrogram describes a spectrum that changes over time.

Arthur Schuster introduced the periodogram in 1898 to hunt for hidden periodicities in meteorological and magnetic data, and reported several cycles that later proved spurious. The theory was developed in the 1930s. Wiener (1930) and Khinchin (1934) showed that the autocovariance function and the spectral density of a stationary process are a Fourier-transform pair: one describes memory in time, the other the distribution of variance over frequency. Bartlett (1948) showed that averaging periodograms of shorter segments reduces their variance, and Welch (1967) turned that into the method still in general use, with tapered, overlapping segments computed by the fast Fourier transform. Percival and Walden (1993) is the standard reference for the whole subject, and the same ideas sit behind Fisher's test for a periodogram peak in searching for cycles in Rhine discharge.

The spectrum of an autoregressive process

For an AR(pp) process Xt=ϕ1Xt−1+⋯+ϕpXt−p+etX_t = \phi_1 X_{t-1} + \dots + \phi_p X_{t-p} + e_t with innovation variance σ2\sigma^2, the spectrum has a closed form:

S(f)=σ2∣1−ϕ1e−2πif−⋯−ϕpe−2πipf∣2,0≤f≤12. S(f) = \frac{\sigma^2}{\big|1 - \phi_1 e^{-2\pi i f} - \dots - \phi_p e^{-2\pi i p f}\big|^2}, \qquad 0 \le f \le \tfrac12 .

The frequency ff is in cycles per sample, so a period of 10 samples is f=0.1f = 0.1. An AR(2) with complex roots, like the one Yule fitted to sunspots, has a peak near its pseudo-cycle. The closer the roots are to the unit circle, the taller and narrower the peak: the rhythm persists longer, so its frequency is better defined.

Inconsistency of the periodogram

The periodogram of nn observations measures how strongly each Fourier frequency j/nj/n is present:

I(fj)=1n∣∑t=1n(xt−xˉ) e−2πifjt∣2. I(f_j) = \frac{1}{n}\left|\sum_{t=1}^{n} (x_t - \bar x)\, e^{-2\pi i f_j t}\right|^2 .

It is asymptotically unbiased: E[I(f)]≈S(f)E[I(f)] \approx S(f). But each ordinate behaves like S(f)S(f) times an exponential random variable with mean 1, whatever nn is. Doubling the data doubles the number of ordinates, each with the same variance as before. The standard deviation of an exponential equals its mean, so the raw periodogram scatters by roughly ±100% around the true value regardless of the sample size. Schuster's spurious cycles were produced by this variability.

The same variability affects machine monitoring. A single raw periodogram of a healthy bearing shows isolated peaks that are estimation noise, and an alarm triggered by one of them would stop the machine without cause. Recording for longer does not help, because the scatter of each ordinate stays the same.

Welch's method

Welch's method divides the series into KK segments, multiplies each by a smooth taper (here a Hann window) to reduce leakage between frequencies, computes a periodogram per segment and averages them. Averaging KK independent exponentials divides the variance by about KK. The price is resolution: segments of length n/Kn/K only resolve frequencies K/nK/n apart. Choosing KK is a bias-variance trade-off in the frequency domain. Vibration analysers and audio software offer averaging over successive segments for this reason: the engineer gives up some frequency detail to obtain a spectrum stable enough to compare from one week to the next.

The spectrogram

Real signals are often non-stationary. A machine speeds up, a market changes regime, a heart rate rises. A spectrogram computes spectra in short sliding windows and draws them side by side: time on one axis, frequency on the other, power as colour. A change of rhythm appears as a shift in the band. The display is familiar from speech analysis and bird-song recordings, and music-recognition services identify a recording by matching the pattern of peaks in its spectrogram against a database.

The figure below illustrates all three ideas with an AR(2) process of adjustable period and persistence.

Suggested settings:

  1. Periodogram variability. With the defaults (period 10, r=0.9r = 0.9), the grey raw periodogram scatters by about 1.25 natural-log units around the dashed truth, while the 8-segment Welch average scatters by about 0.35. Press "New 1,024 samples": the raw ordinates change each time, while the Welch curve changes little.
  2. Resolution and variance. Set the segments to 1: Welch becomes a single tapered periodogram, as noisy as the raw one. Set it to 16: very smooth, but the peak broadens because each segment is only 64 samples long.
  3. Persistence. Increase rr to 0.99: the true peak becomes a spike. Reduce it to 0.5: the peak almost disappears, though the period setting has not changed. A pseudo-cycle needs persistence to be visible.
  4. A change of period. Turn on "Halve the period halfway through". The averaged spectrum shows two peaks, with no hint that they occur at different times. The spectrogram shows the band moving to a higher frequency halfway through.

Variance of the periodogram. For Gaussian white noise with variance σ2\sigma^2, write the Fourier sum at fjf_j as A+iBA + iB. AA and BB are linear combinations of independent normals, uncorrelated with each other, each with variance nσ2/2n\sigma^2/2. So I(fj)=(A2+B2)/nI(f_j) = (A^2 + B^2)/n is σ2/2\sigma^2/2 times a chi-squared variable with 2 degrees of freedom: an exponential with mean σ2\sigma^2, regardless of nn. The log of an exponential has standard deviation π/6≈1.28\pi/\sqrt 6 \approx 1.28, which is the scatter the widget measures. Averaging KK independent segments gives a χ2K2/(2K)\chi^2_{2K}/(2K) shape, whose log has standard deviation ψ1(K)\sqrt{\psi_1(K)}, about 0.35 for K=8K = 8 (ψ1\psi_1 is the trigamma function). For non-white stationary processes the same holds asymptotically, ordinate by ordinate.

A spectrum is a summary of memory, as the autocorrelation function is; Wiener and Khinchin proved they carry the same information. The spectrum is the better view when the question is about rhythm, and the worse one when the series is short, because a rhythm needs many repetitions before it can be measured. In either case the noise level should be reported with the estimate; otherwise readers may infer cycles that are not present.

References

  1. A. Schuster, "On the investigation of hidden periodicities with application to a supposed 26 day period of meteorological phenomena", Terrestrial Magnetism, 3(1), 13-41, 1898.
  2. N. Wiener, "Generalized harmonic analysis", Acta Mathematica, 55, 117-258, 1930.
  3. A. Khintchine, "Korrelationstheorie der stationären stochastischen Prozesse", Mathematische Annalen, 109, 604-615, 1934.
  4. M. S. Bartlett, "Smoothing periodograms from time-series with continuous spectra", Nature, 161, 686-687, 1948.
  5. P. D. Welch, "The use of fast Fourier transform for the estimation of power spectra: a method based on time averaging over short, modified periodograms", IEEE Transactions on Audio and Electroacoustics, 15(2), 70-73, 1967.
  6. D. B. Percival and A. T. Walden, Spectral Analysis for Physical Applications, Cambridge University Press, 1993.