Jorge Quitério Search
Mathematics

Estimating the shape parameter of the γ-order generalized normal

Fitting the γ-order generalized normal requires estimating its shape parameter, which has no closed form. A simulation shows how much data this requires, why large values of γ are poorly determined, and which parametrisation should be used instead.

The appeal of a family like the γ-order generalized normal (γGN) is that the data choose the shape of the tails instead of the analyst assuming it. This is only useful if the shape can be estimated, and the tails are, by definition, the part of a distribution where data are scarce.

This note asks how well the shape can be estimated by maximum likelihood, how much data it takes, and why for some true values the estimate of γ appears unstable, changing sign in about a quarter of cases, although the estimation itself is sound. A simulation of repeated estimations shows both effects.

The relevant literature is as follows.

Kitsos, Vassiliadis and Toulias (2014) derived maximum-likelihood estimation for all three γGN parameters (location, scale, shape) and showed that the shape equation has no closed form.

Agrò (1995) studied numerical maximum likelihood for the exponential power family, which has the same univariate shape, as explained in the exponential power family and the γ-order generalized normal.

On the numerical side, Nelder and Mead (1965) proposed the derivative-free simplex method; Byrd, Lu, Nocedal and Zhu (1995) the bound-constrained L-BFGS-B; and Nocedal and Wright (2006) cover quasi-Newton methods in general.

What has a closed form and what does not

Write the univariate γGN with location μ\mu, scale σ\sigma and shape γ\gamma, and let β=γ/(γ−1)\beta = \gamma/(\gamma-1) be the exponent in its density. If γ\gamma and μ\mu are known, the maximum-likelihood estimate of the scale is explicit:

σ^=(1n∑i=1n∣Xi−μ∣β)1/β, \hat\sigma = \left(\frac{1}{n}\sum_{i=1}^{n} |X_i - \mu|^{\beta}\right)^{1/\beta} ,

a power mean of the absolute deviations, with the familiar root mean square at γ=2\gamma = 2. The shape is a different matter. The likelihood equation for γ\gamma has no closed-form solution, so it is solved numerically: Nelder-Mead needs no derivatives and is robust, BFGS is faster on a smooth likelihood, and L-BFGS-B handles the bounds a scale parameter needs. Several starting values provide inexpensive protection against a local optimum.

Required sample size

I simulated samples from the γGN, fitted the shape and the scale by maximum likelihood with the location known (Nelder-Mead from three starting points), and looked at the spread of the estimates over 150 to 300 replications. Estimating the location as well would only add uncertainty, so the ranges below are, if anything, optimistic.

For a moderately heavy-tailed member, γ=4\gamma = 4:

nn 90% range of γ^\hat\gamma 90% range of β^\hat\beta (true 1.333)
250 2.4 to 9.3 1.11 to 1.73
1,000 3.0 to 5.7 1.21 to 1.51
5,000 3.5 to 4.7 1.27 to 1.40

With 250 observations, about a year of daily data, the estimate is compatible with anything from mildly to strongly heavy-tailed. For a risk manager working with a year of daily returns, the two ends of that range mean that a move of more than three standard deviations occurs about once a year at γ=2.4\gamma = 2.4 and about three times a year at γ=9.3\gamma = 9.3. The data cannot say which, and the amount of capital held against such days depends on the answer. It takes thousands of observations to pin the shape down. Tails are, by definition, where the data are scarce.

Large values of γ

For a very heavy-tailed member, γ=25\gamma = 25, the picture changes:

nn 90% range of γ^\hat\gamma Share of fits with γ^<0\hat\gamma < 0 90% range of β^\hat\beta (true 1.042)
250 −71 to 47 34% 0.89 to 1.34
1,000 −67 to 143 23% 0.95 to 1.14
5,000 11 to 81 3% 1.00 to 1.09

Even with a thousand observations, almost a quarter of the estimates have the wrong sign. This appears problematic, but the corresponding estimates of β\beta are well behaved and centred on the true value.

The explanation is the map between the two parameters. As γ\gamma grows, β=γ/(γ−1)\beta = \gamma/(\gamma-1) approaches 1, the Laplace member. Values of γ\gamma of 25 and 50 correspond to β\beta of 1.042 and 1.020, two almost identical curves. And β\beta slightly below 1 corresponds to large negative γ\gamma. The Laplace distribution sits at γ=±∞\gamma = \pm\infty, so an estimate of β\beta that wanders across 1, as it must when the truth is close to 1, sends γ^\hat\gamma from large positive to large negative values. The likelihood has not changed much; the coordinate has.

Choice of parametrisation

The remedy is to estimate, and report uncertainty, in a parameter that is smooth through the Laplace member. Two natural choices:

  • the exponent β=γ/(γ−1)\beta = \gamma/(\gamma-1), which is the parameter of the exponential power family; or
  • the reciprocal δ=1/γ=1−1/β\delta = 1/\gamma = 1 - 1/\beta, which is 0 at the Laplace member, 1/2 at the normal and approaches 1 at the uniform end, with negative values for the very heavy tails.

Both pass smoothly through the point where γ\gamma jumps. Confidence intervals built on either can then be translated back to γ\gamma if needed, accepting that an interval in δ\delta containing 0 corresponds to a set of γ\gamma values that wraps through infinity.

Further remarks

  • The scale is not the standard deviation. For the γGN, σ\sigma is a scale parameter. The standard deviation is σ\sigma times a factor that depends on γ\gamma: it equals 1 at γ=2\gamma = 2, is about 1.19 at γ=4\gamma = 4 and approaches 2≈1.41\sqrt{2} \approx 1.41 at the Laplace end. Comparing a fitted σ^\hat\sigma with a sample standard deviation without the factor mixes two different quantities.
  • Report the uncertainty with the estimate. A point estimate of γ=25\gamma = 25 from a year of data says very little on its own. A confidence interval in β\beta or δ\delta shows how much the data support it.

The figure below repeats the estimation. Near the Laplace member, estimates of β are well behaved while the same estimates expressed as γ scatter wildly and often change sign; and the shape needs thousands of observations to be pinned down.

Each run draws 300 samples from the γGN (location known), finds the maximum-likelihood exponent β with the scale profiled out, and shows the results in both coordinates.

Suggested settings:

  1. Large γ. With γ = 25 and 1,000 observations, the β estimates form a tidy bell around 1.04, roughly between 0.95 and 1.14. The same estimates as γ run from about −90 to well over +80, and between a fifth and a quarter of them are negative.
  2. Moderate γ. Set γ = 4. Now β sits around 1.33, far from 1, and the γ estimates are a reasonable, if skewed, cloud around 4. No estimate has the wrong sign.
  3. Smaller sample. At γ = 4 with 250 observations, the 90% range of β is roughly 1.11 to 1.73, which in γ means anything from about 2.3 to 10: from mildly to strongly heavy-tailed. A year of daily data cannot tell those apart.
  4. Larger sample. Increase the sample size to 5,000. Both ranges tighten, roughly like 1/n1/\sqrt{n}, and the negative γ estimates almost disappear.

Effect of the parametrisation. For known location 0 and exponent β\beta, the scale that maximises the likelihood is s^β=βn∑∣xi∣β\hat s^\beta = \frac{\beta}{n}\sum |x_i|^\beta (in the parametrisation f∝exp⁡(−∣x/s∣β)f \propto \exp(-|x/s|^\beta)). Substituting it gives the profile log-likelihood

ℓp(β)=n[ln⁡β−ln⁡2−ln⁡Γ(1/β)]−nβln⁡s^β−nβ, \ell_p(\beta) = n\left[\ln\beta - \ln 2 - \ln\Gamma(1/\beta)\right] - \frac{n}{\beta}\ln\hat s^{\beta} - \frac{n}{\beta},

a smooth function of β\beta that the widget maximises by golden-section search. The estimate β^\hat\beta is approximately normal around the truth with a standard error of order 1/n1/\sqrt{n}. Transforming it, γ^=β^/(β^−1)\hat\gamma = \hat\beta/(\hat\beta - 1), and by the delta method

se⁡(γ^)≈∣dγdβ∣se⁡(β^)=se⁡(β^)(β−1)2. \operatorname{se}(\hat\gamma) \approx \left|\frac{d\gamma}{d\beta}\right| \operatorname{se}(\hat\beta) = \frac{\operatorname{se}(\hat\beta)}{(\beta - 1)^2} .

At β=1.042\beta = 1.042 (γ=25\gamma = 25) the factor is about 570. Worse, whenever the interval for β\beta contains 1, the map sends part of it to +∞+\infty and part to −∞-\infty. The likelihood has barely changed; only the coordinate has.

Summary:

  • Expect to need thousands of observations to pin down the tail shape. A year of daily data gives a range, not an answer.
  • Estimate and report in a smooth coordinate: the exponent β\beta, or δ=1/γ=1−1/β\delta = 1/\gamma = 1 - 1/\beta, which is 0 at the Laplace member and 1/2 at the normal. Translate back to γ only at the end, accepting that an interval containing δ=0\delta = 0 wraps through infinity.
  • Always give an interval. A point estimate of γ=25\gamma = 25 says very little on its own.

References

  1. J. A. Nelder and R. Mead, "A simplex method for function minimization", The Computer Journal, 7(4), 308-313, 1965.
  2. C. P. Kitsos, V. G. Vassiliadis and T. L. Toulias, "MLE for the γ-order generalized normal distribution", Discussiones Mathematicae Probability and Statistics, 34(1-2), 143-158, 2014.
  3. G. Agrò, "Maximum likelihood estimation for the exponential power function parameters", Communications in Statistics to Simulation and Computation, 24(2), 523-536, 1995.
  4. R. H. Byrd, P. Lu, J. Nocedal and C. Zhu, "A limited memory algorithm for bound constrained optimization", SIAM Journal on Scientific Computing, 16(5), 1190-1208, 1995.
  5. J. Nocedal and S. J. Wright, Numerical Optimization, 2nd ed., Springer, 2006.