The logarithmic Sobolev inequality and the γ-order generalized normal
The normal distribution is the equality case of several inequalities linking entropy and Fisher information. Generalising one of them, the logarithmic Sobolev inequality, yields a family of equality cases: the γ-order generalized normal. A figure tests the inequality on a family of densities.
Earlier notes used the γ-order generalized normal (γGN) as a flexible family of error distributions, starting with the γ-order generalized normal distribution, and showed in the exponential power family and the γ-order generalized normal that in one dimension it has the same shape as the exponential power family. What distinguishes it is its origin: it emerges from an inequality.
This note explains that statement with minimal functional analysis: what the inequality says, why the normal distribution is its equality case, and what happens when one exponent is changed. A figure then tests the inequality on a family of densities and shows that only the normal attains the bound.
The relevant literature is as follows.
- Stam (1959) proved that entropy power times Fisher information is at least the dimension, with equality only for the normal.
- Gross (1975) introduced the logarithmic Sobolev inequality with respect to the Gaussian measure, and Weissler (1978) studied its Euclidean form.
- Del Pino, Dolbeault and Gentil (2004) found the optimal constant of the version, and its extremal functions.
- Kitsos and Tavoularis (2009) translated this into statistics: generalised Fisher information, generalised entropy power, and the family of equality cases now called the γGN.
- Kitsos and Stamatiou (2025) reviewed the family and argued that this grounding is what distinguishes it from other generalisations of the normal.
- Kitsos, Nyamsi and Stamatiou (2026) carried the chain from information measures to risk, extending the entropic value at risk with the γGN.
The classical Fisher information itself is the subject of Fisher information as the curvature of the log-likelihood.
The normal distribution as an equality case
The normal distribution has several extremal properties beyond the central limit theorem. Among all distributions with a given variance, it has the largest Shannon entropy. It also has the smallest Fisher information: for any density with variance ,
with equality only for the normal. This is the Cramér-Rao bound read from the other side: the normal is the hardest location family to estimate, which is one reason it is a cautious default.
A laboratory calibrating a balance shows what this means in practice. If only the variance of the balance's error is known, assuming normal errors is the cautious choice: no other error distribution with that variance makes the true weight harder to estimate. A number of repeat weighings planned under the normal assumption is therefore sufficient whatever the actual shape, because the sample mean has the same variance for every shape and an estimator adapted to the true shape can only do better.
Stam combined the two quantities in 1959. Define the entropy power of a random vector in dimensions as , where is its entropy. Then
again with equality only for the normal. Entropy and Fisher information cannot both be small, and the normal is the only distribution for which the bound is attained.
The logarithmic Sobolev inequality
Sobolev inequalities bound the size of a function by the size of its derivative. Gross introduced a logarithmic version in 1975, measured against the Gaussian measure. Its Euclidean form, studied by Weissler in 1978, states that for a function on with ,
Put , a probability density, and the left side becomes minus the entropy of , while the integral on the right becomes a quarter of its Fisher information, since . Rearranging the terms gives exactly Stam's inequality, . The logarithmic Sobolev inequality is the same statement about entropy and Fisher information, written for functions instead of distributions. The constant is the best possible, and the functions that achieve equality are Gaussians.
Generalising the exponent
The inequality measures the gradient with a square, . The square is not essential. Replace it by for some , with normalised in the matching way, and one obtains an version of the logarithmic Sobolev inequality. Del Pino, Dolbeault and Gentil found its optimal constant in 2004, together with the functions that achieve equality. They have the form
where is the exponent conjugate to . At the conjugate exponent is again 2, and the Gaussian reappears.
From functions to distributions
Kitsos and Tavoularis took this step from analysis to statistics in 2009. They defined a generalised, γ-order Fisher information,
which reduces to the classical at , together with a matching generalised entropy power. They extended Stam's inequality and the Cramér-Rao bound to these measures, and showed that the equality cases form a family of distributions, with elliptical contours and a location and scale matrix, which they called the hyper normal. It is the distribution now known as the γGN. Later work by Kitsos with Toulias and with Stamatiou developed its properties.
The shape parameter therefore has a precise meaning. It is the order of the information measure for which the distribution plays the role the normal plays for classical Fisher information.
The figure below tests the inequality and illustrates two properties: for every density in the family , entropy power times Fisher information is at least 1, and Fisher information times variance is at least 1; both touch 1 only at the normal, .
Suggested settings:
- The normal. Press "Normal". Both products read exactly 1.0000, which is equality.
- The Laplace. Press "Laplace". and variance . With heavier tails and a sharp peak, it carries more information about location than the normal of the same variance, and more entropy than the bound requires. For the balance, this means that if its errors were known to follow a Laplace law, the median of the weighings would estimate the weight with half the variance of the mean.
- Range of β. Move β from 0.6 to 8. The curve has a single minimum, at , and rises on both sides: towards the very peaked, heavy-tailed members on the left and towards the flat-topped, almost uniform ones on the right.
- No counterexample. No β makes either product fall below 1, as Stam's theorem guarantees for every density with finite Fisher information, not just this family.
Derivation of the closed forms. For with , the variable has a Gamma distribution (the same transform used to sample the γGN), so . Then:
- Entropy: .
- Fisher information: , so , finite for .
- Variance: .
With , the widget evaluates and exactly. At : , , , , and both products equal 1.
The same curves can be fitted without this background, as decades of work on the exponential power family show. The origin matters in three ways:
- The normal is a member of the family. Every result for the family should reduce to the classical one at . This provides an internal check on derivations and code.
- Information measures are natural. Entropy, Fisher information and divergences between members take closed forms on the family, because it was defined through them. That makes it convenient for questions such as "how far is a fitted error law from the normal?"
- The multivariate form is intrinsic. The inequality lives in dimensions from the start, so the family comes with an elliptical multivariate version and its dependence structure.
Whether this is a decisive advantage over other generalisations of the normal is debated; Kitsos and Stamatiou argue that it is. What is not in doubt is that the classical role of the normal in information theory has a clean generalisation, and that the γGN provides it.
References
- A. J. Stam, "Some inequalities satisfied by the quantities of information of Fisher and Shannon", Information and Control, 2(2), 101-112, 1959.
- L. Gross, "Logarithmic Sobolev inequalities", American Journal of Mathematics, 97(4), 1061-1083, 1975.
- F. B. Weissler, "Logarithmic Sobolev inequalities for the heat-diffusion semigroup", Transactions of the American Mathematical Society, 237, 255-269, 1978.
- M. Del Pino, J. Dolbeault and I. Gentil, "Nonlinear diffusions, hypercontractivity and the optimal Lp-Euclidean logarithmic Sobolev inequality", Journal of Mathematical Analysis and Applications, 293(2), 375-388, 2004.
- C. P. Kitsos and N. K. Tavoularis, "Logarithmic Sobolev inequalities for information measures", IEEE Transactions on Information Theory, 55(6), 2554-2561, 2009.
- C. P. Kitsos and I. S. Stamatiou, "On the emerged from LSI γ-order generalized normal distribution", Journal of the Indian Society for Probability and Statistics, 26(2), 737-762, 2025.
- C. P. Kitsos, U. E. Nyamsi and I. S. Stamatiou, "Extending entropic value at risk using the γ-order generalized normal distribution", Stats, 9(3), 55, 2026.