Motivation
Concentration of measure in a product space ΩN asks how much of the space lies close to a set A of non-negligible probability. In the simplest setting closeness is the Hamming distance: the number of coordinates of x that cannot be matched by a point of A. Talagrand's 1995 memoir in Publications Mathématiques de l'IHÉS develops a family of such inequalities with one method, induction over the number of coordinates, and applies them to combinatorial optimization, random graphs, spin glasses and sums of Banach-space-valued random vectors.
Counting a missed coordinate as 1 ignores how far apart the two coordinate values are. Sections 2.4 and 2.5 of the memoir replace the unit cost by a penalty h(xi,yi) that depends on the values being matched. The resulting bounds have the shape of Bernstein's inequality: a Gaussian tail for moderate deviations, governed by the L2 size of the penalty, and an exponential tail for large ones, governed by its sup norm. Theorem 2.4.1 is used again in Chapter 11 of the memoir, Corollary 2.4.4 in the study of the Sherrington–Kirkpatrick model in Chapter 12, and Theorem 2.5.1 in Section 11.5.
Setting
Let (Ω,μ) be a probability space, N≥0 an integer, and P=μ⊗N the product probability on ΩN. A penalty is a measurable function h:Ω×Ω→[0,∞) with h(ω,ω)=0 for every ω (the paper's standing assumption (2.4.2)). For A⊆ΩN and x∈ΩN the penalized distance is
fh(A,x)=inf{i≤N∑h(xi,yi)1{xi=yi};y∈A},
with fh(∅,x)=+∞. For h=1 off the diagonal this is the Hamming distance. Theorem 2.4.1 also uses the symmetrized penalty v(ω,ω′)=max(h(ω,ω′),h(ω′,ω)).
Section 2.5 measures the penalty against sets: for B⊆Ω,
h(ω,B)=inf{h(ω,ω′);ω′∈B},
the infimum being over the second variable, as in fh. The one-coordinate steps of the two inductions use the transforms g^(x)=infy(g(y)+th(x,y)) (2.4.5) and g^(x)=infs>0s+th(x,Bs) with Bs={g≤s} (2.5.7), for a function g≥0 on Ω.
Formalization targets
Goal: Theorem 2.4.1
For each measurable A⊆ΩN and each t>0 with ∬expth(x,y)dμ(x)dμ(y)<∞,
∫ΩNetfh(A,x)dP(x)≤P(A)1(21∫Ω2(etv(ω,ω′)+e−tv(ω,ω′))dμ(ω)dμ(ω′))N.
It is the most general exponential-moment bound of the section; every tail bound of Section 2.4 is derived from it.
Milestones
- Proposition 2.4.2, the one-coordinate inequality ∫eg^dμ∫e−gdμ≤21∬(etv+e−tv)dμdμ for g≥0.
- Theorem 2.4.3: if ∬eh<∞, then for 0≤t≤1
∫etfh(A,x)dP(x)≤P(A)1exp(Nt2∬(eh+e−h−2)dμdμ).
- Corollary 2.4.4: if ∬eh≤2, then P(fh(A,⋅)≥u)≤P(A)−1e−u2/4N for 0≤u≤2N.
- Corollary 2.4.5: if ∥h∥∞=suph<∞, then
P(fh(A,⋅)≥u)≤P(A)1exp(−min(8N∥h∥22u2, 2∥h∥∞u)).
- Proposition 2.5.2, the one-coordinate inequality ∫eg^dμ∫e−gdμ≤e3t2 for the transform (2.5.7), under condition (2.5.2) below.
- Theorem 2.5.1: if ∫Ωexp2h(x,B)dμ(x)≤e/μ(B) for each B⊆Ω (condition (2.5.2)), then for 0≤t≤1
∫etfh(A,x)dP(x)≤P(A)e3t2N.
Significance
The results. Theorem 2.4.1 turns a single integrability condition on the penalty into a bound on the exponential moment of the distance to any set, uniform over all sets of a given probability and factorized over coordinates. Chebyshev's inequality then gives the Bernstein-type tails of Corollaries 2.4.4 and 2.4.5, which hold for every set A, not only for half-spaces or for sums of independent variables. Theorem 2.5.1 replaces exponential integrability of h by condition (2.5.2) on the set functional h(x,B), which is much weaker: when Ω is itself a product of m spaces, m−1/2 times the Hamming distance on Ω satisfies (2.5.2) but not (2.4.12). Applied to blocks of coordinates, it shows that the coordinates a typical point misses in A are concentrated in few blocks.
Formalizing them. All six results are proved in the memoir with short arguments. None of them has a machine-checked proof. A complete development would give a general-alphabet version of Talagrand's induction on product measures, written once for a cost function, that later chapters (the q-point control, the convex-hull distance) also rely on.
Difficulty
The induction over N requires a one-coordinate inequality whose right-hand side does not depend on the set. The difficulty is that the natural one-coordinate quantity, a section measure P(A(ω)), can vanish on part of Ω, so the function g=−logP(A(ω)) takes the value +∞, and a statement proved for finite g must be extended by truncation. The other subtle point is measure theoretic: fh(A,⋅) need not be measurable even when A is, and the paper's convention of ignoring measurability has to be matched by explicit hypotheses. In Section 2.5 the integrand x↦h(x,B) in condition (2.5.2) may be non-measurable for the sets the proof uses, and Fubini's theorem is applied to the function (x,y)↦h(x,Bg(y)).
Formalization scope
Points of ΩN are functions Fin N → Ω (coordinates indexed from 0), P is Measure.pi (fun _ => μ) for a probability measure μ, and double integrals are over μ.prod μ. The penalty is a real function h : Ω → Ω → ℝ with explicit hypotheses h≥0, h(ω,ω)=0 and joint measurability, in every statement. The distance fh is defined by (2.4.1) with the indicator and takes values in [0,∞], so fh(∅,x)=+∞; likewise h(ω,B) and the transform (2.5.7). All integrals are Lebesgue integrals of [0,∞]-valued functions, e+∞=+∞, and 1/P(A)=+∞ when P(A)=0.
Measurability follows the paper's convention (pp. 81–82) as explicit hypotheses: A is measurable and x↦fh(A,x) is measurable; in the propositions, g and g^ are measurable. Condition (2.5.2) is stated for every subset B with the upper integral ∫∗, which the paper prescribes when measurability fails. In Corollary 2.4.5, the quotients are computed in [0,∞], so a quotient by ∥h∥2=0, ∥h∥∞=0 or N=0 is +∞ when its numerator is positive. The bound is +∞ when P(A)=0, as the printed factor 1/P(A) intends. No statement is restricted to a finite or countable Ω: restricting it would turn the general theorems into their finite-alphabet special cases. Dropping h(x,x)=0 or defining the distance by a real infimum (whose value on the empty set is 0) would change the theorems and is ruled out.
A complete development needs product measures on Fin (N+1) → Ω split as ΩN×Ω, sections of measurable sets, Tonelli's theorem, and the elementary convexity inequalities used in the proofs; the induction scaffold is reusable for the other distance functionals of the memoir. Proofs of any milestone are welcome, as are proofs of the extension of Proposition 2.4.2 to g with values in [0,∞], which the paper states without number on p. 92.
Selected references
- M. Talagrand, Concentration of measure and isoperimetric inequalities in product spaces, Publications Mathématiques de l'IHÉS 81 (1995), 73–205. https://doi.org/10.1007/BF02699376
- M. Talagrand, A new look at independence, Annals of Probability 24 (1996), 1–34. https://doi.org/10.1214/aop/1042644705
- M. Ledoux, The Concentration of Measure Phenomenon, Mathematical Surveys and Monographs 89, AMS, 2001. https://doi.org/10.1090/surv/089