The Tight Constant in the Dvoretzky–Kiefer–Wolfowitz Inequality I: P(√n sup(F − F̂ₙ) > λ) ≤ exp(−2λ²) for λ ≥ min(√(log 2/2), 1.0841 n^(−1/6))Research Paper
Motivation
The Kolmogorov–Smirnov statistics measure how far the empirical distribution of a sample lies from a hypothesised distribution. They are the standard goodness-of-fit test for continuous data, and their tail probabilities are the basic tool for confidence bands around a distribution function. A test is only usable at a fixed sample size if those tail probabilities are controlled for every , not just in the limit.
Timeline.
- 1933–1944. Kolmogorov and Smirnov introduce the statistics, prove that their laws do not depend on the continuous distribution, and find the limit laws, e.g. . Smirnov (1944) also gives the exact law of the one-sided statistic and the expansion .
- 1956. Dvoretzky, Kiefer and Wolfowitz prove for every , with an unspecified constant (Ann. Math. Statist. 27).
- 1958. Birnbaum and McCarty conjecture that is admissible.
- 1979–1985. Explicit constants: (Devroye and Wise), (Shorack and Wellner), (Hu).
- 1990. Massart proves the conjecture under a mild lower bound on , and deduces the two-sided bound with the optimal constant (Ann. Probab. 18).
Setting
Let () be independent, identically distributed real random variables with a continuous distribution function . The empirical distribution function is
and the centred, normalised empirical process is . The statistics are
In Lean the sample is X : Fin n → Ω → ℝ, independent (iIndepFun X P) with common law μ (HasLaw (X i) μ P), is cdf μ, and the statistics are Dplus μ X, Dminus μ X, Dtwo μ X. The parameter is written l.
The proof works with Smirnov's point probabilities , whose sum over is , with Csáki's density of the first passage of a Brownian bridge at level , and with the ratio .
Formalization targets
Goal: Theorem 1
For every and every , with ,
Here is the minimum. The constant in front of the exponential is the point of the theorem; the threshold on is the paper's.
Milestones
The milestones follow the proof in the paper's order of attack:
- the reduction to the uniform law, and Smirnov's exact formula (2.3);
- the identity (2.4);
- the local comparison of with : (2.7), Lemma 2, Claims 1–3, and Proposition 1;
- the midpoint bound for log-convex integrands (Lemma 3) and the closed forms of Lemma 4;
- the nonasymptotic expansion (2.11), the negativity of its right side , and with them Theorem 1 for , ;
- Proposition 2 on the slope of , the crude bounds (2.13)–(2.14), and the finite grid check (2.15) for .
Companions
The equality in law of and , Comment 1 ((1.4) whenever ), the reduction , and Corollary 1:
Significance
The result. Theorem 1 and Corollary 1 give non-asymptotic Kolmogorov–Smirnov bands with the asymptotic constants, and , which cannot be improved because of the limit laws. Corollary 1 is the form in which the DKW inequality is quoted throughout statistics and learning theory: confidence bands for distribution functions, sample-size calculations for quantile estimation, and as a building block in bandit and empirical-risk analyses. Comment 1 says that the one-sided test is never anti-conservative at levels up to 50%, whatever the sample size.
Formalizing it. The result is proved on paper but not machine-checked. A formal development would give a verified, constant-explicit DKW inequality that downstream statistical results can import as a black box instead of carrying an unspecified . The milestones also produce reusable pieces: Smirnov's exact law of the one-sided statistic, the Brownian-bridge first-passage integral, and a family of explicit elementary inequalities. The proof uses computer-checked numerics ((2.15), Lemma 2, ); turning those into checked certificates is part of the work.
Difficulty
The obvious route is to compare the exact sum (2.3) term by term with the integral (2.4). The local comparison is only nearly true: a corrective factor is needed, and it is largest on the tail, where the crossing time is close to . Controlling that factor uniformly in while keeping the leading constant exactly is what forces the delicate constants (, , , ). For small the analytic bounds are too weak, and the proof closes the gap with a Lipschitz bound on plus a finite numerical check over a grid; neither half suffices alone.
Formalization scope
Conventions:
- ; the sample is indexed by
Fin n; iscdf μwithContinuous (cdf μ)assumed, as in Theorem 1. - Probabilities are measures of events in and are compared with
ENNReal.ofRealof the bound. Each supremum defining a statistic is of a function with values in , hence a genuine real supremum. - uses an integer power at , so that . is defined from the exact formula (2.3); it equals by the (2.3) milestone, so the analytic milestones carry no probability.
- Statements that the page gives through derivatives of (Proposition 2(i), (iii)) are stated as monotonicity and as an increment bound, because has kinks.
Additions and corrections relative to the page, each disclosed in its item: the printed slips in (2.1) (range of ), (2.8) (swapped numerators) and Proposition 2(i) ("" for nonincreasing) are corrected; Claim 2 carries the hypothesis , without which the corrected claim is false; Lemma 3 uses strict inequalities ; Comment 1 makes the implicit explicit; (2.14) carries Proposition 2's standing range , .
The goal does not mention , , , the case split at , or the computer check; a formalization of the goal through those objects, or with the threshold read as a maximum instead of a minimum, would be a different and weaker statement.
Infrastructure a complete development needs: the empirical process of an i.i.d. sample and its reduction to the uniform law by the probability integral transform; Smirnov's exact one-sided law (a ballot-type combinatorial identity); Gaussian-type integrals on ; elementary but delicate real inequalities; and certified interval arithmetic for the numerical steps. The first two are reusable for any work on empirical processes. Contributions on any milestone are welcome, in particular formal proofs of (2.3), of the numerical steps, and of the companions.
Selected references
- P. Massart, The tight constant in the Dvoretzky–Kiefer–Wolfowitz inequality, Ann. Probab. 18(3):1269–1283, 1990. https://doi.org/10.1214/aop/1176990746
- A. Dvoretzky, J. Kiefer, J. Wolfowitz, Asymptotic minimax character of the sample distribution function and of the classical multinomial estimator, Ann. Math. Statist. 27:642–669, 1956. https://doi.org/10.1214/aoms/1177728174
- Z. W. Birnbaum, R. C. McCarty, A distribution-free upper confidence bound for P(Y<X), based on independent samples of X and Y, Ann. Math. Statist. 29:558–562, 1958.
- N. V. Smirnov, Approximate laws of distribution of random variables from empirical data, Uspekhi Mat. Nauk 10:179–206, 1944.
- E. Csáki, Investigations concerning the empirical distribution function, Magyar Tud. Akad. Mat. Fiz. Oszt. Közl. 23:239–327, 1974; English translation in Selected Transl. Math. Statist. Probab. 15:229–317, 1981.
- G. R. Shorack, J. A. Wellner, Empirical Processes with Applications to Statistics, Wiley, 1986.