Theorem 2 — central limit theorem for the TPG estimator and consistency of the nonstationary HAC variance estimator
ProvedTPG.theorem_2_clt_and_hacSetting. is a finite state space and the action is control or treatment. Time is , which is the source's time . Under action at time the state moves according to a transition matrix , i.e. ; the matrices may depend on arbitrarily. The initial state has distribution , and the reward has distribution on , with for all and conditional independence from everything else given . Under the policy every action is treatment with probability , independently of everything else; , and refer to the law of the trajectory under .
Asymptotic setting. The truncation size is fixed and the horizon ; for each the trajectory is generated under the experimental policy . Assumption 1 holds for all times with one constant . Write and
Assumption 2 (asymptotic variance): as , with .
Assumption 3 (Cesàro- mean stability): with , .
The bandwidth is a deterministic integer sequence with for every , and , and .
With , the nonstationary HAC variance estimator is
Theorem. Under Assumptions 1, 2 and 3, as ,
where is the TPG estimator.
The theorem gives an estimated standard error for the TPG estimator from a single nonstationary trajectory, and hence confidence intervals for .
Formalization Note For each the statistics are computed under the horizon- law. Convergence in distribution is stated as convergence of for every bounded continuous , and convergence in probability as for every . The limit is a parameter constrained by Assumption 2.
import Mathlib import Definitions.Def_TPG_Model import Definitions.Def_TPG_Estimators import Definitions.Def_TPG_Inference open MeasureTheory ProbabilityTheory open scoped BigOperators ENNReal
namespace TPG
theorem theorem_2_clt_and_hac {S : Type} [Fintype S] [MeasurableSpace S] [MeasurableSingletonClass S]
(E : Env S) (k : ℕ) (γ M v : ℝ) (L : ℕ → ℕ)
(hγ0 : 0 < γ) (hγ1 : γ < 1) (hmix : ∀ T : ℕ, E.Mixing T γ) (hM : E.RewardBound M)
(hvar : E.AsymptoticVariance k v) (hstab : E.MeanStability k) (hL : IsBandwidth L) :
E.TendstoNormal
(fun (T : ℕ) (ω : Traj S T) =>
Real.sqrt (T : ℝ) *
(tpgEst (1 / 2) T k ω - ∫ ω', tpgEst (1 / 2) T k ω' ∂(E.expLaw T (1 / 2))))
v
∧ E.TendstoInProb (fun (T : ℕ) (ω : Traj S T) => hacEst T k (L T) ω) v := by sorry
end TPGRead-back
What the Lean code literally says, in plain math · claude-opus-5-5
Data quantified over. The statement is asserted for every choice of the following.
- A set with finitely many elements, carrying a -algebra in which every one-point set is measurable (as is finite, every subset of is then measurable). Non-emptiness of is not listed as an assumption, but the datum below cannot exist on an empty set.
- An environment on , i.e. a package consisting of:
- for every time and every , a real matrix all of whose entries are and each of whose rows sums to ;
- a function with for every and ;
- for every and , a Borel probability measure on (the package itself imposes no moment or support condition on it).
- A natural number (any value, included; one fixed number, not varying with ).
- Real numbers , , .
- A function .
Sample space and probability measure, one for each . For each let
the set of sequences , with the product -algebra (the given -algebra on , all subsets of , Borel sets of ). Write , , for the coordinates at time .
For and define the weight
(The transition from time to time uses the matrix indexed by the departure time and by the value . The last product comes from a general recipe "factor if is true, if false", used here with for all . Empty products are : for the weight is , and the factor is present only when .) The measure on is the finite mixture of product measures
The clipping is in the definition; here every weight is already and the weights sum to , so is a probability measure. In words: has law ; each is a fair coin flip independent of everything generated before it and of ; is drawn from row of ; and, given all the 's and 's, the are independent with . For , is a single point and is the unit mass on it. Write for integration against .
Statistics on . Throughout, is read as when (so every statistic below is at ), all integrals are Bochner integrals (value if the integrand is not integrable), and means computed in and then converted to a real number with sent to .
Two sign weights occur. The first is if is true and if is false, evaluated at : so , . The second is : so , .
The centred and scaled statistic in the first conclusion is built from
where the inner sum runs over the times with (so it is cut off at the horizon ), and the centring constant is the -expectation of itself.
The second conclusion and two of the hypotheses are built from, for ,
(the sum is over the indices with ), together with
(the sum is over pairs of indices with ; it is empty, hence , when ; the normalisation is at every lag), and
The deterministic quantities used in the hypotheses are
Hypotheses. All of the following are assumed.
- .
- .
- (Contraction of every transition matrix.) For every and every (equivalently, for every ; the instance is empty), every , and every pair of functions that are each nonnegative with entries summing to :
- (Bounded rewards.) For every and every : . Here is an arbitrary real number; if the interval is empty and this hypothesis cannot hold.
- (Limit variance.) , and as .
- (Stability of the means.) as ; explicitly, for every real there is such that for all . (The power is a real power; its value at is by convention, which plays no role in the limit.)
- (Conditions on .) All three of: for every (nothing is required of ; in particular ); as ; and , i.e. there are a real and a with for all .
Conclusion. Both of the following hold.
(i) For every bounded continuous function ,
where is the Gaussian measure on with mean and variance , defined to be the point mass at when . Under hypothesis 5, .
(ii) For every real ,
Literal behaviours to note.
- The limits are along , , and both the sample space and the measure change with : (i) is convergence of the laws of under , tested against bounded continuous functions, and (ii) is a statement about -probabilities, each computed on its own space.
- The numbers and occur only in hypotheses 1 to 4, not in the conclusion. The number in both conclusions is the same as in hypothesis 5.
- The two sign weights and take the same values, so exchanging the order of the finite double sum gives and pointwise on ; consequently, for and when the integrals involved are finite, equals the variance of under . (This identity follows from the definitions; it is not written in the statement.)
- Boundary effects are as written: for the inner sum in has fewer than terms, and for the sum defining has rather than terms. Each is centred at the overall average and divided by , not by . When the sum over is empty and .
- The transition matrices with do not enter .
Confirmed by the mission captain (proposal self-audit).