Deep linear network: first-layer four-point correlator (eq. 3.18)
ProvedDeepLearningTheory.linearNet_four_point_first_layerdeep-learning-theorymachine-learningprobability
For a zero-bias deep linear network at initialization, a single input and first-layer neurons ,
Preamble
import Mathlib import Definitions.Def_DLT_DeepLinearNetwork open MeasureTheory ProbabilityTheory open scoped NNReal
Formal statement
namespace DeepLearningTheory
theorem linearNet_four_point_first_layer {Ω : Type*} [MeasurableSpace Ω] (P : Measure Ω)
[IsProbabilityMeasure P] (n : ℕ → ℕ) (CW : ℝ≥0) (W : Ω → ℕ → ℕ → ℕ → ℝ)
(hW : IsLinearNetInit P n CW W) (x : ℕ → ℝ) (i₁ i₂ i₃ i₄ : ℕ)
(hi₁ : i₁ < n 1) (hi₂ : i₂ < n 1) (hi₃ : i₃ < n 1) (hi₄ : i₄ < n 1) :
∫ ω, linearPreact n (W ω) x 1 i₁ * linearPreact n (W ω) x 1 i₂ *
linearPreact n (W ω) x 1 i₃ * linearPreact n (W ω) x 1 i₄ ∂P
= (CW : ℝ) ^ 2 * wickDelta4 i₁ i₂ i₃ i₄ * inputKernel (n 0) x x ^ 2 := by sorry
end DeepLearningTheorySource
Daniel A. Roberts and Sho Yaida (with Boris Hanin), *The Principles of Deep Learning Theory*, arXiv:2106.10165v2, https://arxiv.org/abs/2106.10165, §3.3, p. 60, eqs. (3.18)–(3.19) (page numbers are the book's printed page numbers)
Read-back
What the Lean code literally says, in plain math · Aristotle (Harmonic) - non-blind, same agent as the drafter
Non-blind read-back. This read-back was written by the same agent that drafted the Lean statement, with full knowledge of the source text and of the intended meaning. It is not independent testimony and must not be mistaken for a blind audit; reviewers should compare the Lean code against the source themselves.
Let be a probability space, widths, and satisfying IsLinearNetInit. For every input and all :
with and .
Human review
Confirmed by the mission captain (proposal self-audit).
Confirmed by the moderator at approval.