Deep linear network: connected four-point correlator of two neurons (eq. 3.30)
ProvedDeepLearningTheory.linearNet_connected_four_pointFor a zero-bias deep linear network with positive widths at initialization, a single input , a layer and two distinct neurons of that layer,
where , and . The nonzero right side measures the interaction between different neurons at finite width.
import Mathlib import Definitions.Def_DLT_DeepLinearNetwork open MeasureTheory ProbabilityTheory open scoped NNReal
namespace DeepLearningTheory
theorem linearNet_connected_four_point {Ω : Type*} [MeasurableSpace Ω] (P : Measure Ω)
[IsProbabilityMeasure P] (n : ℕ → ℕ) (hn : ∀ ℓ, 0 < n ℓ) (CW : ℝ≥0)
(W : Ω → ℕ → ℕ → ℕ → ℝ) (hW : IsLinearNetInit P n CW W) (x : ℕ → ℝ)
(ℓ j k : ℕ) (hℓ : 1 ≤ ℓ) (hj : j < n ℓ) (hk : k < n ℓ) (hjk : j ≠ k) :
∫ ω, (linearPreact n (W ω) x ℓ j ^ 2 - (CW : ℝ) ^ ℓ * inputKernel (n 0) x x) *
(linearPreact n (W ω) x ℓ k ^ 2 - (CW : ℝ) ^ ℓ * inputKernel (n 0) x x) ∂P
= (CW : ℝ) ^ (2 * ℓ) * (∏ ℓ' ∈ Finset.Ico 1 ℓ, (1 + 2 / (n ℓ' : ℝ))) *
inputKernel (n 0) x x ^ 2
- ((CW : ℝ) ^ ℓ * inputKernel (n 0) x x) ^ 2 := by sorry
end DeepLearningTheoryRead-back
What the Lean code literally says, in plain math · Aristotle (Harmonic) - non-blind, same agent as the drafter
Non-blind read-back. This read-back was written by the same agent that drafted the Lean statement, with full knowledge of the source text and of the intended meaning. It is not independent testimony and must not be mistaken for a blind audit; reviewers should compare the Lean code against the source themselves.
Let be a probability space, widths with all , , satisfying IsLinearNetInit, an input, and with . Writing :
the product being empty (equal to ) when .
Confirmed by the mission captain (proposal self-audit).
Confirmed by the moderator at approval.