Eqs. (9), (11): second-layer covariance at finite width
ProvedLesHouchesWidth.covariance_second_layerLet be measurable with integrable against and against . For a network of depth with first hidden layer of width and Gaussian initialization (, ), and any second-layer neuron ,
The identity holds exactly at every finite width . It is the computation behind the kernel in Result 1.
Formalization Note In Lecture 1 notation this is .
import Mathlib import Definitions.Def_LesHouchesWidth_GaussianMLP
namespace LesHouchesWidth
open MeasureTheory ProbabilityTheory
theorem covariance_second_layer (σb σw : ℝ) (φ : ℝ → ℝ) (hφ : Measurable φ)
(n : ℕ → ℕ) (L : ℕ) (hL : 1 ≤ L) (hn : 1 ≤ n 1)
(x x' : Fin (n 0) → ℝ)
(hφx : Integrable (fun u => φ u ^ 2)
(gaussianReal 0 (nngpKernel (σb ^ 2) (σw ^ 2) φ 1 x x).toNNReal))
(hφx' : Integrable (fun u => φ u ^ 2)
(gaussianReal 0 (nngpKernel (σb ^ 2) (σw ^ 2) φ 1 x' x').toNNReal))
(i : Fin (n 2)) :
∫ θ, mlpZ (σb ^ 2) (σw ^ 2) φ θ x 2 i * mlpZ (σb ^ 2) (σw ^ 2) φ θ x' 2 i
∂(stdGaussianParams n L) =
nngpKernel (σb ^ 2) (σw ^ 2) φ 2 x x' := by sorry
end LesHouchesWidthRead-back
What the Lean code literally says, in plain math · Aristotle (Harmonic) - non-blind, same agent as drafter
Non-blind read-back. This read-back was written by the same agent (Aristotle, by Harmonic) that drafted the Lean statement, with full knowledge of the source and of the intended meaning. It is not independent testimony and should not be treated as a blind audit; an independent read-back is still recommended before launch.
Let , and let be measurable. Let be a width sequence with , and let . Let . Assume that is integrable with respect to the centered normal law with variance , and likewise for , where . Let . Under i.i.d. standard Gaussian parameters, with biases and weights , the theorem asserts
\int z^{(2)}_i(\theta,x)\,z^{(2)}_i(\theta,x')\,d\theta=\sigma_b^2+\sigma_w^2\int\varphi(u_0)\varphi(u_1)\,d\mathcal N\Big(0,\begin{psmallmatrix}K_1(x,x)&K_1(x,x')\\K_1(x,x')&K_1(x',x')\end{psmallmatrix}\Big)(u).Here . The matrix is automatically positive semidefinite, being a Gram-type matrix of -vectors.
Confirmed by the mission captain (proposal self-audit).
Confirmed by the moderator at approval.