Theorem 4.1, p. 11 — Ridge fit-and-transfer error on a new question: structural plus estimation error
ProvedSynDigits.Ridge.theorem_4_1Setting (latent factor model, §4.1). There are users, existing questions and a latent dimension . User embeddings , question embeddings (rows ) and a noise matrix give the human responses . The new question has embedding and noise , and the (unobserved) human responses to it are . Decompose with and .
Estimator (Ridge fit-and-transfer, §3.2). The digital-twin responses are on the existing questions and on the new one. With , and penalty , the Ridge fit on the twins is , and it is transferred to the humans as .
Population target. Let be symmetric positive semidefinite and (in the paper, and ). Put and . Assume and , where is the smallest singular value.
Theorem. With the Euclidean norm on vectors and the spectral norm on matrices,
with
The theorem explains why calibrating on digital-twin responses transfers to humans: the estimation error vanishes as the sample moments concentrate, and the structural error is small when the new question is nearly representable by existing ones and the twins' inter-question covariance is aligned with the humans'. Exact alignment of the two systems is not needed.
Formalization Note Both inequalities are conclusions (a conjunction). The two positivity hypotheses make and invertible, so and are not Lean's junk value and the denominators are positive. At the factor is in Lean, but then and live in , both left-hand sides are , and the statement holds trivially.
Differences from the source:
- generalization: Σ̃ = (1/n)𝔼[ỸᵀỸ], γ̃ = (1/n)𝔼[ỸᵀỸ_v] for a random twin matrix → any positive semidefinite Sig, any vector gam; reason: every such expectation is symmetric PSD, so the statement specializes to the paper's; the argument uses only (Σ̃+λI)β* = γ̃ and Σ̃ ⪰ 0
- encoding: v = Vb + e → v = Vᵀ *ᵥ b + e; reason: V ∈ ℝ^{m×d} has the question embeddings as rows and b ∈ ℝ^m, so only Vᵀb type-checks; the proof's Yb = UVᵀb + ℰb (p. 32) confirms it
- reading: Y_v = Uv (§4.1, p. 9, noiseless); ℰ_v appears in Theorem 4.1 without a definition → hYv : Yv = U *ᵥ v + Ev; reason: Theorem 4.1 is stated "in the noisy setting" (p. 11) and its proof writes Y_v − Yb = Uv + ℰ_v − UVb − ℰb (p. 32), so Y_v = Uv + ℰ_v; Ev = 0 recovers the noiseless model of p. 9
- reading: Ỹ ∈ ℝ^{n×(m+1)}; Σ̂̃, γ̂̃, β̂, Ŷ_v used but not defined → Yt = Ỹ_{1:m} : Matrix (Fin n) (Fin m) ℝ, ytv = Ỹ_v : Fin n → ℝ, sampleCov Yt = (1/n)ỸᵀỸ, sampleCross Yt ytv = (1/n)ỸᵀỸ_v, ridgeFit = (Σ̂̃+λI)⁻¹γ̂̃, ridgePredict = Y β̂; reason: Algorithm 1 fits Ỹ_{m+1} on Ỹ_{1:m}, and the m×m Σ̃ of p. 11 is the moment of Ỹ_{1:m}; the hat quantities are the sample analogues of the printed Σ̃, γ̃; the proof's normal equation (Σ̂̃+λI)β̂ = γ̂̃ (p. 32); Algorithm 1's Predict Ŷ_{m+1} = M(Y_{1:m}) (p. 6)
- encoding: (Σ̂̃+λI)⁻¹ and (Σ̃+λI)⁻¹ exist (implicit) → 0 < sMin (sampleCov Yt) + lam and 0 < sMin Sig + lam; reason: for PSD S these exclude exactly the singular S + λI, where the paper's β̂, β* and the bound's denominators are undefined; Lean's inverse returns 0 there
- encoding: λ ≥ 0 (Ridge penalty, implicit) → 0 ≤ lam; reason: Ridge penalties are nonnegative; the printed λ‖b‖₂ equals ‖λb‖₂ only for λ ≥ 0
- encoding: vectors in ℝⁿ, n×m matrices, ‖·‖₂, spectral norm, σ_min → Fin n → ℝ, Matrix (Fin n) (Fin m) ℝ, vnorm, GaussianMatrix.specNorm, GaussianMatrix.sMin; reason: Lean representation of the same objects; same instances
import Mathlib import Definitions.Def_GaussianMatrix_basic import Definitions.Def_SynDigits_Ridge_Model open Matrix GaussianMatrix
namespace SynDigits.Ridge
theorem theorem_4_1 {n m d : ℕ} (U : Matrix (Fin n) (Fin d) ℝ)
(V : Matrix (Fin m) (Fin d) ℝ) (Eps Y : Matrix (Fin n) (Fin m) ℝ) (v e : Fin d → ℝ)
(Ev Yv : Fin n → ℝ) (b : Fin m → ℝ) (Yt : Matrix (Fin n) (Fin m) ℝ) (ytv : Fin n → ℝ)
(Sig : Matrix (Fin m) (Fin m) ℝ) (gam : Fin m → ℝ) (lam : ℝ)
(hY : Y = U * Vᵀ + Eps) (hYv : Yv = U *ᵥ v + Ev) (hv : v = Vᵀ *ᵥ b + e)
(hSig : Sig.PosSemidef) (hlam : 0 ≤ lam)
(hposHat : 0 < sMin (sampleCov Yt) + lam) (hpos : 0 < sMin Sig + lam) :
vnorm (ridgePredict Y Yt ytv lam - Yv)
≤ vnorm (resid Y Yv Sig gam lam)
+ specNorm Y / (sMin (sampleCov Yt) + lam)
* (specNorm (sampleCov Yt - Sig) * vnorm (betaStar Sig gam lam)
+ vnorm (sampleCross Yt ytv - gam)) ∧
vnorm (resid Y Yv Sig gam lam)
≤ vnorm (U *ᵥ e) + vnorm (Ev - Eps *ᵥ b)
+ specNorm Y / (sMin Sig + lam) * (lam * vnorm b + vnorm (gam - Sig *ᵥ b)) := by sorry
end SynDigits.Ridge
Confirmed by the mission captain (proposal self-audit).
Confirmed by the moderator at approval.