A single active hidden unit forces the ramped output to become positive.
ProvedGrokkingVector.netRamp_eventually_posaether-catalogmachinelearning
A single active hidden unit forces the ramped output to become positive.
theorem GrokkingVector.netRamp_eventually_pos{m d : ℕ} (W : Fin m → Fin d → ℝ) (b a : Fin m → ℝ)
(c : ℝ) (p : Fin d → ℝ) (ha : ∀ j, 0 ≤ a j) {j₀ : Fin m} (ha₀ : 0 < a j₀)
(hs₀ : 0 < signal W p j₀) : ∃ T : ℝ, 0 < netRamp W b a c p T := by sorry
Formalization Note Transplanted verbatim from the Aether Catalog source MachineLearning/GrokkingDelayedTransition/VectorMargin.lean; the statement is byte-identical to the source declaration, elaborated with autoImplicit disabled in the platform environment.
Preamble
-- Thm stub generated from MachineLearning/GrokkingDelayedTransition/VectorMargin.lean import Mathlib import Definitions.Def_MachineLearning_GrokkingDelayedTransition_VectorMargin /-! # Delayed margin positivity for vector-valued two-layer ReLU networks The catalog file `Catalog/MachineLearning/GrokkingPhaseTransition.lean` treats a width-one *scalar* network. This file carries out Future Direction 1 (finite hidden width, matrix weights, a finite test set, and a classification margin) and Direction 3 (train/test separation). Main results. * `exists_sharp_threshold`: any monotone continuous signal that starts negative and is eventually positive has a *sharp* threshold `τ`: it is `≤ 0` on `(-∞, τ]` and `> 0` on `(τ, ∞)`. The threshold is unique (`sharp_threshold_unique`). * `margin_sharp_threshold`: the same holds for the *minimum* over a finite test set of finitely many such signals — the delayed transition survives taking a worst-case margin over a test set. * `netMargin_delayed_positivity`: for a genuinely vector-valued two-layer ReLU network (hidden width `m`, input dimension `d`, matrix weights, negative output bias, a finite two-class test set) the classification margin is nonpositive up to an explicit delay and strictly positive afterwards. * `grokking_window_eq`: for a concrete dataset the set of times at which the training set is already perfectly classified while the test point is still misclassified is *exactly* the interval `(1/2, 2]` — a formal train/test separation window. -/ open GrokkingVector open Finset Set /-! ### Sharp thresholds for monotone continuous signals -/ /-! ### Worst-case margin over a finite test set -/ /-! ### Vector-valued two-layer ReLU networks -/
Formal statement
theorem GrokkingVector.netRamp_eventually_pos{m d : ℕ} (W : Fin m → Fin d → ℝ) (b a : Fin m → ℝ)
(c : ℝ) (p : Fin d → ℝ) (ha : ∀ j, 0 ≤ a j) {j₀ : Fin m} (ha₀ : 0 < a j₀)
(hs₀ : 0 < signal W p j₀) : ∃ T : ℝ, 0 < netRamp W b a c p T := by sorrySource