§2 — logarithmic loss is a proper scoring rule
ProvedForecastAggLearn.logLoss_properforecast-aggregationonline-learningp2o-gran-per-chapterp2o-plan-paperp2o-v1probability
Let play the role of the Bayes posterior , and for a forecast let
be the expected logarithmic loss. Then is minimized at :
This is why the benchmark of the paper is : with the signals in hand no forecast has smaller expected loss, and the regret of any aggregator is nonnegative in expectation given .
Formalization Note Losses take values in with and .
Differences from the source:
- encoding: expected loss L(r) = −r*(s) ln r − (1 − r*(s)) ln(1 − r) in ℝ → q · l(r, 1) + (1 − q) · l(r, 0) in ℝ≥0∞ with l = logLoss and q ∈ [0, 1] in place of r*(s); reason: Lean convention for ln 0 = −∞; r*(s) ranges over [0, 1]
Preamble
import Mathlib import Definitions.Def_ForecastAggLearn_logLoss
Formal statement
namespace ForecastAggLearn
/-- **Logarithmic loss is a proper scoring rule.**
Babichenko, Garber, arXiv:1802.07107v1, §2, p.10: "The aggregator's objective is to minimize his
expected logarithmic loss, which is given by L(y) = −r*(s) ln(r) − (1 − r*(s)) ln(1 − r). The
expected loss L(y) is minimized at r = r*(s), because logarithmic loss is a proper scoring rule."
For every `q ∈ [0, 1]` (the role of `r*(s)`) and every forecast `r ∈ [0, 1]`, the expected log
loss `q l(r, 1) + (1 − q) l(r, 0)` is at least its value at `r = q`.
**Formalization Note.** The expected loss is computed in `ℝ≥0∞`, with `l(0, 1) = l(1, 0) = +∞`
and `0 · ∞ = 0`. -/
theorem logLoss_proper (q : ℝ) (hq0 : 0 ≤ q) (hq1 : q ≤ 1) (r : ℝ) (hr0 : 0 ≤ r) (hr1 : r ≤ 1) :
ENNReal.ofReal q * logLoss q true + ENNReal.ofReal (1 - q) * logLoss q false ≤
ENNReal.ofReal q * logLoss r true + ENNReal.ofReal (1 - q) * logLoss r false := by sorry
end ForecastAggLearn
Source
Babichenko, Garber, Learning of Optimal Forecast Aggregation in Partial Evidence Environments, arXiv:1802.07107v1, §2, p.10
Human review
Confirmed by the mission captain (proposal self-audit).
Confirmed by the moderator at approval.