Fully Discrete Entropic Bellman Control with Finite Insurance States and Retention Actions
Insurance motivation and valuation horizon
An insurer choosing a retention or other reinsurance action in one year may affect both the immediate net claim cost and the distribution of liabilities in the following year. When the insurer values uncertainty using an exponential risk measure, the relevant policy objective applies the same conditional risk valuation recursively across policy years. A finite-horizon state-and-action model makes the timing, transition probabilities and policy optimality statement mathematically explicit.
This mission studies a finite-state, finite-action, discrete-time cost minimisation problem. Actions may represent a menu of excess-of-loss retention levels, but the formal statements allow any finite action menu and any state-dependent insurer cost schedule. It extends the cost-control ideas in prior actuarial missions into an explicitly risk-sensitive dynamic model. The result is a specialisation of finite-horizon recursive risk-sensitive Markov decision theory, with its own finite real-valued hypotheses and theorem proof obligations.
Timing, transition probabilities and discounting
Let S be the finite collection of financial, underwriting or liability states and A a finite nonempty set of insurance actions. At the beginning of a policy year the insurer observes the current state s∈S and selects an action a∈A. At the end of that year, the state becomes t∈S with conditional transition probability P(s,a,t).
We require, for each current state and action,
P(s,a,t)≥0for all t,t∈S∑P(s,a,t)=1.
An individual destination is allowed to have probability zero. The annual real-valued cost c(s,a,t) includes the insurer's net premium, claim and treaty cashflows for the covered year, with its sign convention chosen so that larger positive values mean higher losses. A deterministic terminal amount h(s) values residual liabilities when the chosen finite horizon ends. With consistent payment-date conventions, let β≥0 be the factor applied to the next year's continuation cost and γ>0 the entropic risk-aversion parameter for costs.
The real-valued stage costs are finite by construction. Hence, because the transition row is a genuine probability law, the exponential moment defined below is strictly positive.
Conditional entropic valuation of one year
For any future state-value function F:S→R, define
Mγ,β(s,a;F)=t∈S∑P(s,a,t)exp(γ[c(s,a,t)+βF(t)])
and the one-year risk-sensitive insurer cost
Qγ,β(s,a;F)=γ1logMγ,β(s,a;F).
The cost and continuation value are included in one conditional exponential moment. The former is paid or valued within the current period, and the latter carries the next-year discount factor. The same chosen action changes both the costs and the transition row.
If every destination has the identical current-period cost c and continuation u, then probability normalisation implies Qγ,β=c+βu. If every annual cost is increased by a fixed k, the stage value increases by k under the unit-mass probability row. With nonnegative transition probabilities, γ>0 and β≥0, increasing either annual costs or future costs cannot decrease the entropic stage value.
The discount factor must remain inside the exponential
For an annual cost c(s,a,t)=d(s,a) independent of the next state, the expression simplifies to
Qγ,β(s,a;F)=d(s,a)+γ1log(t∈S∑P(s,a,t)eγβF(t)).
Writing ρη(F)=η−1log∑tP(s,a,t)eηF(t), the continuation term for β>0 is βργβ(F). In general it differs from βργ(F). Entropic risk valuation is cash-additive and monotone under the stated conditions, but it is not generally homogeneous in the random cost. For β=0, continuation costs make no contribution and no division by γβ is performed.
This matters in actuarial applications: transplanting an expectation-based Bellman formula and simply replacing each conditional expectation with a fixed-parameter entropic premium would alter the intended discounted model.
Remaining-horizon policy and Bellman recursions
Let n∈N denote the number of policy years remaining. A deterministic nonstationary Markov policy π:N×S→A selects action π(k,s) when k+1 years remain and the current state is s.
Its recursively valued insurer cost is
J0π(s)=h(s),Jk+1π(s)=Qγ,β(s,π(k,s);Jkπ).
Define the finite-action entropic Bellman minimum by TF(s)=mina∈AQγ,β(s,a;F). Its finite-horizon optimum satisfies
V0(s)=h(s),Vk+1(s)=a∈AminQγ,β(s,a;Vk).
At horizon zero the terminal amount is the entire valuation. At each successor horizon, the minimum is attained because the action set is finite and nonempty. Positive risk aversion, nonnegative discounting and a stochastic transition row make the entropic stage map monotone in its continuation argument. Those properties support induction from the terminal year backwards.
Proposed fundamental theorem
For the finite model above, the capstone combines domination of every deterministic nonstationary Markov policy with existence of an attaining Markov policy:
Vn(s)≤Jnπ(s)∃π∗Jnπ∗(s)=Vn(s)for every policy π and state s,for every state s.
The second line asserts a single policy that works simultaneously for all starting states. Its state- and remaining-horizon-dependent actions may be obtained by selecting a minimiser at each finite Bellman step.
The formal graph contains definitions for the transition exponential moment, entropic stage valuation, finite-action minimum, fixed-policy value and Bellman value. The supporting theorem targets establish strict positivity, deterministic-value reduction, monotonicity, cash additivity, minimum comparison and attainment, the base and successor horizon identities, policy cost domination and an attaining policy. The capstone joins the latter optimality statements.
Mathematical scope and boundary cases
All probability distributions here have finite support in a finite state space, and the horizon is finite. Stochastic rows are required only where a conditional cost interpretation or positivity is used. Zero transition probabilities for particular destinations cause no division-by-zero problem. A zero-horizon value is exactly its terminal cost; β=0 removes future exposure from each one-step valuation. The type-level definition permits an empty state type, for which assertions at each starting state are vacuous, while the action type remains nonempty.
The positive-risk-aversion assumption excludes γ=0. Any analytical expected-value limit as γ→0 would require a separate result. Negative gamma is outside the proposed monotone insurer-cost interpretation.
The capstone is stated for deterministic nonstationary Markov policies only. It does not formally establish comparison with all randomised or history-dependent strategies, optimality for infinite horizons or an HJB verification theorem for continuous-time stochastic factors. A finite treaty price from other actuarial missions may later be substituted into c, after its own publication and consistency checks. No additional imported theorem dependencies are assumed here.
Evidence and source attribution
The source declarations have been checked by Prove2Me's private remote Lean compiler at the pinned Mathlib revision. The remote compiler's overall failure is deliberately caused by the incomplete proof bodies and a terminal type-mismatch sentinel; there were no unexpected declaration elaboration errors. This validates the proposed types and definitions, not any Lean proof. Independent high-precision finite model evaluation supports the mathematical direction, subject to the limitations of numerical evidence. The listed mathematical results remain proof obligations until accepted proof submissions exist.
[1] Bäuerle, Nicole and Glauner, Alexander (2022), Markov decision processes with recursive risk measures, European Journal of Operational Research, 296(3), 953–966. In the author preprint, Definition 4.6 states the one-step recursive risk operators, Theorem 4.7 establishes the finite-horizon Bellman value and existence of Markov optimal policies under Assumptions 3.1 and 4.2, and equation (4.2) is the one-step minimisation in the backward-induction proof. This mission explicitly specialises the framework to finite insurance costs and entropic valuation.
[2] Bäuerle, Nicole and Jaśkiewicz, Anna (2024), Markov decision processes with risk-sensitive criteria: an overview, Mathematical Methods of Operations Research, 99, 141–178. Example 1(a), equation (2) gives an entropic certainty equivalent in reward convention. The sign has been translated to an insurer cost/loss convention here.
[3] Actuarial Mathematics XVII: Finite-Horizon Markov Decision and Optimal Policy Valuation, previously published finite-state/action Markov recursion declarations. The present mission uses a distinct cost-minimising entropic stage operator, so it does not import XVII's reward-maximising Bellman definitions.
[4] Actuarial Mathematics XXI: Finite Exponential Premiums and Risk-Sensitive Retention, an earlier owner Draft concerning one-period entropic reinsurance premiums. Its unpublished declarations are not imported as available source modules.