Motivation
An agent whose preferences change over time cannot simply optimize once: a plan that is optimal today may be abandoned tomorrow by the agent's own "future selves". This is time inconsistency, and its standard source is non-exponential discounting, studied since Strotz (1955). The usual response, going back to Strotz and made rigorous in continuous time by Ekeland and Lazrak (2006), is to look for an equilibrium strategy: a strategy from which no future self wants to deviate. In continuous time "no incentive to deviate" is encoded through a first-order condition, which Huang and Zhou call a weak equilibrium.
Björk, Khapko and Murgoci (2017, Remark 3.5) observed that this first-order condition need not express the absence of an incentive to deviate: it only says that a derivative vanishes or is nonnegative, and a stationary point need not be a maximum. Huang and Zhou (Math. Oper. Res. 46(2), 2021; read here as arXiv:1809.09243v3) introduce a strong equilibrium, which requires the agent's payoff to be no better after any short deviation, and study the two notions for continuous-time Markov chains with controlled generators. This mission formalizes their main example (Example 4.3): a concrete two-state model in which a weak equilibrium fails to be strong, so that an agent in one state strictly gains from deviating on every short interval.
Setting
The state space is S={1,…,N}. A generator is a matrix Q∈RN×N whose i-th row Qi lies in Ei={q:qj≥0 (j=i), qi=−∑j=iqj}. Admissible rows form sets Di⊆Ei, and the control space is Q={Q:Qi∈Di ∀i}. A payoff function f(t,i,q) gives the payment rate at time difference t in state i when row q is in force. The expected payoff of using Q from state i is
F(i,Q)=Ei,Q[∫0∞f(t,Xt,QXt)dt]=∫0∞j∑(etQ)ijf(t,j,Qj)dt.
The concatenation Q⊗εQ∗ uses Q on [0,ε] and Q∗ afterwards. A generator Q∗∈Q is a weak equilibrium if liminfε↓0[F(i,Q∗)−F(i,Q⊗εQ∗)]/ε≥0 for all i∈S, Q∈Q, and a strong equilibrium if for all i,Q there is ε>0 with F(i,Q∗)≥F(i,Q⊗ε′Q∗) for every ε′∈(0,ε].
The paper's first-order and second-order quantities are
ΓQ∗(Qi)=f(0,i,Qi)+Qi⋅F(Q∗),ΛQ∗(i,Q)=ft(0,i,Qi)+Qi⋅(2G(Q∗)+ΓQ∗(Q)),
with G the payoff computed with the time derivative ft in place of f.
In the two-state model of §4, S={1,2}, Di=Ei, every generator is Q∼(a,b)=[−aba−b] with a,b≥0, and f(t,1,(−a,a))=δ(t)g1(a), f(t,2,(b,−b))=δ(t)g2(b) with the pseudo-exponential discount δ(t)=λe−ρt+(1−λ)e−ρ′t. Example 4.3 takes λ=21, ρ=1, ρ′=2, g1(a)=−a2, and g2(b)=144193+65b for b<127, g2(b)=2−(1−b)2 for b≥127.
Formalization targets
Goal: Example 4.3
Q∗∼(125,127) is a weak equilibrium and not a strong equilibrium.
The statement is fully explicit and has no hypotheses.
Milestones
- (4.3)–(4.4): closed forms of F1−F2 and G1−G2 for Q∼(a,b) under pseudo-exponential discounting.
- (4.5), (4.6), (4.8): ΓQ∗(Q1), ΓQ∗(Q2) and ΛQ∗(2,Q) in the two-state model.
- Theorem 3.1: under (2.3) and (3.3), Q∗ is weak iff ΓQ∗(Qi∗)≥ΓQ∗(Qi) for all (i,Q).
- (4.11): at Q∗, Γ1 is maximized uniquely at a∗ and argmaxb≥0Γ2(b)=[0,7/12].
- Proposition 3.3: the second-order test on the set R of equality cases.
- (4.12): Λ2(a∗,b)=−b/12−579/288 on [0,7/12], so Λ2(a∗,b∗)<Λ2(a∗,b) for b∈[0,7/12).
Further statements
Remark 4.1: for Q∼(5/12,b), b∈[0,7/12), F(2,Q∗)<F(2,Q⊗εQ∗) for all small ε>0. And (4.13): the equation 65≤2a=21(1+a1+2+a1)(a2+144193) has a unique solution aˉ≥0, and Qˉ∗∼(aˉ,0) is a strong equilibrium.
Significance
The example shows that the weak equilibrium concept, the standard one in the time-inconsistency literature since Ekeland and Lazrak, admits strategies that an agent would abandon: Remark 4.1 exhibits a deviation that strictly increases the payoff on every short interval. Together with (4.13), it shows that in one and the same model a weak-but-not-strong equilibrium and a strong equilibrium coexist, so the choice of equilibrium concept changes which strategies are predicted. The example is the paper's justification for introducing strong equilibria.
The result is proved in the paper, through the first-order characterization (Theorem 3.1), the second-order test (Proposition 3.3, resting on the second-order expansion Lemma 3.2) and explicit computations in the two-state model. None of it has been machine-checked. A formal proof would provide checked versions of Theorem 3.1 and Proposition 3.3 for finite-state chains, reusable for any other example, and an independent check of the arithmetic of Example 4.3, where the source contains at least one misprinted formula.
Difficulty
Both conjuncts of the goal concern the small-ε behaviour of F(2,Q⊗εQ∗), an integral of products of matrix exponentials. Weakness reduces to a first-order comparison by Theorem 3.1, but its proof needs the expansion (3.6) with an o(ε) remainder controlled uniformly enough to pass under the integral over [0,∞). Failure of strong equilibrium cannot be read off the first-order term: at b∗=7/12 the maximizer of Γ2 is not unique, every b∈[0,7/12) ties at first order, and the decision moves to the second-order coefficient Λ of Lemma 3.2, whose derivation needs a second-order expansion of the concatenated payoff. The regularity hypotheses of Lemma 3.2 and of (3.3) must also be verified for the example's discount function.
Formalization scope
States are Fin N; the paper's state k is index k−1, so in the two-state model state 1 is 0 : Fin 2 and state 2 is 1 : Fin 2. Expectations are written through the transition matrices etQ (NormedSpace.exp (t • Q)) and Fubini, exactly as the paper computes on p. 23; no Markov process is constructed. Payoffs are Bochner integrals over (0,∞). The weak-equilibrium condition is written as "for every η>0, eventually as ε↓0 the quotient is ≥−η", which is liminf≥0 without a real-valued liminf. The norm in (2.3) is the sup norm, equivalent to the paper's Euclidean norm because (2.3) quantifies over every c>0.
Standing assumptions: Di⊆Ei, continuity of f in t (2.2) and the integrability condition (2.3). Theorem 3.1 adds the shift bound (3.3)–(3.5); Proposition 3.3 assumes the conditions of Lemma 3.2, with the derivative ft tied to f. The closed forms (4.3)–(4.4) assume ρ,ρ′>0 (the page allows ρ,ρ′≥0, but with a zero rate the payoff integral diverges); the algebraic identities (4.5)–(4.8) keep the page's ρ,ρ′≥0; Example 4.3 has ρ=1, ρ′=2. Statement (4.11) gives Γ2(b)=g2(b)−65b and the argmax; the page's case formula for b≥7/12 contains a misprint and is not used.
No trivializing reading is available: the goal has no hypotheses, F is built from etQ and the concatenation uses Q before and Q∗ after ε, so neither conjunct holds by a junk value. Integrability of every payoff follows from (2.3) and is never assumed.
Theorem 3.1 and the second-order package are also drafted in the companion mission on existence of equilibria; they are restated here in the sub-namespace StrongWeakEq.TwoState. The goal needs the verification of (2.3), (3.3) and the Lemma 3.2 conditions for the example (convexity of δ gives (3.15)), the closed forms of etQ for two-state generators, and the expansions of the paper. Contributions of proofs of the general results (Theorem 3.1, Proposition 3.3) are welcome independently of the example.
Selected references
- Y.-J. Huang and Z. Zhou, Strong and Weak Equilibria for Time-Inconsistent Stochastic Control in Continuous Time, Math. Oper. Res. 46(2), 2021; arXiv:1809.09243v3. https://arxiv.org/abs/1809.09243, https://doi.org/10.1287/moor.2020.1066
- I. Ekeland and A. Lazrak, Being serious about non-commitment: subgame perfect equilibrium in continuous time, 2006. https://arxiv.org/abs/math/0604264
- T. Björk, M. Khapko and A. Murgoci, On time-inconsistent stochastic control in continuous time, Finance Stoch. 21 (2017), pp. 331–360.
- R. H. Strotz, Myopia and inconsistency in dynamic utility maximization, Rev. Econ. Stud. 23 (1955), pp. 165–180.