Motivation
Multistage stochastic linear programs model decisions taken over time under uncertainty: reservoir releases in hydro-thermal scheduling, production and inventory plans, capacity expansion. Their standard solution method, stochastic dual dynamic programming (SDDP, Pereira and Pinto 1991), approximates each stage's cost-to-go function from below by cutting planes. This gives a lower bound on the optimal value and a candidate policy, but no guaranteed upper bound: in the risk-neutral case the policy's expected cost is estimated by Monte Carlo simulation.
Planners who are averse to risk replace the expectation in each stage by a coherent risk measure (Artzner, Delbaen, Eber and Heath 1999), nested stage by stage so that the resulting dynamic risk measure is time consistent (Ruszczyński and Shapiro 2006). Under such a nested objective, simulation no longer estimates the cost of a policy: the nested risk of a policy is not an average over sampled scenarios. Philpott, de Matos and Finardi (2013) address this by an inner approximation of the cost-to-go functions from above, built from convex combinations of finitely many points with known upper bounds. Their Proposition 5 shows that the optimal value y of the resulting first-stage problem bounds the nested risk-adjusted cost of the policy defined by the inner approximation. This gives a deterministic upper bound that complements the SDDP lower bound, and they apply it to the Brazilian hydro-thermal system.
Setting
There are T≥2 stages. The decision at stage t is xt∈Rn; At,Et are m×n matrices and ct∈Rn. Stage 1 is deterministic: A1x1=b1. For t≥2 a random outcome ωt ranges over a finite set Ωt, independently of earlier stages, and enters through the right-hand side bt(ωt). The feasible set of stage t given the previous decision xt−1 is
Xt(ωt)={xt≥0: Atxt=bt(ωt)−Etxt−1}.
A coherent risk measure ρ maps random costs Z:Ω→R to R and is subadditive, monotone (Z1≤Z2⇒ρ(Z1)≤ρ(Z2)), positively homogeneous and translation equivariant (ρ(λ+Z)=λ+ρ(Z)). Larger is worse: these are costs. Each stage t≥2 carries a one-step coherent risk measure ρt on random costs over Ωt. It is extended to costs in R∪{+∞} by ρˉt(Z)=+∞ whenever some outcome has infinite cost.
For s=2,…,T fix points xs−11,…,xs−1Js−1 and numbers qs1,…,qsJs−1. The inner approximation is the lower boundary of the convex hull of the pairs (xs−1j,qsj):
Q^s(x)=min{j∑λjqsj: j∑λjxs−1j=x, λ≥0, j∑λj=1},
which is +∞ off the convex hull of the points, with Q^T+1≡0. The policy's stage problem (13) and the first-stage problem (8) are
qt(xt−1,ωt)=xt∈Xt(ωt)minct⊤xt+Q^t+1(xt),y=A1x1=b1minc1⊤x1+Q^2(x1).
The numbers qsj are upper bounds when qsj≥ρˉs(qs(xs−1j,ωs)) for all s and j. A policy defined by the inner approximation, π^, takes at each state xt−1 and outcome ωt an action x^t(xt−1,ωt) solving (13), and a first-stage action x^1 solving (8). The actual risk-adjusted cost of π^ over stages t,…,T is
v^t(xt−1,ωt)=ct⊤x^t+ρˉt+1(v^t+1(x^t,ωt+1)),
with no future term at t=T. The nested risk-adjusted cost of the policy is v(π^)=c1⊤x^1+ρˉ2(v^2(x^1,ω2)).
Formalization targets
Goal: Proposition 5
v(π^) ≤ y.
The bound holds for every T≥2, every family of coherent ρt, every choice of points satisfying the upper-bound property, and every minimizing selection π^.
Milestones
- The convexity display of §2: subadditivity and positive homogeneity imply ρ(αZ1+(1−α)Z2)≤αρ(Z1)+(1−α)ρ(Z2).
- Q^s is the lower boundary of the convex hull H of the pairs (xs−1j,qsj) (its epigraph is H plus the vertical half-line), hence convex with Q^s(xs−1j)≤qsj, and every convex function bounded by the qsj at the points lies below Q^s.
- qt(⋅,ωt) is convex in xt−1.
- The interpolation bound ρˉt(qt(xt−1,ωt))≤Q^t(xt−1).
- The induction claim ρˉt(v^t(xt−1,ωt))≤Q^t(xt−1) for t=2,…,T and every xt−1.
Companions
Proposition 3, the risk-neutral case ρt=E. Proposition 4, the subgradient rule ∑mpmζ~mg(x~,ωm)∈∂xρ(Z(x,ω)) for ρ given by its dual representation. The existence of a policy π^, which shows that the hypotheses of the goal can be met.
Significance
Proposition 5 makes the inner approximation a certificate. Any policy computed by minimizing against Q^t+1 has nested risk-adjusted cost at most y, a number obtained by solving one linear program. Together with the lower bound z from outer approximation it brackets the risk-adjusted optimal value. The gap y−z then measures the quality of a risk-averse SDDP policy without simulation, which the nested objective does not allow. The bound covers every coherent nesting used in practice, such as mixtures of expectation and CVaR.
The results are proved in the paper. None of them is machine-checked. A formalization adds a precise account of the infinite values the paper does not discuss: infeasible stages, points outside the convex hull, and the extension of ρt to +∞. It also pins down the exact form of the upper-bound hypothesis the induction needs.
Difficulty
The obvious reading of the hypothesis on p. 16, "qt+1j is an upper bound on ρt+1(Qt+1(xtj,ωt+1))" with the true value function Qt+1, does not support the induction. The policy π^ does not act optimally, so its cost can exceed the true value, and the inductive step compares v^t with the policy's own stage value qt, not with Qt. The hypothesis that works bounds the risk-adjusted optimal value of the policy's stage problem (13) at the points, which is what the paper's Upper Bound Computation actually computes.
The second difficulty is infinite values, which the paper never discusses. A risk measure is defined on real random costs, yet stage problems can be infeasible, Q^s is +∞ off a convex hull, and a convex combination may put zero weight on a point whose bound is +∞. Every inequality of the paper has to be re-read in R∪{+∞}, and the extended-real arithmetic must not silently produce a value the paper does not intend.
Formalization scope
Vectors are Fin n → ℝ, all stages share n and m, and costs that may be infinite are EReal. Minima are EReal infima, +∞ on an empty set. Attainment of a minimum is never built into a definition: the policy is a selection with a minimizing property, required at every state where the stage problem is feasible. Stages are natural numbers. Points and bounds are indexed by the stage s of the function Q^s they define, so pts s j is xs−1j and qv s j is qsj. Stagewise independence is built in: ωt ranges over Ωt whatever happened before.
Standing assumptions made explicit in the statements:
- T≥2;
- ρt coherent for t=2,…,T (all four axioms, though the proof uses only monotonicity and convexity);
- the terminal convention ρT+1(QT+1)=0, so Q^T+1≡0;
- (8) without the constraint x1≥0, as printed;
- the upper-bound hypothesis in the form above.
The risk measures ρt act on real random costs and are extended by ρˉt(Z)=+∞ when some outcome has infinite cost. In Proposition 4, ρ is given by its dual representation (10); the representation theorem itself is not part of the mission. The existence statement adds solvability of A1x1=b1 and boundedness below of the last-stage program, without which (8) or the last stage has no solution.
The goal is not to be trivialized. Its hypotheses mention neither v^t nor the induction claim; the policy must minimize at every feasible state, not only along visited states; and the statement is for general T and general coherent ρt, not the expectation. The bound is trivially true when y=+∞ and is informative when x^1 lies in the convex hull of the stage-1 points.
A complete development needs:
- convexity of extended-real linear-programming value functions in their right-hand side;
- convexity of the convex-hull function Q^;
- finite Jensen-type inequalities for monotone convex risk measures;
- linear-programming attainment, for the existence companion.
The first two are reusable for any SDDP-type development. Proofs of the milestones in any order are welcome.
Selected references
- A. B. Philpott, V. L. de Matos, E. Finardi, On Solving Multistage Stochastic Programs with Coherent Risk Measures, Operations Research 61(4), 2013. https://doi.org/10.1287/opre.2013.1175
- P. Artzner, F. Delbaen, J.-M. Eber, D. Heath, Coherent Measures of Risk, Mathematical Finance 9(3), 1999. https://doi.org/10.1111/1467-9965.00068
- A. Ruszczyński, A. Shapiro, Conditional Risk Mappings, Mathematics of Operations Research 31(3), 2006. https://doi.org/10.1287/moor.1060.0204
- M. V. F. Pereira, L. M. V. G. Pinto, Multi-stage stochastic optimization applied to energy planning, Mathematical Programming 52, 1991. https://doi.org/10.1007/BF01582895
- A. Shapiro, Analysis of stochastic dual dynamic programming method, European Journal of Operational Research 209(1), 2011. https://doi.org/10.1016/j.ejor.2010.08.007