Proposition 3 — J_N = T^N(J̄) under D and either D.1, or D.2 with T^N(J̄) > −∞

In the abstract dynamic programming model of Bertsekas (1977), let $T(J)(x)=\inf_{u\in U(x)}H(x,u,J)$ and let $J_N(x)=\inf_{\pi\in\Pi}(T_{\mu_0}\cdots T_{\mu_{N-1}})(\bar J)(x)$ be the optimal value function of the $N$-stage problem, for a positive integer $N$. Suppose Assumption D holds, and that either

1. Assumption D.1 holds, or
2. Assumption D.2 holds and $T^N(\bar J)(x)>-\infty$ for all $x\in S$.

Then

That is, the dynamic programming algorithm $\bar J, T(\bar J), T^2(\bar J),\dots$ computes the finite-horizon optimal values. The paper's Counterexamples 2 and 3 show that the conclusion can fail when D.1 is dropped and either D.2 or the condition $T^N(\bar J)>-\infty$ is dropped.

Formalization Note $N$ is a fixed natural number with $N\ge1$, and the condition $T^N(\bar J)(x)>-\infty$ (written $\ne\bot$) belongs to the D.2 branch only, for the same $N$.