The unrolled gradient $\nabla v_\pi(s) = \sum_x \sum_k \Pr(s \to x, k, \pi) \sum_a \nabla\pi(a\mid x) q_\pi(x,a)$

Initializing...

The unrolled gradient $\nabla v_\pi(s) = \sum_x \sum_k \Pr(s \to x, k, \pi) \sum_a \nabla\pi(a\mid x) q_\pi(x,a)$ · Prove2Me