Appendix E — Randomized AC0 Sampling of Regular Languages and Markov ChainsResearch Paper
Motivation
Sampling an entire finite-state execution is a different computational task from recognizing a supplied word. A membership circuit must answer correctly on its given input. A sampler instead chooses the input together with its associated output, while preserving their joint distribution. This distinction is relevant when finite-state models describe allowable executions and when parallel generation should avoid a sequential pass through every transition. The source paper establishes constant-depth sampling results for regular languages and finite Markov chains in Appendix E (Zhang et al., 2026).
The mission concerns that appendix's finite-state sampling results. It formalizes all six numbered results E.1–E.6, together with the seed bounds stated in the surrounding text. Each milestone points to a published theorem with a separately checked proof. The objects to sample are complete words, word–acceptance pairs, and complete state trajectories. Sampling only the final state would lose the joint temporal information required by the targets.
Finite-state models and fair-bit circuits
A binary deterministic finite automaton has finitely many states, a designated start state, a transition for each state and bit, and a set of accepting states. Starting from its initial state, a binary word of length determines a run. Let be the set of words whose final state is accepting. Put . An arbitrary such automaton represents an arbitrary binary regular language, with no assumption that membership belongs to .
A finite Markov chain consists of a nonempty finite state set , an initial probability vector , and a row-stochastic transition matrix . Entries are nonnegative real numbers, not restricted to rational numbers. For , its probability is . The distribution of all states is denoted . For , conditioning is allowed when the probability that is positive.
A randomized sampler is a deterministic Boolean circuit supplied with independent, unbiased random bits. AND and OR gates have unbounded fan-in; NOT gates and Boolean constants are permitted. Its size is polynomial and its depth has a constant bound independent of trajectory length. Approximate families are nonuniform in both and their real error tolerance : existence does not require efficiently computing circuit descriptions or numerical tables from a supplied transition matrix.
Formalization targets
The main goal is the conjunction of all results E.1–E.6, including the linear seed guarantee for E.5. This organizational theorem is not an additional numbered result attributed to the paper. All six milestones must be proved for the goal to hold. The exact Markov sampling criterion in Theorem E.5 is one component. A nonnegative real number is dyadic if it equals for natural numbers , including zero. A chain is path-dyadic when every finite trajectory, including a trajectory of length zero, has a dyadic probability. The target is
On the left, the family has polynomial size in and depth bounded independently of . A supporting target records the linear fair-bit budget when path-dyadicity holds. Lemma E.4 supplies another target: every path-dyadic chain is the coordinatewise projection of a finite chain whose initial and transition probabilities are all dyadic. Path-dyadicity must not be replaced by the stronger requirement that the original transition matrix itself be dyadic (Appendix E.3).
Theorem E.1 concerns exact sampling of a uniformly random word together with its acceptance indicator:
The sampler uses exactly fair bits, polynomial size, and depth depending only on the fixed automaton. This is a sampling assertion and does not assert that membership can be computed in constant depth on an arbitrary externally supplied word.
Theorem E.2 concerns the uniform law on nonempty . For every , the target is
Every possible output is accepted. Size is polynomial in and , and depth depends only on the fixed language or its automaton. In particular, the leading coefficient of the term in the random-bit bound is one (Appendix E.2).
Theorem E.6 is the corresponding approximate target for trajectories conditioned on . It requires total variation error at most , only positive-probability outputs ending in , fair bits, polynomial size in and , and depth depending only on the chain. There is no lower bound on the conditioning probability beyond strict positivity (Appendix E.4).
Lemma E.3 supplies the exact representation used in the Markov construction. A transition-dyadic chain is represented by one binary DFA and a fixed state projection. On a uniform input of length , with positive constants , the states observed after bits project to exactly the complete trajectory law of the chain. The same DFA and projection work for every length, including zero.
Significance of the targets
These results separate sampling an input together with its finite-state behavior from computing that behavior on a prescribed input. They also identify a precise obstruction to exact sampling with a finite fair-bit seed: each output probability must be dyadic. Approximate sampling allows arbitrary real transition probabilities while retaining the valid trajectory support. The complete trajectory requirement preserves correlations among all time coordinates.
The formalization makes the circuit model, seed accounting, support guarantees and asymptotic quantifiers explicit and reusable. The theorem statements and their separately submitted proofs have passed the platform verifier. The goal proof imports all six milestones; its E.5 milestone in turn imports both the equivalence and the linear seed theorem. Thus the platform dependency graph covers the entire appendix. Platform verification establishes the formal statements; semantic correspondence is documented in the statement descriptions and source references.
Difficulty
A literal step-by-step implementation of a run makes circuit depth grow with , so it does not meet the constant-depth target. Recognition results alone do not furnish the required joint sampler. For approximate word sampling, an error estimate alone is insufficient: the output must remain in the accepted support, and the coefficient-one seed bound must also hold. For exact Markov sampling, unreachable states and non-dyadic individual transitions prevent identifying the desired criterion with entrywise dyadicity. Conditioning on a rare terminal event must not introduce an unstated dependence on its probability.
Formalization scope
States are indexed by finite types . Binary words have type ; trajectories have coordinates. Markov vectors and matrices are real-valued and carry normalization and nonnegativity proofs. Sampling probabilities are defined by counting all seeds that give a particular encoded outcome and dividing by . Total variation is half the finite sum of absolute probability differences.
The circuit representation is an explicit topologically ordered gate list with checked wire references. Size counts input bits, gates, gate-input wires and output wires; this yields the usual polynomial-size class. State trajectories use a fixed one-hot encoding at each time coordinate. The invalid-output predicate is checked over every seed, so neither arbitrary decoders nor unrepresented outcomes can satisfy an exact or support-preserving claim. The encodings used by the targets are injective. The circuit family is existentially quantified after the fixed finite-state model; the depth and polynomial constants precede and .
Zero-length words and zero-transition Markov paths are included. The word target excludes empty accepted slices. The conditioned target excludes zero-probability conditioning events. Definitions of division by zero still exist as total Lean functions, but the hypotheses exclude those cases from approximate claims. Extensions of these results, reusable finite-state probability lemmas, and further circuit constructions are welcome. No sampler-existence axiom, vacuous model or final-state-only replacement is part of the interface.
Selected references
- Jiarui Zhang, Chengwei Liang, Haozhe Jiang, Binhang Yuan and Jingzhao Zhang. Revision Provably Reduces Sequential Computation in Diffusion Language Models. Zenodo preprint, 2026. DOI 10.5281/zenodo.23105365, Appendix E, Theorems E.1, E.2, E.5, E.6 and Lemmas E.3, E.4, printed pages 33–36.