Don’t Spill the Coffee

Daniel Kang

August 2026

A 2-D miniature of the behavioral-cloning mysteries in Seohong Park's post. Every policy on this page is a real trained network running live in your browser.

Abstract

Behavioral cloning with action-chunking policies is the workhorse of modern robot learning, yet it keeps violating supervised-learning intuition: Park (2026) documents four such mysteries at robot scale, including open-loop chunking beating closed-loop control and validation loss anticorrelating with performance. Each observation sits behind large models and days of compute, exactly the regime where the ablations that would explain it are unaffordable. We shrink all four mysteries onto a 2-D point mass with multimodal, non-Markovian demonstrations that trains in minutes on a laptop CPU, and use the miniature to separate each mystery into its mechanism. Three mysteries reproduce: rescaling input features moves success from 0.22 to 0.71 while validation loss stays 0.213 in both arms, a task whose state is a single 2-vector keeps improving through width-512 MLPs, and validation loss rises while success climbs from 0.50 to 0.69, so early stopping forfeits a fifth of the success rate. The central mystery inverts: closed-loop cloning wins every geometric task we can build (0.97–1.00 versus 0.22–0.72 for chunking), because feedback resolves multimodality on its own. Chunking becomes necessary exactly when the plant carries a fragile high-frequency mode, here a hanging payload that the temporally white noise of per-step sampling excites and smooth demonstrations do not; sampling the exact nonparametric conditional spills the payload in 96% of episodes while chunk replay from the same demonstrations succeeds in 98%, so the effect needs no neural network at all. These results locate open-loop superiority in the plant rather than in learning, predict which benchmarks can and cannot exhibit it, and supply a label-free metric, rollout drift, that ranks policies correctly where validation loss is blind.

success – spills – crashes – drift –

Figure 1. The same policy carries the payload as 25-step chunks and spills it when re-sampled every step. The testbed, live: faint gray trails are expert demonstrations; colored trails are rollouts of the selected policy, running in-page on the weights this project trained. In carry the mass hauls a stiff hanging payload; the strip under each pane traces the payload angle |θ| against the spill threshold (dashed red). The default comparison shows the same flow policy executed as 25-step chunks (left) and re-sampled every step (right): per-step sampling pumps the payload with white action noise and spills. Switch to gate or obstacle and the comparison inverts. "Oracle" policies use no network: they sample or replay the demonstration set directly. Drift is the mean distance of visited states to the demonstration manifold.

1 Introduction

Behavioral cloning is supervised learning, and it keeps refusing to behave like supervised learning. Training flow-matching action-chunking policies π(a_t:t+24|s_t) on scripted robot demonstrations at effectively infinite data, Park (2026) documents four mysteries: validation loss gets worse while task performance gets better, and a 10K-episode dataset can beat a 50K one; open-loop execution of 25-step action chunks beats closed-loop single-step control; a 37-dimensional pick-and-place needs 500M-parameter MLPs; and rescaling input features changes performance while leaving every training metric identical. His unifying reading is test-time distribution shift. Each observation sits behind GPU-scale simulation and large models, which is exactly the regime where the ablation that would explain it is unaffordable.

We take the opposite approach: shrink the setting until every experiment is cheap, and see which mysteries survive. Our testbed is a point mass in the plane whose demonstrations are smoothly random splines committed to a random latent plan (pass left or right, which slot, how fast). The policy sees position only, so the demonstration distribution is non-Markovian and p(a|s) is multimodal, the two properties Park's data has and standard 2-D benchmarks lack. Every policy trains in one to ten minutes on two CPU threads; the full study is 164 runs and about 700 lines of NumPy and PyTorch.

Our main claim is that Park's four mysteries have different mechanisms, and only one of them needs a robot. Three reproduce in two dimensions essentially intact, so they require nothing but multimodal non-Markovian demonstrations and a learned sampler. The fourth, open-loop superiority, is not a fact about learning at all: it is a fact about the plant, and it appears exactly when the plant carries a fragile high-frequency internal mode that temporally white action noise excites and smooth demonstrations do not.

The evidence for the plant claim does not involve neural networks. In every free-space geometric task we can build, closed-loop policies win (0.97–1.00 success against 0.22–0.72 for 25-step chunks), including the policy that just outputs the conditional mean. Add a hanging payload, and sampling the exact nonparametric conditional per step spills it in 96% of episodes while replaying 25-step chunks retrieved from the same demonstrations succeeds in 98%. The deterministic mean policy still scores 0.97 with the payload, so the failure is specifically per-step stochastic re-sampling, not feedback control and not multimodality.

This decomposition has a practical consequence: whether chunking helps is decidable by an oracle experiment on the raw demonstrations before any training is run, and a state-based integrator benchmark cannot exhibit open-loop superiority no matter how it is tuned. Our contributions:

A four-task 2-D testbed where multimodal, non-Markovian, smoothly random demonstrations are exact and cheap, with nonparametric oracle policies that bound what learning could achieve (§2).
Compelling evidence that open-loop superiority is a plant property: it inverts in every geometric task, returns with a payload, and is isolated at the oracle level with no network in the loop (§3–5).
Reproductions of the other three mysteries, including validation losses identical to three decimals across a 3× success gap (§6).
A label-free metric, rollout drift, that ranks policies correctly in every comparison where validation loss is blind or undefined (§7).

2 The testbed

A point mass moves from a random start at the bottom to a goal at the top, one hundred steps per episode, with small Gaussian position noise. Demonstrations are Catmull–Rom splines through randomized waypoints, committed to a latent plan drawn independently of the start. Four tasks share this machinery. Obstacle: a disc in the center, passed on a random side. Gate: a full-width wall with two narrow slots, a random slot per demonstration. Carry: no geometry, but the mass hauls a stiff hanging payload, natural period about five steps, driven by the commanded acceleration; swing it past 0.22 rad and the episode fails. Demonstrations are smooth, so their payload stays calm by construction. Carry+gate: both pressures at once.

Policies are flow-matching chunk policies executed by replanning every H steps, H ∈ {1, 5, 25}, as three-layer MLPs; H=1 is closed-loop control. An MSE policy of the same architecture outputs the conditional mean. Two oracles use no learning at all: oracle closed-loop samples the exact nonparametric conditional (a uniform draw among demonstration actions within a ball of the current state), and oracle chunking replays the next 25 actions of the nearest demonstration state. The oracles bound what learning could possibly do, which is what makes the dissection below possible. Figure 1 runs all of it live; the numbers in the text are 200-episode evaluations averaged over two seeds.

3 Closed-loop cloning wins every geometric task

The headline mystery does not survive miniaturization; it reverses. On the two geometric tasks, closed-loop policies are essentially perfect (0.97 to 1.00 success), and the 25-step chunk policies are the ones that fail (0.22 to 0.42), executing stale, mode-blended plans open-loop. The mechanism is visible in Figure 1: environment noise breaks the left/right symmetry, the conditional field amplifies the asymmetry, and commitment emerges from feedback. This works even for the plain MSE policy that outputs the mode-averaged conditional mean, which the standard multimodality story says should crash into the obstacle.

Validation loss actively prefers the losing side. On the obstacle task the chunked MSE policy reaches validation 0.037 against 0.047 for the closed-loop one, and 0.28 success against 1.00. A model-selection procedure reading validation loss picks the policy that crashes three times out of four.

One number in Figure 2 matters for everything that follows: nonparametric chunk replay, nearest demonstration state, copy its next 25 actions, succeeds at 0.98–0.99 on every task. Chunking as such is not the problem anywhere; learned interpolation executed blind is. This predicts that any training change pushing the learned policy toward retrieval should help, which is Park's memorization hypothesis, and §6 confirms it from the training-dynamics side.

Figure 2. Chunking hurts in the integrator worlds and rescues the payload worlds, and the oracles show the same reversal with no learning involved. Success by task, execution mode, and learner: 200 episodes per cell, learned policies averaged over 2 seeds, N = 2,000 demonstrations. Oracles use no network: closed-loop samples the exact nonparametric conditional p(a|s); chunked replays the nearest demonstration's next 25 actions. Lower panel: flow-policy success against chunk length. The geometric tasks fall monotonically with H, carry collapses only at H=1, and carry+gate peaks at the intermediate H=5.

4 Open-loop superiority lives in the plant

Adding one high-frequency plant mode brings Park's mystery back, and an oracle experiment shows learning was never the bottleneck. Why should Park's manipulators differ from a point mass? Our hypothesis: their plants punish temporally white action sequences. A policy re-sampled independently at every step draws from a wide conditional, since demonstrations at a given state differ in speed and direction, so the executed action sequence is white noise around the intent. Demonstrations are smooth. An integrator cannot tell the difference; its response to jitter is a harmless random walk that feedback absorbs. Contact dynamics can.

The carry task adds the smallest such discriminator: a stiff hanging payload (natural period about 5 steps, well above the demonstrations' motion frequencies) driven by the commanded acceleration. Swing it past 0.22 rad and the episode fails. Smooth demonstrations barely couple to it. White per-step noise has a flat spectrum, includes the resonance, and rings it up.

The oracle experiment isolates the effect with no network in the loop, and it is the strongest evidence on this page. Sampling the exact nonparametric conditional per step, which is the best possible stochastic closed-loop cloner, succeeds in 4% of carry episodes. Replaying 25-step chunks retrieved from the same demonstrations succeeds in 98%. The learned versions agree (0.11 against 0.92). What per-step marginalization destroys is the temporal correlation of the expert's actions, and this plant charges for it.

A result we did not predict sharpens the diagnosis. The deterministic MSE policy queried every step scores 0.97 on carry: it outputs the conditional mean, which is smooth, so nothing excites the payload. The failure is therefore not closed-loop control, and not closed-loop control of multimodal data either. It is per-step stochastic re-sampling. Three distinct fixes exist: remove the sampling (conditional mean), keep the sampling but hold each draw for H steps (chunking), or use a plant that does not care (integrators).

Claim (where open-loop superiority lives) Per-step re-sampling from the state-conditional makes the executed action sequence temporally white; chunking preserves within-chunk correlation. The open/closed gap for stochastic policies has the sign of the plant's transfer of that whiteness into the failure set: integrator plants low-pass it and closed-loop wins, plants with a fragile high-frequency mode rectify it and chunking wins. This is a property of the plant, measurable with oracles before any training is run.

5 Chunk length has an interior optimum

With both pressures active, the best chunk is long enough to be smooth and short enough to re-plan. Superimposing payload and slotted wall (carry+gate) yields the ordering the plant claim predicts among flow policies: per-step sampling spills (0.71), 25-step chunks commit to stale plans and crash (0.31), and the intermediate H=5 wins (0.87). Park's practical regime of 25-step chunks at 50 Hz is about half a second of commitment, the same kind of interior optimum, and the engineering literature on real-time chunk switching is working exactly this tradeoff.

The miniature also has a boundary, and stating it matters for interpreting everything above. The deterministic conditional-mean policy survives everything we can build in free-space 2-D (0.97 on carry+gate): smoothness for free, commitment from feedback. What kills conditional means on real robots is that under contact the average of two valid actions is often invalid; the mean of grasp-from-left and grasp-from-right passes through the object. That ingredient, mean-invalidity under constraints, does not fit in a free-space point mass, and it is the one thing Park's tasks have that ours do not. The full account of his mystery is then a conjunction: expressive samplers are needed because means become invalid under contact, and chunking is needed because per-step sampling is white and contact-rich plants charge for whiteness. Only expressive-plus-chunked survives both, and that is precisely the policy class Park trains.

6 Three mysteries reproduce in two dimensions

Model scale, feature scaling, and beneficial overfitting all survive miniaturization, so none of them needs robots, images, or high-dimensional state. Each needs only multimodal demonstrations and a learned sampler, and each becomes cheap enough here to see its mechanism.

Model scale (Mystery 3). The gate task's state is one 2-vector, yet chunked success climbs from 0.04 at width 8 to 0.60 at width 256 and 0.76 at width 512, about 580K parameters, with no sign of saturation. The validation-loss curve flattens far earlier: from width 128 to 512 it improves 1.7× while success improves 3.4×. The capacity is not spent on representing the state; it is spent on sharpening a 50-dimensional multimodal output distribution until samples stop blending modes, which is invisible to intuitions calibrated on input dimensionality. This offers an answer to Park's open question of whether his 500M-parameter requirement reflects fundamental difficulty: output-distribution sharpness alone can demand orders of magnitude more capacity than the state suggests.

Feature scaling (Mystery 4). Multiplying the slot-relative input features by c ∈ {0.1, 1, 10} changes no information and no validation loss: 0.217, 0.213, 0.213. Success moves from 0.21 to 0.22 to 0.71. Park saw identical training metrics with different performance and asked how to select between such policies; the drift column in Figure 5 answers it, 0.047 against 0.012, without a single reward query.

Beneficial overfitting and data scaling (Mystery 1). The classic signature appears exactly where memorization can: for MSE chunk policies on 50 demonstrations, validation loss bottoms out at 4K–8K steps and then rises (0.025 → 0.028 mean over seeds) while success climbs from 0.50 to 0.69. Early stopping at minimum validation loss forfeits about 20 points of success rate, and drift moves the right way (0.038 → 0.023) precisely where validation loss moves the wrong way. The overfitting policy is drifting toward the nonparametric retrieval oracle of Figure 2, which is Park's memorization-as-nearest-neighbor hypothesis with the mechanism visible. Flow policies at this scale resist: their objective re-noises every sample, validation flow loss never turns upward, but the decoupling is still severe. At 16K steps validation loss is 0.18 for every dataset size from 50 to 5,000 while success ranges from 0.16 to 0.34, and in extended runs success keeps climbing through 64K steps, reaching 0.68 at N=1,000, while validation loss creeps from 0.18 to 0.136. More data does not help past 1,000 demonstrations; the 5,000-demonstration run lands at 0.60 against 0.68, Park's 10K-beats-50K in point estimate, though within noise at one seed.

Figure 3. Validation loss cannot see what training is doing to performance, and for MSE at N=50 it points the wrong way. Gate task, H=25. Each pair of panels: rollout success by training step, one line per dataset size (mean over seeds; flow points beyond 16K are the extended single-seed runs), with the validation loss of the same runs below. Flow validation curves for different N are indistinguishable while success differs by 2× and keeps climbing after validation flattens. For MSE at N=50, validation loss turns upward after 4K–8K steps while success keeps rising: early stopping on validation loss picks the worst end of the curve.

Figure 4. A task whose state is one 2-vector keeps paying for width through 580K parameters. Gate task: success against MLP width at fixed data and steps, beside the validation loss of the same runs. The state is 2-dimensional; the output distribution being sharpened is 50-dimensional.

Figure 5. Same information, same validation loss to three decimals, 3× the success. Gate task: slot-relative input features multiplied by c ∈ {0.1, 1, 10}. Validation loss and drift are printed under each bar; only drift distinguishes the arms.

7 Rollout drift ranks policies without labels

One label-free number, the distance of visited states from the demonstration manifold, ranks policies correctly in every comparison where validation loss is blind or undefined. Park closes by asking for a usable metric of test-time distribution shift, noting the natural ones need expert relabeling at test time. States do not need labels. Define drift as the mean nearest-neighbor distance from the states a policy visits to the states of its training set (position, plus payload angle and velocity in the carry tasks). It costs rollouts but no reward function, no success detector, and no expert queries.

Pooled over every run on this page, drift and success correlate at Spearman ρ = −0.75. The pooled number understates the useful part. Validation loss cannot compare policies across execution modes at all (the objectives differ), and within the comparisons that matter it is blind, as Figures 3 and 5 show. Drift is measured in behavior space, so it compares anything with anything. Selecting by lowest drift picks the best or statistically tied-best policy in every family on this page:

Figure 6. Drift orders every run on this page; validation loss mostly tracks training progress with itself. One dot per run. Left: success against rollout drift (log axis), colored by task. Right: success against validation flow loss for flow policies; the decisions that matter (Figures 2, 3, 5) live inside vertical slices of this panel.

8 Limitations

In decreasing order of how much they worry us:

Mean-invalidity under contact, the ingredient we argue completes Park's mystery, is asserted from the literature, not demonstrated here. A minimal contact task is the natural next experiment.
Learned results are averaged over two seeds with 200 evaluation episodes; seed variance on the gate task is large, and individual cells carry roughly ±0.05.
All four tasks share one demonstration generator, so the inversion in §3 is a statement about this family of smooth, feedback-rescuable geometries, not all geometries.
The carry plant was tuned so that demonstrations survive and white noise does not. The claim predicts the direction of the gap, not its magnitude, and a payload resonant below the demonstration band couples to the demonstrations' own motion and flips which policy suffers, which we observed while tuning.
The flow-overfitting negative in §6 is bounded by compute; 64K steps may simply be early. The 10K-beats-50K data point is one seed and we do not lean on it.
The oracles condition on position only, which slightly favors them on the carry tasks where the payload state is hidden from retrieval and visible to the networks.

9 Related work

The compounding-error account of closed-loop cloning (Ross et al., 2011) predicts closed-loop failure with chunking as mitigation; our geometric tasks show the opposite sign, so compounding error cannot be the whole story, and the plant account in §4 says when each sign obtains. Action chunking entered practice through low-cost bimanual manipulation (Zhao et al., 2023) and diffusion policies (Chi et al., 2023), justified by non-Markovian experts and multimodality; the carry task splits those justifications apart, since its closed-loop failure persists with the exact conditional and vanishes for the deterministic mean. Real-time chunking (Black et al., 2025) engineers the smoothness-versus-staleness tradeoff that §5 exhibits as an interior optimum in chunk length.

10 Takeaways

If you remember three sentences from this page, we suggest these. Whether action chunking helps is a property of the plant, not the learner, and you can measure it with a nearest-neighbor oracle before training anything. A state-based integrator benchmark cannot exhibit open-loop superiority, so closed-loop cloning working fine there says nothing about robots; a contact-rich, high-rate platform should not expect closed-loop stochastic policies to work at any data scale. Training metrics are the wrong instrument for imitation: across execution mode, feature scale, width, dataset size, and checkpoint, validation loss was blind or backwards wherever the decision mattered, and rollout drift, which needs no labels, ranked every one of those comparisons correctly.

Everything above is about 700 lines of NumPy and PyTorch, 164 runs, a few CPU-hours end to end. The policies in Figure 1 are the actual artifacts of those runs, executed by your browser. The full results table is below.

All runs (table view)