When a robot hesitates or jerks between action chunks, first separate an empty action queue from a disagreement between two available predictions. Then check the RTC mode and the timing of the overlap. A guidance setting cannot supply actions that arrived late, and a timely chunk can still begin at the wrong point in the robot’s trajectory.
LeRobot’s current RTC documentation distinguishes guided continuation from trained prefix conditioning. This article follows that main documentation, checked October 15, 2026; it requires a source installation and may differ from a stable package. No policy, simulator or robot was run for this article.
Check the checkpoint before tuning the join
In guided mode, LeRobot applies guidance during flow-matching denoising to keep the new prediction consistent with the previous chunk. Ordinary compatible flow-matching checkpoints can use this route. The trained route instead hard-conditions a prefix and currently requires a Pi05 checkpoint trained with policy.rtc_training_max_delay > 0. Selecting trained mode on an unsupported policy is rejected rather than silently falling back to guidance.
That difference changes the first debugging question. If a checkpoint was never trained for prefix conditioning, adjusting its guidance weight cannot turn it into a trained-mode checkpoint. Record the actual mode and checkpoint configuration before comparing two runs described only as “RTC enabled.”
Also distinguish where inference runs from how the prediction is conditioned. The documentation allows RTC locally or through the remote backend. Moving the model to another process can change transport and queueing, but it does not establish that the next prediction follows the intended continuation method.
Match the new chunk to the actions already consumed
Imagine an illustrative executor applying actions at 50 Hz. A prediction takes 80 milliseconds while the robot continues moving. Four action periods pass during that calculation. The beginning of the returned sequence therefore cannot be interpreted as though the robot were still at the state photographed when prediction began.
Those numbers are a timing example, not a measured LeRobot result. A real run needs the queue position at prediction start and at handoff, as well as capture, inference and merge timestamps. Scheduling and transport can make elapsed time differ from the nominal number of action periods.
The documented inference_delay is expressed in action timesteps. Keep its physical duration beside it: four steps at 50 Hz means 80 milliseconds; four at 20 Hz means 200 milliseconds. Reusing the same integer after changing the action rate does not preserve the same delay.
For one troublesome handoff, retain the remaining old actions supplied to prediction, the returned chunk and the first command actually taken from the merged queue. This is a proposed diagnostic record, not a claim that one built-in log already contains the entire path. It lets you ask whether the new chunk was aligned to the same timeline as the actions the robot consumed.
If the queue emptied before the new result arrived, investigate why the available execution interval was insufficient. If actions remained available, compare the boundary commands and alignment. Increasing smoothness guidance before separating those cases can hide the original fault behind a second change.
Prefix capacity is not the playback interval
For trained mode, the current documented admission rule is rtc_training_max_delay <= execution_horizon <= prediction_steps - rtc_training_max_delay. Pi05 uses the checkpoint’s chunk_size for prediction_steps. Local rollout also requires its queue threshold to cover the maximum trained delay.
Here execution_horizon means supplied prefix capacity. Pi05 hard-conditions only the inference_delay prefix; the capacity is not a command to play that many actions before requesting another chunk. Keep this meaning separate from the generic “execution horizon” used in the action-chunking guide.
For example, a hypothetical checkpoint with a maximum trained delay of four steps and a 50-step chunk gives a permitted capacity range of four through 46 steps under that rule. Passing the configuration check would not prove that a particular returned chunk arrived in time or still had enough usable actions. Inspect the actual overlap and freshness at the handoff.
In guided mode, changing how much of the preceding trajectory influences the new prediction can trade reaction against continuity. Compare settings on the same action rate, checkpoint and observation path; a change in any of those can otherwise masquerade as a better guidance choice.
Measure the response as well as the boundary
For a controlled comparison, record three things separately: whether the queue ran empty, how much the applied command changed at each handoff, and when a changed observation first produced the intended applied response. Use the actual action units and define the response criterion before reviewing the run.
A smaller jump can come from following an outdated plan longer. A quicker response can introduce an unsuitable transition. Neither a smooth video nor a completed inference call resolves both questions. The useful result is a handoff trace showing which actions were available, which continuation mode produced the replacement, and what the robot was actually commanded to do next.