RACER-rich versus RACER-simple is the useful comparison for asking what detailed recovery language contributes within the project’s simulated-plus-real data setting. RACER-rich versus RACER-scratch changes another major factor: scratch uses only real-world training data, while rich also uses simulation training. A difference between those two cannot be attributed to richer wording alone.
The RACER project defines these real-world variants. Its paper, version 1 dated September 23, 2024, explains how recovery trajectories receive simple and rich annotations and how the supervisor guides the actor. Both sources were checked October 15, 2026. This article interprets the reported method and comparisons; no data generation, policy training or robot experiment was performed here.
Read the language and data columns together
The project uses these three names for its real-world demonstrations:
| Variant | Instruction detail | Training-data setting |
|---|---|---|
| RACER-scratch | Rich | Real-world data only |
| RACER-simple | Simple | Simulated and real-world data |
| RACER-rich | Rich | Simulated and real-world data |
The paper describes generating both simple and rich instructions for the recovery-augmented dataset as a comparative study. Its real-world experiment then separates training from scratch on real data from training that uses simulated and real data. The simple/rich contrast is therefore more informative about annotation detail than the scratch/rich contrast: both simple and rich retain the mixed-data setting.
Read the comparison with its reported robot, task set and training procedure. The evaluation episodes provide the performance evidence; the project videos illustrate individual behaviors.
The paper also separates two factors in simulation. It compares task goals alone, goals plus simple instructions, and goals plus rich instructions, then repeats the comparison without failure-recovery transitions. This arrangement asks whether the recovery experience helps separately from whether its language is detailed. Calling both changes “better prompting” would erase the data contribution.
Rich instructions are attached to corrective transitions
RACER’s recovery data is more than a set of longer captions on ordinary successful demonstrations. The pipeline identifies keyframes associated with alignment, grasping and releasing. It perturbs expert actions with bounded noise, steps the simulation into an off-nominal state and adds a recovery transition toward the expert continuation.
The paper distinguishes one-step and two-step recovery. Returning directly toward the expert action can work for an alignment deviation. For grasping or releasing, the authors found that immediate recovery could collide with the target, so their two-step construction inserts an intermediate expert waypoint before returning. The augmented episodes are rolled out and filtered by task success. These are the authors’ simulation construction rules, not a general instruction to repeat a previous command on hardware.
Language is then attached to the resulting transitions. The annotation procedure combines task information, object locations, failure types and template descriptions of changes in pose, gripper state and collision. An LLM paraphrases that structured information. Simple instructions mainly name an action and an object or direction. Rich instructions can add failure explanation, spatial detail, target attributes and the expected outcome.
For an illustrative missed grasp, a simple instruction might request a leftward correction. A richer instruction could identify the intended cup, describe that the gripper stopped to its right, and ask for alignment before grasping. The extra information is tied to the same recovery situation. Merely lengthening the sentence with unrelated commentary would not reproduce that training distinction.
The paper’s additional real-world variant without failure analysis narrows the question further: even within rich language, the explanation of the error is an identifiable component. “Rich” should therefore be read as structured task-relevant information, rather than a synonym for maximum sentence length.
The actor receives both the goal and the current instruction
At runtime, RACER’s supervisor uses the front-view image, task goal, previous instruction and a description of the robot’s recent state change to generate the next instruction. The actor receives that instruction together with the high-level goal, visual observations and robot state. A correction is thus part of a continuing interaction, not a standalone text command detached from the task.
This matters in the project’s goal-change experiments. If a drawer task changes from the top drawer to the bottom drawer, the active goal changes what counts as useful continuation. An instruction that was suitable before the switch can become irrelevant afterward. Reading the current instruction together with the current goal explains what the policy is being asked to adapt to.
It also explains why a broad RACER-versus-RVT result is not a pure wording ablation. The reported actor adaptation replaces RVT’s original CLIP language encoder with T5-11B and removes the timestep input, alongside the changed training data and supervisor interaction. The simple/rich variants within the RACER setting answer a narrower question than a comparison against the original baseline architecture.
More detail can still describe the wrong situation
The project reports two distinct failure sources: the supervisor sometimes misreads the scene and generates hallucinated instructions, and the actor sometimes predicts inappropriate actions. A detailed but incorrect explanation can guide the actor toward the wrong target just as a correct instruction can be executed poorly. Better language coverage does not remove either possibility.
The project’s RACER+H examples allow a person to modify the supervisor’s instruction. Those examples show a different assistance condition and should stay separate from autonomous supervisor results. Similarly, a policy trained with rich instructions being able to follow simpler human corrections is a different question from whether simple and rich training annotations produce equal policies.
The VLA evaluation guide explains controlled language contrasts more broadly. RACER provides a concrete way to apply that distinction: compare simple and rich annotations within the same recovery-data setting, read scratch versus mixed-data training as a transfer comparison, and keep the supervisor’s factual accuracy separate from the actor’s ability to follow its guidance.