FailSafe turns a failed robot trajectory into recovery supervision by pairing a deviated pose with a sampled corrective pose, then replaying the task through both. A candidate enters the dataset when that corrected rollout succeeds. Reversing the original perturbation alone can send the gripper into the object, so the pipeline searches for a usable correction along the trajectory instead.
The method in the FailSafe v4 paper has three distinct steps: generate a failure, collect possible corrections, and verify their effect on task execution. The explanation here follows Sections III-A–E and V; the simulator, planner and trained model were not executed for this article.
Change one stage, then observe what fails
The starting point is a motion-planned task decomposed into stages. Imagine the nominal sequence as A → B → C → D. A configuration specifies the failure type, perturbation range and eligible stage. The wrapper changes one stage pose, producing a rollout such as A → B′ → C → D.
FailSafe defines translation errors, rotation errors and no-ops, where the arm remains stuck for a period. It records a perturbed rollout for action collection when the task ultimately fails. Thus, injecting noise and obtaining a useful failure example are separate events: the task outcome determines whether the candidate proceeds.
The failure may appear later than the changed stage. The paper gives the example of an initially poor grasp causing an object to slip during transport. This is why the corrective action is sought before the failure fully unfolds, rather than only at the terminal image showing that the object has been lost.
Choose a corrective pose instead of negating the noise
For each nominal/failed trajectory pair, the generator selects a deviated pose P_d from the failed trajectory and a corrective pose P_c from the nominal one. Their seven-dimensional pose difference becomes a candidate action, ΔA.
The selection windows are part of the method. The search for P_d begins at the tenth step of the failed trajectory and continues to its end. For the corresponding nominal trajectory, candidate P_c poses are restricted to the window beginning ten steps after its start and ending three steps before its end. The authors choose this window to reduce collisions between the gripper and object. In the no-op case, P_c is sampled three to ten steps ahead of P_d along the nominal trajectory.
These are the paper’s discrete trajectory-step windows. Translating them into milliseconds would require the trajectory’s sampling period; the selection rule itself is expressed in steps.
Why can the correction differ from the injected noise? By the time the arm reaches P_d, it may have advanced along several pose coordinates. The sampled P_c can also lie at a different point on the nominal trajectory. Returning from P_d to P_c therefore represents a state-to-state correction, while negating the original noise only reverses one earlier edit.
For example, an illustrative lateral grasp offset might be followed by upward motion. A usable return could change both lateral position and height. FailSafe explicitly permits adjustments across all seven action dimensions even when the recorded failure label names one dominant error type. The label describes how the failure was induced; it does not constrain the correction to that one component.
Replay decides which pairs become training data
The candidate difference still needs a task-level check. In the verification stage, the motion planner first reaches P_d, then P_c, and then continues through the remaining task poses. The paper depicts this as A → P_d → P_c → B → C → D.
If the originally failed task succeeds on that corrected execution, the pipeline retains the corrective action with the failure observation and type. This acceptance rule gives the pair its meaning: under the reconstructed simulator state and planner, the selected correction supports a successful continuation.
A planner reaching P_c alone would answer a narrower question about motion feasibility. Completing the subsequent task checks whether the correction leaves the robot and object in a state from which the original objective can still be reached. That distinction explains why verification is a separate stage after candidate generation.
The model learns an observation-to-correction relationship
Each dataset entry includes ten consecutive image observations. The collection uses front, side and hand viewpoints, and includes both failure entries and nominal ground-truth examples. The output format asks for the current subtask and a failure judgment; failure entries additionally provide the failure type and recovery action.
That format connects visual evidence to an action target. The authors use it to fine-tune LLaVA-OneVision 7B into FailSafe-VLM, which can supply reasoning and corrections to a VLA. The dataset’s nominal examples also supply cases where a failure correction should not be requested.
The current pipeline concentrates on robot-motion correction. Section V identifies object-level errors as outside its present recovery support. Correcting a poor approach before a slip and finding a new grasp after an object has fallen require different starting states and action generation. Keeping that boundary attached to the example makes the role of the collected pair clear.
For data handling, the robot data-quality audit covers observation/action coherence. FailSafe adds a specific relationship to preserve: the image history and deviated state must still correspond to the corrective target that passed the continuation replay.