A robot world model can predict a different future for the same commanded motion when joint targets are written as absolute positions instead of offsets from the current state. Converting the numbers correctly is necessary, but it does not make a predictor trained on one form behave consistently on the other. Before using its forecasts to choose actions, test the two forms as paired inputs to the same model.
The distinction is narrower than general action tokenization or cross-robot action normalization. Those guides address how commands are represented or mapped to hardware. The question here is whether an already trained action-conditioned predictor gives the same answer when a physical command is expressed two ways.
The subtraction that can change a prediction
For a position target, let the current joint state be s_t, the absolute target be a_abs, and the offset be a_rel = a_abs − s_t. If a joint is at 0.40 radians and the target is 0.45, an absolute value of 0.45 and a relative value of 0.05 describe the same target, provided the frame, unit, controller mode and timing also match. The offset is state-dependent: a bare 0.05 without the matching s_t is not enough to reconstruct the target.
That example is a contract check, not a claim that every absolute/delta dataset is interchangeable. Velocity commands, integrated deltas, clipped targets or differing control rates can describe different motions even when arrays have the same shape. First prove equivalence for the state and action window actually supplied to the predictor. Only then does divergent output indicate sensitivity to notation rather than a changed physical instruction.
The September 2026 Robot World Models Are Not Invariant to How the Actions Are Written paper tests this mismatch with a frozen visual encoder and a predictor of future latent representations. Its absolute-target and relative-offset inputs use the corresponding state.
What the offline result shows—and what it cannot
On the paper’s ALOHA mobile-cabinet test, switching to the equivalent held-out encoding reduced top-one action selection from 53.1% to 15.4% among 32 candidate windows. This is an author-reported offline action-selection result, not a robot task-success rate.
The result isolates a possible handoff failure. A planner may pass valid relative commands into a dynamics model trained on absolute targets and receive plausible-looking, inconsistent forecasts. A low error on the original encoding will not reveal that handoff. The result does not establish how an actual robot would execute those selections, or how a larger, jointly trained model would respond.
Run a paired-input audit before trusting a planner
A useful local audit needs one held-out trajectory and two views of each exact action window. Keep observations, state, target time and candidate set fixed. Convert commands using the corresponding state, then verify the round trip to the original absolute targets. Record any clipping, missing state, frame change or timing mismatch before comparing model outputs. If a conversion cannot be reconstructed, the two inputs are not a valid invariance test.
| Check | Evidence to keep | Interpretation |
|---|---|---|
| Command equivalence | State and action schema, units, frame, controller mode, sampled horizon, round-trip error | A failed check means the commands may differ physically. |
| Prediction agreement | Both predicted latents for every paired window; mean and lowest cosine agreement | Average agreement can hide an opposed prediction on a consequential case. |
| Task effect | Retrieval rank and fixed-candidate action choice on episode-separated cases | Compare against each encoding’s own baseline; do not call this hardware success. |
If an absolute-trained model disagrees on a paired window, trace its state, action record and forecast. Check first for a pipeline mistake, such as subtracting a different reference state from the one specified by the action schema. If conversion is sound, stop treating the forms as exchangeable at that model boundary. Test the action ranges and contact phases the application will encounter, beyond a corpus average.
This is distinct from shuffling actions between episodes. Shuffling can show that the predictor reads the action channel, but it changes the command and therefore cannot show whether two writings of one command yield the same prediction. Changing a camera or dropping part of an action history may remove information, even if both inputs retain the same label. An audit should distinguish those cases before imposing an agreement target.
Training on both forms is a baseline, not a universal cure
A useful training comparison fits the same future target under both encodings while holding the data split and compute budget fixed. The paper finds that an extra consistency penalty helps worst-case forecast agreement without a meaningful average action-selection gain over training on both forms.
That suggests two separate measurements for a proposed fix. First, does the predictor choose the appropriate action more often? Second, does it avoid a few severe disagreements that an average hides? Keep both columns in the report. A smaller disagreement is not automatically a better forecast: two identical wrong answers would agree perfectly. Compare each prediction with the held-out future as well as with its paired prediction.
Also separate averaging training losses from averaging forecast vectors. The former asks one model to learn from both inputs; the latter constructs a new forecast after inference. For example, if one forecast points toward grasping and another toward withdrawing, their midpoint has no automatic claim to be the correct next state. Evaluate that combination as a new estimator rather than treating it as a free consistency fix.
Finally, test cases near the intended action-range limits and at contact transitions. A model could appear stable on easy, almost static examples while disagreeing on the action windows that change the outcome. Keep the same difficult cases across experiments, and record whether a proposed improvement shifts errors from one encoding to the other.
The practical decision is model-specific. Preserve the action form in the dataset and model interface; adapt a mismatched input explicitly or train and evaluate both forms. Recheck the tails and the downstream choice, not just the mean forecast. This paper gives a reason to demand that evidence before a world model guides a robot; it does not supply a hardware safety guarantee or a universal penalty weight.
No predictor, dataset or robot was run for this article. Source checked September 29, 2026.