An Octo checkpoint needs an explicit binding between camera inputs, task requests, action statistics and robot commands. A matching action-vector length is not enough: in our AI-assisted NumPy reconstruction of the released restoration rules, omitting a statistics mask changed a synthetic component from 0.75 to 10.5 without changing the output shape. No checkpoint or robot was run. Start from the checkpoint specification, then bind both the input interface and the meaning of its restored actions.
Read the checkpoint before building an adapter
Start with a named checkpoint and record its exact version. The README demonstrates loading an Octo 1.5 checkpoint and printing its specification. Treat that specification as the first integration document: list the expected image keys, temporal observation shape, task conditioning, and action output. Then make a separate sheet for the new robot. Do not infer the robot’s action semantics from a matching vector length.
| Contract field | Questions to settle before training |
|---|---|
| Cameras | Which physical view maps to each image key, and what crop, size, and timing reach the model? |
| Proprioception | Which measured joints, gripper state, or tool state are present, in what order and units? |
| History | Which frames are real observations and which are padding at episode start? |
| Task | Is the request language, goal image, or both, and does that match the training examples? |
| Action | What does each output component command, over what interval, and through which controller? |
This mapping sheet is a proposed engineering artifact, not an Octo output. It makes a shape-compatible but semantically wrong adapter visible. For example, a seven-number action could be interpreted as joint targets by one controller and tool-pose change plus gripper by another. Successful tensor construction would not make those controls equivalent.
Distinguish absent data from padded time
The README FAQ distinguishes timestep_pad_mask (valid history positions) from pad_mask_dict (available inputs within a timestep). Missing observation keys count as unavailable. The documented pretrained history uses two timesteps; a one-step window marks its only observation valid.
This distinction matters for a robot with one overhead camera and no wrist camera. Repeating the overhead image under the wrist key would assert a second viewpoint that was never measured. Leaving the wrist input unavailable tells the model the truth. Likewise, copying the first frame into the previous-time slot without masking it asserts an observed history that does not exist. Check both masks on the first several steps of a recorded episode, not only after the history buffer fills.
An original diagnostic is to save one fully decoded sample before any optimization: image views as the model receives them, the proprioceptive vector with labels and units, task conditioning, observation masks, and the corresponding desired action. Ask a person familiar with the robot to verify that this sample describes the physical scene and intended command. It is cheaper to find a swapped camera or reversed gripper convention here than after a training run.
Training adds a separate action_pad_mask. In the pinned action-head implementation, it has the action tensor’s shape and marks non-padding action dimensions. The continuous head combines it with the timestep mask when calculating loss. Check this mask when replacing an action head: padded targets should not silently become training labels. This is loss masking, not a controller safety limit.
The statistics mask changes values without changing their shape
Octo’s action-restoration code at revision 241fb35 uses a separate mask inside unnormalization_statistics. In normal mode, selected components become action * std + mean; unselected components retain their predicted values. If that mask is omitted, every component is selected. This statistics mask has a different job from the observation and training-loss masks described above.
We executed an AI-assisted NumPy reconstruction of these restoration rules with Python 3.12.14 and NumPy 2.3.5. No JAX operation, Octo checkpoint, transformer, action sampler or robot was executed. Our synthetic prediction was [1, -1, 0.75, 99]; the three-component statistics caused the isolated restoration stage to retain only its first three entries. The full sampling method also passes an embodiment dimension to the action head; this four-entry fixture does not establish a real checkpoint’s output shape.
For the first two rows, we supplied mean [0.1, -0.2, 9] and standard deviation [0.02, 0.04, 2]. These invented values have no assigned robot units and are not statistics from a released dataset.
| Change to the synthetic restoration | Result |
|---|---|
Normal mode; mask [True, True, False] |
[0.12, -0.24, 0.75] |
| Same statistics; mask omitted | [0.12, -0.24, 10.5] |
| Same mask; different mean and standard deviation | [0.7, -0.3, 0.75] |
Bounds mode with p01 [-0.1, -0.2, 0], p99 [0.1, 0.2, 1] |
[0.1, -0.2, 0.75] |
| Same bounds; first predicted value changed from 1 to 2 | [0.2, -0.2, 0.75] |
Every result has the same length. A shape check alone therefore misses both the changed statistics and the lost mask. In the third row, the alternative mean was [0.5, 0.1, 9] and standard deviation [0.2, 0.4, 2].
The bounds branch applies an affine conversion from the predicted scale using p01 and p99. It does not clip inside this restoration block: the last row extrapolates beyond the supplied first-channel p99 of 0.1. We deliberately injected a value of 2; this is not evidence that a particular Octo sampler produces that value. Neither branch is a controller safety limiter.
This NumPy snippet reproduces all five rows. It is our isolated translation of the selected rules, not an Octo inference example.
import numpy as np
def restore(action, stats, mode="normal"):
basis = stats["mean"] if mode == "normal" else stats["p01"]
mask = np.asarray(stats.get("mask", np.ones_like(basis, dtype=bool)))
action = np.asarray(action)[..., :len(mask)]
if mode == "normal":
converted = action * stats["std"] + stats["mean"]
elif mode == "bounds":
converted = (action + 1) * (stats["p99"] - stats["p01"]) / 2 + stats["p01"]
else:
raise ValueError("unknown mode")
return np.where(mask, converted, action)
a = np.array([1., -1., .75, 99.]) # synthetic prediction, not model output
s = {"mean": np.array([.1, -.2, 9.]), "std": np.array([.02, .04, 2.]),
"mask": np.array([True, True, False])}
print("normal masked:", restore(a, s))
print("mask omitted:", restore(a, {k:v for k,v in s.items() if k != "mask"}))
wrong = {**s, "mean":np.array([.5, .1, 9.]), "std":np.array([.2, .4, 2.])}
print("other statistics:", restore(a, wrong))
b = {"p01":np.array([-.1, -.2, 0.]), "p99":np.array([.1, .2, 1.]),
"mask":s["mask"]}
print("bounds:", restore(a, b, "bounds"))
print("bounds input 2:", restore(np.array([2., -1., .75, 99.]), b, "bounds"))
We also checked a (2, 3, 4) synthetic batch: restoration produced (2, 3, 3), applying the same component mask across its leading dimensions. One hundred seeded normal-mode samples recovered the two transformed inputs on inverse conversion within 1e-12; the masked third channel stayed exactly unchanged. Those controls verify this reconstruction’s broadcasting and arithmetic, not JAX device or dtype equivalence.
Bind the normalization mode, statistic vectors, mask and dataset identity to the checkpoint and controller adapter. Print one action before and after restoration with named dimensions and units. If the shape matches but a component changes unexpectedly, inspect that binding before retraining the model. Quantile endpoints describe the chosen statistics; they do not establish physical command limits.
Decide what can be reused
Octo’s project page describes language and goal-image conditioning, different camera and proprioceptive inputs, and fine-tuning to new action spaces. Its authors evaluated nine real-robot setups, including examples with new force/torque observations and joint-position actions. These are reported author conditions, not evidence that your robot will work without an adapter or target data. The project overview establishes the design scope.
The repository provides head_only, head_mlp_only, and full fine-tuning modes. Choose among them only after the I/O mapping is fixed. A new output convention may require replacing the action head; freezing a mismatched head is not a small-data shortcut. A substantially new observation may need a new tokenizer and training signal. The official new-observation/action example demonstrates removing a wrist-camera tokenizer, adding proprioception, replacing the action head for a 14-dimensional bimanual example, and merging compatible pretrained parameters. The new components do not inherit trained weights merely because other parameters are reused.
Do not turn those example dimensions or horizons into defaults for another platform. Instead, compare adaptation scopes on one held-out validation set with the same target demonstrations and action definition. Record which modules were initialized anew, which were frozen, and which checkpoint supplied the retained weights. A lower training loss is useful for debugging but does not tell you whether commands work when the robot closes the loop.
The robot action-data guide covers episode-level records. This Octo mapping should resolve every retained field to a specific model input or controller command.
Validate the bound model at the controller boundary
Run an offline pass first: verify input shapes, missing-key behavior, sampled action dimensions, numeric range, and mapping back to the controller. Check a few known episodes for sign errors, frame confusion, stale images, and gripper inversion. Then measure the entire closed-loop cycle on the actual setup: capture, preprocessing, model inference, action conversion, controller update, and observation of the result. A fast isolated model call cannot establish the delay of that complete path.
In supervised rollout, start with a narrow task and record each commanded action beside the measured robot state. If the model recognizes the scene but commands a motion in the wrong frame, revisit the output binding before changing the backbone. If the command is correctly interpreted but the model does not adapt to the new observation, inspect coverage of the target demonstrations and the chosen tuning scope. If timing makes observations stale, the problem is in the deployed loop even if offline validation looks good.
The useful handoff is a versioned I/O contract plus a small reproducible validation set. Pin the checkpoint, repository revision, camera and proprioception transforms, task format, masks, action head, and controller mapping. Octo offers a modular path to new embodiments; the integration succeeds only when those named components describe the same physical robot at training and inference time.
The added NumPy experiment reconstructs selected action-restoration rules. No checkpoint, JAX runtime, training job or robot rollout was executed.