Before pooling visual-navigation trajectories for GNM, verify the local coordinate frame and both sides of the action-scale conversion. In our CPU check of the official coordinate helpers, a synthetic 0.5-meter forward waypoint became a normalized value of 1; the repository’s example deployment settings then mapped that value to 0.05 meters on the target robot. Neither value is a universal physical distance. The GNM paper motivates shared actions and temporal context across robots, but each dataset and target adapter still needs an explicit conversion convention.
Suppose a team has forward-camera logs from a small indoor base and a larger outdoor robot. Both record RGB images and two-dimensional positions, so combining folders looks easy. Yet one camera samples at a different rate, one odometry stream uses a different frame convention, and their waypoint spacing differs. If the model is trained on raw displacement per frame, the fast robot may dominate large moves while the slow robot teaches near-zero commands for similar-looking corridors. GNM’s practical lesson is to preserve enough context to interpret each command, then test the conversion back to each robot separately.
Keep the episode intact before normalizing the action
The official GNM code and checkpoint repository describes a processed trajectory as ordered forward RGB images plus odometry containing planar position and yaw. Its README’s custom-dataset example calls the scale metric_waypoints_distance, but the pinned implementation and sample configuration use metric_waypoint_spacing; the executed check below shows the difference. The dataset normalization and target robot’s deployment scaling also have distinct roles and should be recorded separately. A trajectory is a timed sequence, not a bag of unrelated frames. The repository also says some paper training data were not publicly released; reproducing the original mixture from downloads alone should not be assumed.
| Field to retain for each source | Why the shared model needs it | A failure a pooled dataset can hide |
|---|---|---|
| Image order, capture rate and history window | A turn becomes interpretable from motion across frames. | Two datasets offer the same frame count but different elapsed time. |
| Pose and yaw frame, units and timestamp association | Relative waypoints need a consistent local reference. | A left turn in one frame convention becomes rightward supervision in another. |
| Robot speed and command-conversion rule | The normalized action has to map back to feasible motion. | A waypoint is reachable on one platform but exceeds another’s limits. |
| Dataset and robot identity | Performance must be attributable by source and embodiment. | Aggregate success rises while a minority robot regresses. |
Write down how images align with pose samples, how gaps and resets are treated, and whether a source’s camera orientation changes. In the paper’s shared abstraction, local odometry yields relative planar waypoints and yaw change; each robot’s waypoint is scaled by a factor tied to its top speed, and its controller unnormalizes and tracks the prediction. Keep that factor with each source and target adapter. The learned embodiment context comes from recent images, not a required hand-coded robot-description input. The table is an engineering audit proposed here; it is not a claim that GNM’s public code automatically detects every inconsistency.
The broader Octo observation and action binding guide addresses the checkpoint-to-robot interface. GNM asks an earlier, navigation-specific question: whether trajectories from several robots describe comparable motion before they become training targets. The global planner and local controller guide is useful when assigning responsibility for route selection and low-level tracking after the learned policy proposes a local move.
A waypoint can be right in meters and wrong for the target robot
In an AI-assisted CPU check, we executed the official yaw_rotmat and to_ego_coords functions on a CPU on October 4, 2026, using Python 3.12.14 and NumPy 2.3.5. The inspected GNM helper source was pinned to commit 94dd70b. Only those two functions were extracted from the module; its Torch and image-processing dependencies were not imported. The inputs were synthetic positions, not recorded odometry or learned predictions.
Place the robot at world position (10, 20) meters, facing positive world Y: yaw is π/2 radians. Future positions (10, 20.5), (9.5, 21) and (9, 21.5) become local forward/left coordinates (0.5, 0), (1, 0.5) and (1.5, 1). The inverse transformation recovered all three world positions, with zero measured floating-point error for this fixture. Five separately constructed heading cases also recovered their expected local coordinates within a 1e-12-meter tolerance.
| Conversion of the first waypoint | Observed local result, meters | What to check |
|---|---|---|
| Official helper, yaw π/2 | (0.5, 0) |
Positive local X is forward in this fixture. |
Official helper, yaw 90 |
(0.4470, -0.2240) |
NumPy trigonometry receives radians; a degree value silently changes direction. |
| Our deliberately transposed rotation | (-0.5, 0) |
Copying a column-vector convention into this row-vector calculation reverses the example. |
The dataset normalization expression divides XY by metric_waypoint_spacing * waypoint_spacing. We evaluated that exact expression with 0.25 meters and a stride of 2 samples: the three local waypoints became (1, 0), (2, 1) and (3, 2). Omitting the stride doubled every normalized displacement.
The deployment conversion uses a different factor: max_v / frame_rate. With the checked-in robot settings, 0.2 m/s divided by 4 Hz gives 0.05 meters. Feeding our normalized fixture through that expression produced (0.05, 0), (0.10, 0.05) and (0.15, 0.10) meters. This target scaling is not the inverse of the source dataset’s normalization. It also does not establish that a controller can track those waypoints.
There is a configuration trap in this revision: the README example names metric_waypoints_distance, while the executable expression reads metric_waypoint_spacing. Evaluating that expression with only the README key raised KeyError: 'metric_waypoint_spacing'; using the implementation’s key produced the values above. This was an isolated lookup test, not an end-to-end dataset-loader failure.
To reproduce the numerical checks in a Python environment with NumPy, run this compact equivalent of the 2D conversion. It was checked against the extracted official helper for the fixture above; the scale operations mirror the inspected expressions.
import numpy as np
np.set_printoptions(precision=4, suppress=True)
origin = np.array([10., 20.])
future = np.array([[10., 20.5], [9.5, 21.], [9., 21.5]])
def local(yaw): # compact 2D equivalent of the inspected helper
c, s = np.cos(yaw), np.sin(yaw)
return (future - origin) @ np.array([[c, -s], [s, c]])
xy = local(np.deg2rad(90))
np.testing.assert_allclose(xy, [[.5, 0], [1, .5], [1.5, 1]], atol=1e-12)
print("local meters:", xy)
print("90 passed as radians:", local(90)[0])
cfg, stride = {"metric_waypoint_spacing": .25}, 2
normalized = xy / (cfg["metric_waypoint_spacing"] * stride)
target = normalized * (.2 / 4) # max_v m/s divided by frame_rate Hz
print("normalized:", normalized)
print("target meters:", target)
print("stride omitted:", xy / .25 * (.2 / 4))
try:
{"metric_waypoints_distance": .25}["metric_waypoint_spacing"]
except KeyError as err:
print(type(err).__name__, str(err))
Before mixing real logs, preserve this small known-direction test for each adapter, check the selected stride, and test the target conversion separately. These results cover coordinate and scale handling only. We did not execute GNM inference, ROS, a tracking controller or a physical robot; they provide no measurement of navigation success or cross-robot transfer.
Judge transfer by platform, not only by pooled success
The GNM project page groups GNM with later ViNT and NoMaD research and links code and data resources. It presents cross-embodiment visual navigation and zero-shot transfer as author results. To assess a new dataset mixture, reserve entire trajectories, environments and robot instances for held-out tests rather than splitting neighboring frames across train and test. Report success, path length, collision or intervention count, and command-feasibility violations separately for every platform. An aggregate score is insufficient if one robot contributes many more easy episodes.
A practical ablation starts with a single-source model, adds each new robot’s data under the same observation and action convention, then compares per-platform changes. If a source improves one robot and degrades another, inspect scale, camera and timing metadata before concluding that diversity itself is harmful. A visually similar hallway can still require different steering because of wheelbase, camera height or actuation delay. Those are hypotheses to test with matched routes, not measurements supplied by this article.
Before hardware use, replay a handful of normalized model outputs through each robot’s adapter without commanding motion. Check axis signs, bounds, rate and stop behavior. Then use guarded trials in representative environments. The paper’s quadrotor evaluation restricted flight to a horizontal plane; it does not establish unrestricted 3D navigation. The acceptance check remains local: the target platform must receive commands it can execute and recover from. The public repository describes an older Ubuntu/CUDA stack and notes that its training/deployment infrastructure was later subsumed by the shared visualnav-transformer repository; pin the artifact version you actually evaluate. No model training or robot run was performed for this article.