Cross-embodiment action normalization maps commands from robots with different joints, tools and controllers into a representation that a shared policy can learn. The goal is semantic alignment, not merely padding every vector to the same length.
A seven-dimensional end-effector action can still differ in frame, rotation convention, absolute or delta meaning, control period and gripper semantics. The inverse mapping to each robot must respect reach, joint limits, controller type and safety envelope.
Use this guide with the robot action-data guide and robot data-quality audit. Preserve native actions and reversible transforms so failures can be traced back to physical commands.
Normalize meaning before vector length
Start with a schema for command type, controlled frame, units, rotation convention, reference state, time interval, limits and missing dimensions. Two arrays of equal shape are incompatible if one is joint position and another is Cartesian velocity.
Keep native field names and documentation with each dataset. A common tensor is a derived view, not a replacement for the original robot contract.
Choose a common action representation deliberately
Cartesian end-effector translation, rotation and gripper command is common because it abstracts some joint differences. It does not capture every mobile base, multi-finger hand, bimanual relationship or force-control mode.
Define extensions and validity masks instead of pretending unsupported dimensions are equivalent. A policy architecture may need embodiment tokens or robot-specific action heads when one universal space loses essential control meaning.
| Native action | Common mapping option | Required metadata | Main loss risk |
|---|---|---|---|
| Joint position | Forward kinematics to tool pose | Robot model and joints | Redundancy |
| Cartesian delta | Common frame delta | Frame and rotation rule | Reference mismatch |
| Velocity | Delta over declared interval | Control period | Time scaling |
| Binary gripper | Normalized open or close state | Tool direction and limits | Force meaning |
| Multi-finger hand | Latent or robot-specific head | Finger schema and synergy | Dexterity collapse |
Separate absolute, delta and velocity commands
Absolute pose specifies a target, delta specifies a change from a reference, and velocity specifies change per time. The same small number has different meaning and error behavior in each interface. Labeling all of them action without type creates silent scaling failures.
Convert through an explicit state and time interval. Store which observation defines the delta reference and whether rotations compose in local or world coordinates.
Preserve coordinate frames and rotation conventions
Record origin, axis direction, handedness and whether translation is expressed in base, world, tool or camera frame. For rotation, preserve quaternion ordering, axis-angle convention, Euler sequence and composition order.
Run signed unit tests: positive x, y, z and positive rotations should move every robot in the declared physical direction. Numeric distribution checks alone can miss mirrored or permuted axes.
Invert the common action through each robot stack
The target robot needs inverse kinematics, limits, collision checks and a controller interface that converts the common action into an executable command. The same Cartesian delta can require different joint motion and may be unreachable for one embodiment.
Validate the inverse mapping with the exact controller used in deployment. Clipping after normalization can change direction or timing, so expose saturation and infeasibility instead of returning an apparently valid command.
Define gripper semantics per tool
Grippers may accept finger width, motor position, force, velocity or a binary state, with open and close directions reversed across drivers. Multi-finger hands add synergies and contact modes that one scalar cannot preserve.
Store physical range, units, direction, neutral behavior and whether the command is position or effort. Test empty motion, object contact, stall and release after inverse mapping.
Carry validity masks for absent dimensions
Padding a missing base or finger dimension with zero makes absence indistinguishable from a commanded zero. A model can learn dataset identity from padding or output unsupported motion without a clear rejection path.
Use masks and embodiment metadata, then define how the loss and policy head treat invalid fields. Never send an unmasked common vector directly to hardware.
Preserve dataset-specific normalization statistics
Mean, standard deviation, min-max or percentile scaling can make different datasets numerically comparable, but global statistics may let large datasets dominate and hide robot-specific ranges. Statistics also leak information if computed across evaluation splits.
Version statistics by feature, dataset, embodiment and split. Keep physical units recoverable and audit saturation after denormalization.
Align control rate and action horizon
A 20 Hz delta and a 100 Hz delta with the same numeric value imply different physical velocity. Action chunks further encode a horizon whose duration depends on rate and execution policy. Resampling must preserve time meaning, not only sample count.
Use the robot action-chunking guide to define chunk execution and replanning. Record sample period, latency and whether the controller interpolates, holds or queues commands.
Learn from cross-embodiment datasets without erasing provenance
The Open X-Embodiment project aggregates data from many robots and tasks and provides important evidence for cross-embodiment training. Its paper describes a shared action-space approach while retaining dataset-specific transformations.
The Octo project is another generalist-policy example. Treat reported gains as evidence under their datasets and evaluation, and keep robot, task and source contributions visible in your own mixture.
Test round trips in arrays and on hardware
First map native action to common space and back, then compare type, direction, magnitude, frame and saturation. Next replay bounded commands in simulation or a restrained robot and verify measured motion, gripper behavior and timing.
Inspect per-robot distributions and task outcomes. Average training loss can look healthy while one robot consistently flips yaw, saturates reach or closes the gripper backward.
| Test | Input | Pass condition | Failure exposed |
|---|---|---|---|
| Schema | Native metadata | Complete type, unit, frame, rate | Ambiguous action |
| Numeric round trip | Signed basis commands | Bounded reconstruction error | Scale or axis error |
| Limit test | Near-boundary actions | Explicit saturation and mask | Silent clipping |
| Timing test | Rate and chunk variants | Equivalent physical duration | Velocity mismatch |
| Hardware replay | Safe bounded sequence | Expected physical motion | Controller semantic error |
Release with a normalization contract
Document native schemas, common representation, frame conversions, rotation math, statistics, masks, controller inverse maps, rates, limits and versioned tests. Preserve original actions beside normalized data whenever licensing and storage allow.
Close validation with the following checklist.
- Label action type, frame, units and time meaning.
- Preserve native commands and reversible transforms.
- Mask absent dimensions instead of unexplained zeros.
- Version statistics by dataset, robot and split.
- Verify signed round trips and restrained hardware execution.
Frequently asked questions
Does making every action seven-dimensional finish normalization?
No. Frame, rotation, command type, timing, limits and gripper meaning can still differ.
Can joint and Cartesian actions train one model?
Yes with explicit transformations or robot-specific heads, but their semantics and missing information must remain visible.
Should statistics be computed once over all robots?
Not automatically. Preserve per-dataset and per-robot statistics and avoid evaluation leakage or dominance by large sources.
Can unused dimensions simply be zero?
Only with an accompanying validity mask and model behavior that distinguishes absent from commanded zero.
What is the fastest trustworthy check?
Send signed basis actions through common and inverse transforms, then verify physical direction, magnitude, timing and gripper behavior safely.
Common-Action and Robot-Specific Command Boundary
A common action tensor is safe only when its physical meaning remains reversible. Preserve native semantics, validity and timing through every dataset and robot-specific inverse map.