The UBTECH GHRC 2026 baseline is an official reference pipeline: run the Walker S2 simulator, teleoperate with a keyboard, capture four cameras plus joint data in LeRobotDataset v3, then train and evaluate ACT or π0. A reliable sequence is to match the Isaac Sim 5.1.0 container and GPU driver, reproduce one complete official run, and only then add custom data. Starting with a large training job makes environment and dataset failures difficult to separate.
Define exactly what the GHRC baseline includes
The official GHRC 2026 technical documentation provides an integrated flow from physics simulation and data collection through model training and policy deployment. The Walker S2 state space is 20-dimensional: 14 arm joints, 4 gripper joints and 2 gripper commands. Four RGB streams come from left and right head cameras and left and right wrist cameras.
Official assets and training data are hosted separately on Hugging Face, and code repositories require submodules and datasets. This is a common implementation for comparison, not a guarantee of a perfect digital twin of every Walker S2 sensor, safety control and dynamic response. A high simulator score still needs Sim-to-Real validation.
| Component | Role in the official baseline | First check |
|---|---|---|
| Isaac Sim | Walker S2 physics environment and sensor rendering | Container version, GPU detection and real-time factor |
| Keyboard teleoperation | Task demonstration and episode generation | Input-device path, control latency and stop key |
| LeRobotDataset v3 | Video, state, action and metadata storage | Camera order, fps and episode-success label |
| ACT | Imitation baseline predicting action chunks | Chunk length, observation window and memory use |
| π0 | Policy baseline combining image, language and action | Checkpoint, preprocessing and task-sentence consistency |
Driver and container compatibility blocks more runs than model code
The documented minimum is a 4-core CPU, 32 GB RAM, 50 GB SSD, GeForce RTX 4080 and 16 GB VRAM. Recommended hardware is 8 cores, 64 GB RAM, a 500 GB SSD and RTX 5080. The ideal line lists 16 cores, 64 GB RAM, a 1 TB NVMe drive and RTX PRO 6000 Blackwell with 48 GB. Training alongside simulation needs additional storage and VRAM.
Ubuntu 22.04/24.04 and Windows 10/11 appear in the OS table, but the Docker and keyboard examples are Linux-oriented. The base image is `nvcr.io/nvidia/isaac-sim:5.1.0`, with CUDA 12.8. The guide recommends the 580 driver line and warns that Docker Isaac Sim may later crash after installing 595. Newer does not always mean compatible.
Check `nvidia-smi`, Docker GPU access, NVIDIA Container Toolkit and storage separately. Launch an empty scene from the untouched image before loading GHRC assets. Installing every dependency at once makes a black screen, renderer crash and missing asset difficult to distinguish. Compare the pinned version with known issues in the official NVIDIA Isaac Sim documentation.
Four cameras and the 20-dimensional state must share one clock
A visible Walker S2 and responsive keyboard do not prove the data pipeline. Confirm that left and right head and wrist streams use the correct names and order. Joint state and command must share timestamps, and metadata must record dropped frames. Swapping cameras can produce a model that trains successfully on the wrong hand–object relationship.
Teleoperation discovers the keyboard at Linux `/dev/input/event*`. Without mounting that device into Docker, the host sees keypresses while the simulator does not. If automatic scanning returns several devices, use the stable `/dev/input/by-id/` link to reduce event-number changes across container runs.
.jpg){kind=link}
LeRobotDataset v3 succeeds or fails on episode definitions
LeRobotDataset v3 stores and loads video, state and action data from several robots in a common form. The official Hugging Face LeRobot documentation brings dataset tools, robot interfaces and training policies into one ecosystem. GHRC uses this structure for both public and newly collected episodes.
Readable files are not sufficient. Fix the episode start, task sentence, success and failure criteria, operator intervention, camera fps and joint rate. `Part_Sorting`, `Conveyor_Sorting`, `Foam_Inlaying` and `Packing_Box` require their own objects and terminal conditions. Silently deleting failures removes recovery evidence and distorts the evaluation denominator.
| Data QA item | Example pass condition | Failure symptom |
|---|---|---|
| Camera mapping | head_left/right and wrist_left/right match the sample | Left and right hands map to the wrong video |
| Time synchronization | Video, state and action align within tolerance at contact | Force commands appear before the grasp |
| Initial state | Object and robot distributions plus reset rules are recorded | The policy succeeds only in one layout |
| Task sentence | One task uses one name and meaning | π0 treats wording changes as different tasks |
| Split | Training and evaluation separate scenes and objects | Validation is high but a new layout fails |
| Failure retention | Stop reason and last valid frame remain | Failure rate and recovery cannot be calculated |
ACT is the simpler pipeline check; π0 asks more of the inputs
ACT predicts short action chunks to stabilize long manipulations. Its input and output structure is relatively explicit, making it a useful check that the data pipeline works. First reproduce official training and inference, then remove a small data slice or occlude one camera to test sensitivity.
The π0 family combines images, natural-language tasks and robot state in a more general policy direction. It may extend across more tasks, but sentence preprocessing, checkpoint compatibility, memory and action representation all need to match. A newer name does not make it better than ACT; compare on the same episodes and success rule. See robot foundation models for context.
Do not compare only training loss and average success. Separate failures by object, long-horizon accumulation, camera occlusion, start-state change and incorrect task text. Changing at least three seeds and splits exposes variance and reduces checkpoint cherry-picking.
Record policy rollouts back into a dataset before scaling training
The official inference example passes `walker_s2_sim` and a checkpoint to `lerobot_record.py`, then saves rollouts as an evaluation dataset. The documentation table enables video and lists defaults of 30 fps, 60-second episodes and 50 evaluation episodes, with an `eval_` naming pattern.
Preserve that structure. A failed rollout must reopen with video, state and action so teams can distinguish perception from control. Manual video scoring easily separates the checkpoint from the environment version. Add the Git commit, asset version, policy hash, random seed and driver to the results dataset for cross-team reproduction.
Reality adds latency, friction error and safety control after simulation
Isaac Sim friction, joint response and camera noise do not exactly match Walker S2. A simulated hand may contact a box while calibration error, communication latency, motor backlash and reflections move the real contact point. This is where the domain randomization and small real-data correction discussed in Sim-to-Real basics become necessary.
Before hardware deployment, keep speed, force and workspace limits outside the learned policy. Start with an empty area and light objects, and confirm immediate stopping when a person approaches. A simulator score selects a policy candidate; it is not safety certification for work around people. See what physical AI means for the wider stack.
Day one should end with a reproducible record, not a leaderboard score
On the first day, build the official environment and run two episodes from the supplied checkpoint, checking saved video, state and action. On the second day, train ACT on a small official subset and reproduce loss and inference. Then run π0 on the same evaluation set before adding a small amount of custom teleoperation data.
Keeping intermediate evidence identifies whether the container, assets, data or model settings broke. GHRC's value is not merely executing a fashionable model; it gives participants a comparable environment, dataset and evaluation flow.
Keep diagnostics beyond the official score. Policies with identical episode success may differ if one stops safely early and the other drops an object at the end. Record median and worst completion time, pre-failure behavior and safety-limit triggers. Results by new object, lighting and initial pose help separate overfit checkpoints from policies that generalize.
Frequently asked questions
Must GHRC teams choose only ACT or π0?
No. The official baseline provides both for comparison. Validate the data flow with ACT first, then measure π0's added benefit and cost under the same evaluation conditions.
Does LeRobotDataset v3 guarantee data quality?
No. It standardizes storage, but users still must verify camera mapping, timing, initial state, labels, and the training–evaluation split.
Will a successful simulation policy work immediately on Walker S2?
No. Physics, sensors and latency differ, and real hardware needs calibration and independent safety limits. Simulation narrows candidates for hardware testing.
Official sources checked
- Official UBTECH GHRC 2026 technical documentation
- Official NVIDIA Isaac Sim documentation
- Official Hugging Face LeRobot documentation
Last checked: August 23, 2026