A robomimic-compatible HDF5 file supplies the starting demonstration, but MimicGen also needs to know how recorded commands relate to the robot’s end-effector targets, which objects each movement refers to, and where one subtask ends. Prepare those interfaces and boundaries before asking it to generate more trajectories. Otherwise, a perfectly readable demonstration can become the wrong movement in a new scene.
MimicGen generates new demonstrations by adapting and composing object-relative motion segments. Consider a robot that picks up a block and places it on a base: the pickup refers to the block, while placement refers to the base. That change of reference is why a video, action array, or successful replay alone cannot supply everything the generator needs. The broader synthetic-data guide covers data quality and transfer to real conditions; this article follows the preparation of one source task.
The following details come from the official MimicGen tutorials on the repository’s main branch, checked October 15, 2026. They describe the documented workflow, not a tested installation or a generation run performed for this article.
Give each motion an object and a controller meaning
The custom-environment tutorial separates simulator-wide conversion from task-specific information. The simulator interface supplies the current end-effector pose, converts between environment actions and target end-effector poses, and extracts the gripper command. The returned end-effector pose must use the frame expected by that controller.
This distinction matters when two systems store equally sized action vectors. One might encode a scaled relative movement while another expects an absolute target. Matching vector dimensions does not establish that the commands describe the same motion. Check the frame, scaling, rotation convention, relative or absolute interpretation, and gripper slice against the selected interface before reusing a recording.
The task interface then supplies named object poses through get_object_poses. The documented return values are 4×4 pose matrices. Each subtask’s object_ref in the task configuration must match one of those names. MimicGen observes the relevant object’s pose at the start of an object-centric subtask; choosing the wrong reference changes what the movement is adapted around.
For a hypothetical block-and-base task, a pickup segment named block and a placement segment named base express different relationships even if the robot uses the same controller throughout. This is an explanatory example, not a claim that an included configuration implements that task. Before generation, write down the object reference for each segment and check that its dictionary key resolves to the intended physical object.
A brief grasp signal can cut the demonstration too early
The same tutorial specifies how get_subtask_term_signals is used when source demonstrations are parsed: the first transition from 0 to 1 identifies the end of a subtask. It is a boundary annotation for the source trajectory. It should not be interpreted as an instruction to keep watching that signal for a successful grasp during every generated rollout.
Imagine a source grasp signal that changes from 0 to 1 at frame 40, returns to 0 at 41, and becomes 1 again at 75 when the grasp is stable. Under the documented first-transition rule, the early event supplies the boundary. Waiting until frame 75 in your own interpretation does not repair the annotation consumed by the generator. These frame numbers illustrate the failure mode; no recording was measured here.
Inspect the proposed cuts on the source demonstrations before increasing the generation count. If an environment heuristic is unreliable, the documented alternative is manual segmentation with annotate_subtasks.py, after source preparation. The interface may return an empty signal dictionary for that route, and the annotation signal names must still agree with the task configuration. The final subtask needs no termination signal because its end is the end of the source demonstration.
Also inspect subtask_term_offset_range in the configuration: the example allows an offset to the detected boundary. A sensible signal and a poorly chosen offset can still produce an unsuitable segment. The useful unit of review is the resulting cut and its object reference, not merely the existence of a binary signal.
Prepare the source before expanding the collection
The official getting-started workflow first requires a robomimic environment wrapper and source metadata that points to it. prepare_src_dataset.py then uses the selected environment interface to add the per-timestep information required for generation. A custom task also needs a registered task configuration describing its ordered subtasks.
Use this preparation order for a first inspection:
- Confirm that the source metadata resolves to the intended environment wrapper and controller configuration.
- Check pose/action conversion and the named object-pose dictionary for that task.
- Prepare the source dataset, then inspect heuristic cuts or add manual annotations.
- Match the task configuration’s object and signal names to the prepared data; inspect its boundary offsets.
- Start a small generation attempt and retain the configuration, outcome statistics, and available failure examples for diagnosis.
The last step is a proposed validation action. We have not run it here, and successful source preparation by itself supplies no measured generation success rate.
Decide whether the requested count means attempts or successes
In the tutorial’s small example, NUM_TRAJ = 10 with GUARANTEE = False means ten generation attempts. The tutorial warns that the unedited example instead continues until it collects 1,000 successful trajectories. A success-target setting therefore does not promise that every attempt succeeds or that collection finishes within a fixed time.
Keep attempts and successful outputs separate when judging the prepared task. For example, six successes from ten attempts would leave six successful demonstrations, not ten; this is hypothetical accounting, not a MimicGen result. The documented output includes important_stats.json, the run configuration, a successful-demonstration file, and a failed-demonstration file whose retained size is limited. A failure file’s length should therefore not be assumed to equal the number of failed attempts.
If a first inspection fails, return to the affected interface, reference object, or source boundary before collecting more. More trajectories cannot establish that those meanings were correct, and a generator’s successful output still needs its own downstream policy evaluation.