CIVIL augments robot demonstrations with markers and language so the learner can distinguish task-relevant visual information from incidental background. Those extra teaching signals help train a representation; the deployed policy is designed to work without the teacher replacing markers or supplying prompts for every execution.
Imagine demonstrations in which a blue carton always sits behind the button being pressed. Ordinary imitation data contains both the button and the carton whenever the action occurs. Additional guidance can identify the button as relevant, while an evaluation moves the carton to see whether the learned behavior follows the intended object.
Marker pose and language answer different questions
The CIVIL project describes demonstrations containing robot states, images, actions, marker data and language prompts. Markers identify relevant poses or positions. Language helps indicate objects and visual features that matter to the task.
These channels complement one another. A marker can identify a switch location, while the relevant decision may depend on whether its indicator is lit. Recording a position does not automatically record the visual property used to choose an action.
For a collection protocol, write the intended relevance beside each instrument. Record whether a marker refers to a goal pose, a waypoint or a reference object. Keep the language associated with the correct episode and time interval. An unexplained marker stream becomes another ambiguous sensor field.
Two learning phases remove the teaching instruments from runtime
The paper, Section 4 and Figure 3, describes a first phase that trains features with marker supervision and language-informed image masking while learning the action policy. A second phase keeps the encoder and policy fixed and trains a network to recover the corresponding features from the original image input. The project calls this component a causal network.
The practical separation is between information supplied by a teacher during learning and information expected from the robot during execution. Inspect both input lists when reproducing the method. A training script that receives marker poses and a deployment script that silently still requires them would implement a different workflow.
The method also removes markers from the environment after collection and inpaints their appearance in the training images. That image-processing step deserves its own review: it is intended to prevent the visible teaching instrument from becoming a required visual cue.
Check what the image editing leaves behind
An inpainted region can differ from a naturally unmarked scene. For a proposed diagnostic, compare an original marked image, its processed version and a separately captured unmarked image of the same setup. Inspect object edges, occlusions and textures around the former marker location.
Then vary two things separately. Move an irrelevant distractor while keeping the task-relevant arrangement fixed. In another case, move or change the relevant object while keeping the distractor fixed. Those cases ask whether the action follows the intended evidence. They are proposed evaluations, not experiments performed for this article.
| Record | Question it makes inspectable |
|---|---|
| Marker identity and intended role | What did the teacher mean to identify? |
| Language prompt and associated frames | Which object or property did the teacher describe? |
| Mask and inpainting revision | What visual content was changed before training? |
| Runtime input list | Which teaching signals were actually removed? |
| Relevant-object and distractor variations | Does the action track the task evidence? |
The word “causal” in the model name should be read with the method and evaluated interventions. A highlighted region or a plausible explanation alone is insufficient to identify which visual dependency actually controls an action.
Measure useful teaching, not only demonstration count
The GELLO collection guide focuses on operator-to-robot motion mapping. CIVIL adds a different collection decision: whether extra relevance annotations make the resulting data more informative for a visual policy.
Track annotation time, unusable episodes and policy behavior under changed scenes alongside the number of demonstrations. A smaller set is useful only if the intended task conditions remain covered. Retain examples where the teacher’s marker or language was ambiguous so the collection process itself can improve.
Sources were checked October 16, 2026. No CIVIL model was trained or tested here. The actionable result is a data specification that distinguishes teaching instruments, edited training images and the sensor inputs available during deployment.