LeLaN learns language-conditioned object approach from videos that lack recorded robot actions. Its training pipeline adds object descriptions, estimated object locations and navigation supervision. At deployment, a local policy approaches the described object; reaching an object outside the current view requires a separate search and navigation stage.
This distinction answers a common question about learning from walking videos: the camera recording supplies varied visual experience, but additional models must turn that experience into a robot-learning target. “Action-free” describes the starting data, not the entire supervision pipeline.
The labels contain more than an object name
The LeLaN paper, Sections 3.1–3.2, describes an annotation pipeline that finds object regions, generates descriptions and turns accepted descriptions into navigation instructions. Depth estimation supplies an approximate object location. A pretrained navigation model supplies additional behavior supervision.
These stages create different opportunities for error. An instruction can describe the right object while the estimated location is wrong. A location can be plausible while an object crop includes a neighboring object. The action target also reflects the navigation prior used to produce it. Treat those labels as related estimates with separate origins.
For a hypothetical corridor recording containing a chair and a plant, retain which crop generated each instruction and which depth estimate generated its position. If an instruction says “the plant beside the chair,” the useful audit is whether the crop and position identify the plant. A fluent sentence by itself does not resolve that association.
Reaching and avoiding obstacles need different training signals
The paper trains virtual velocity sequences through a differentiable kinematic model. Its objective includes target reaching, trajectory smoothness and a term distilled from NoMaD for collision-avoidant behavior. The authors describe a distance-dependent mask on that last term near the target. That detail is part of their training formulation, not a general stopping-distance setting for another robot.
A useful way to read the method is to ask what each supervision source contributes:
| Training information | What it contributes | A diagnostic question |
|---|---|---|
| Object description and instruction | Which visible object is requested | Does the text still refer to the selected region? |
| Estimated object position | Where the approach should end | Is the depth appropriate for the camera geometry? |
| Navigation-model trajectory | A motion prior around obstacles | Which behavior is inherited from the teacher? |
| Smoothness objective | Preference over changes in commands | Does the resulting motion still reach the intended location? |
The table is an interpretation of the pipeline, not a reproduced ablation. It helps separate an annotation failure from a motion-learning failure before collecting more footage.
A distant target introduces a handoff
The project’s long-distance example searches a topological memory of previously collected images for the target. ViNT first navigates toward the selected node. LeLaN then handles the final object approach. The memory and graph-navigation stage provide context that the local object-approach policy does not obtain from its current view alone.
Suppose two rooms contain similar chairs. Selecting an image of a chair in the wrong room is a memory-retrieval error even if the final approach works perfectly. Conversely, arriving at the correct room and approaching the wrong chair points toward local language grounding. Keep those outcomes separate when reviewing a failed instruction.
The existing ViNT goal-interface explanation covers how goal representations enter that navigation policy. Here the additional concern is when control switches from a retrieved place to a language-described object.
Build an evaluation around the interfaces
For a first evaluation, distinguish visible-object instructions from instructions requiring memory. Record the selected object region, retrieval node, arrival condition and final approach outcome. Include similar objects and cases where the requested object is absent from the stored views. Those cases expose different decisions rather than collapsing everything into one route-success number.
The paper and project were checked on October 16, 2026. No policy or robot was run here, and the authors’ reported experiments are not measurements of a new deployment. The useful implementation record is the chain from video region to language label to navigation target, followed by a visible transition from place-level navigation to object approach.