NoMaD uses one visual navigation policy for two jobs: propose movement without a goal image while exploring, and use a goal image when approaching a known place. A binary mask controls whether the policy can attend to the goal token. In the NoMaD paper, mask value 1 blocks that token for undirected exploration; value 0 exposes it for goal-directed navigation. This is a useful local behavior switch, but it does not decide by itself which distant region to search or when the mission is complete.
Treat the mask as a change in task, not a missing-image patch
The paper trains with the goal masked on roughly half of examples and visible on the other half. Its diffusion action head can represent more than one plausible short-horizon path at a junction. That matters in an unknown hallway: a controller should be able to consider turning left or right without averaging those possibilities into an obstacle. With a goal image present, the same policy can bias its sampled actions toward that place. The official project presents the pair of behaviors as exploration and visual-goal reaching, rather than as two separately deployed policies.
For an operator, the mask is only one state variable. The mission layer still needs to know whether a usable target image exists, whether that image describes the current environment, and whether the robot has reached it. An absent target is not equivalent to a stale target. If a robot is trying to find a charging dock but only has yesterday’s image of a moved dock, leaving goal mode active could produce confident motion toward the wrong visual cue. Masking the goal removes that cue; it does not make the robot understand where the dock moved.
| Mission condition | Policy input | What the local policy can contribute | Decision still outside the policy |
|---|---|---|---|
| No target view is known | Recent camera history, goal masked | Candidate short-horizon exploration actions | Which frontier is worth visiting next |
| A credible target view is selected | Camera history and goal image | Actions toward a visually specified subgoal | Whether this subgoal advances the mission |
| The target is reached or invalidated | Reassessed mask and target | New local action distribution | Stop, choose another target, or resume search |
| Obstacle or localization failure | Current observations may be unreliable | No guaranteed safe answer | Recovery, human intervention or a separate safety stop |
The original system pairs the learned policy with a topological memory and higher-level planner for long-horizon navigation and frontier-based exploration. The paper explicitly treats that graph as separate from the local policy. A topological node is a remembered view and an edge represents navigable connectivity. The planner can therefore choose the next subgoal; NoMaD can generate the short action sequence to approach it. This division is also useful when inspecting a global planner and local controller: a bad subgoal and a bad action sample call for different fixes.
Make the switch condition observable
Consider a warehouse robot looking for a door indicated by a stored photograph. It starts with no reliable match, so a search process picks an unexplored corridor and the policy explores without attending to the goal image. When the graph or a vision matcher identifies a credible door candidate, the mission layer supplies that candidate’s view and changes the mask to goal mode. If the candidate is later rejected, it withdraws the goal and returns to exploration. This sequence is an illustrative deployment design, not a rollout reported by the authors.
Log the cause of each switch. Record the goal-image identifier, the topological node or frontier selected, the mask value, the policy’s candidate actions, and the reason the mission layer accepted or rejected the target. Also record the observation interval and executed controller command. Otherwise, a replay may show a smooth right turn but leave no way to tell whether it was chosen by the graph, sampled by the diffusion policy, or imposed by a local controller. The public shared research repository provides navigation and exploration examples; it does not define these operational acceptance rules for every robot.
One useful evaluation is to replay matched starts in held-out environments under two conditions: goal absent and goal image available. Count exploration coverage or target discovery separately from goal-reaching success after a valid subgoal is chosen. Then stratify failures by false goal match, poor graph choice, local collision and missed stop. These are proposed checks, not newly measured NoMaD results. The paper reports its own real-world comparisons; those reported outcomes do not guarantee the same rates for another camera, corridor width or controller.
Keep action diversity within a bounded controller
Diffusion can express several plausible action sequences, but diversity is not a safety certificate. At a junction, two sampled paths might be physically possible while only one respects a blocked aisle or a human work zone. A deployment needs feasibility filters, a limit on command speed, an emergency stop path and a decision about how often new samples replace an executing sequence. The Nav2 RateController explanation is a related timing reference: ticking a navigation behavior is not necessarily the same as issuing a new motor command.
NoMaD is most useful when a robot truly alternates between finding a destination and approaching a visually specified one. If a valid goal is always known, the goal-free branch may add little operational value; if no trustworthy target view or topological search exists, the mask cannot create a mission plan. Evaluate the switch, the graph and the local action stream as separate interfaces, then decide whether the shared policy reduces complexity on the actual robot.
No model training, simulator run or robot experiment was performed for this article.