A robot can retain the right demonstration and still receive too little of it when choosing an action. If a policy compresses a growing history into a fixed visual input, unrelated sessions compete with the useful session for that input. Storage capacity and access to relevant experience become two different parts of the memory problem.
RoboMME-Interference makes this distinction concrete. Its detailed benchmark protocol places a relevant demonstration first, appends unrelated robot sessions, and then asks the policy to perform a query task. The useful experience remains in the history while its distance from the query increases. This is a practical way to study whether a memory interface continues to deliver the information the policy needs.
Follow the demonstration through the history buffer
In the official project’s Benchmark section, the history conditions have the following order. The query is the execution episode, separate from the prior demonstration.
no-history: query
k0: relevant demonstration → query
k1: relevant demonstration → unrelated → query
k3: relevant demonstration → 3 unrelated sessions → query
k7: relevant demonstration → 7 unrelated sessions → query
Here, k counts intervening sessions. It does not specify a number of seconds, frames, or tokens. The relevant demonstration is present in every k condition; no-history removes it altogether.
The distractors come from other task families. That choice makes the protocol about accessing useful experience amid unrelated experience. It avoids introducing a second demonstration of the same task with a deliberately conflicting answer. For example, a history can contain the route the robot should repeat, followed by sessions about other manipulation tasks. The query still needs the earlier route.
This also explains the task selection. The benchmark uses nine RoboMME families whose necessary information can be supplied in a self-contained prior demonstration video. That demonstration can be lifted into a history buffer and separated from execution. Tasks that reveal their cue only inside the current episode require a different intervention.
What the remembered session contributes
The prior video carries different kinds of information across these families. VideoRepick supplies the identity of an object to pick again. VideoUnmask supplies the location of an occluded cube. PatternLock supplies a continuous pattern to trace over a grid. RouteStick supplies a path, including turn directions, through obstacles.
These examples help interpret a success score. Selecting the right object, recalling a hidden location, and reproducing a path place different demands on the representation of history. A single written label may convey an object’s identity; a path can require ordered spatial information. This is a reason to inspect the task families as well as the overall aggregate.
The memory systems also expose history to the policy differently. The project describes perceptual representations that sample frames or drop visual tokens, recurrent representations that compress information into hidden state, and systems that read the buffer with a vision-language model and produce subgoals. Integration mechanisms include concatenating memory tokens with the policy input, conditioning through adaptive LayerNorm, and using a dedicated expert.
Those choices locate the information bottleneck. A larger stored buffer helps only if the relevant content survives the representation and reaches the action policy in a useful form. In a sampled-frame interface, one concrete place to look is the selection of frames that the policy actually receives.
A fixed frame budget can spread across irrelevant sessions
The project’s Adding Retrieval section offers a specific account of the observed decline: sampling the whole history down to a fixed set of frames leaves fewer frames from the useful demonstration as unrelated sessions accumulate. The authors describe this dilution as a likely cause, rather than an isolated measurement of every contributing mechanism.
A hypothetical allocation illustrates the idea. Suppose an interface accepts 32 history frames, all sessions have the same length, and the sampler allocates the budget equally across sessions. With only the relevant session, all 32 frames can describe it. With one relevant session and seven unrelated sessions, the relevant session receives four frames. The remaining 28 describe other experiences.
That arithmetic is an illustration, not RoboMME’s stated frame budget or an implementation trace. It shows why total stored experience and useful policy input can move in opposite directions. Even if the required demonstration remains intact on disk, its decisive moments may occupy only a small part of the policy’s visual context.
The effect can depend on the task. A small sample might preserve a distinctive object while missing an intermediate turn in a route. This suggests examining which relevant moments survive sampling, alongside the success rate, when diagnosing a particular policy.
Retrieval changes which history enters the policy
The reported Retrieval-FrameSamp-Modul extension adds selection before the existing FrameSamp-Modul policy. According to the project, it chunks history using visual embeddings, selects a chunk matching the current view, and passes only that chunk onward. The frame-sampling budget then concentrates on the selected demonstration instead of the entire mixed history.
This changes the input selection stage while retaining the policy named in the comparison. It gives the experiment a focused question: can the policy use the useful experience when a retrieval stage brings that experience back into its visual input?
The reported retrieval setting makes the matching problem relatively favorable. The relevant demonstration is always present and belongs to the query’s task family, while distractors come from other families. The project explicitly notes that retrieval is easy under these conditions. A collection containing several visually similar demonstrations from the same family, or no suitable demonstration at all, would require additional choices about selection and rejection.
For a practical illustration, consider several routes through the same obstacle layout. Selecting a video of that layout identifies the family of experience, but the query may still require a particular route. A memory system for that hypothetical collection would need a cue that distinguishes the requested route among similar candidates. That is a different retrieval problem from separating a route demonstration from unrelated manipulation sessions.
Read the controls before comparing systems
The main results grid and the retrieval extension answer different comparison questions. The main grid includes each evaluated memory variant under the history conditions, plus a separate no-memory π0.5 backbone baseline. The retrieval section studies an extension of FrameSamp-Modul; it does not add retrieval to every system in that grid.
Within a memory variant, the comparison from its own no-history condition to k0 shows how much the nearby demonstration helps that variant. The comparison from k0 to k7 then shows how that benefit changes as unrelated sessions intervene. The separate backbone baseline supplies another reference, but it is not interchangeable with each memory variant’s own no-history result.
For the retrieval extension, the useful pairing is the same FrameSamp-Modul policy with and without the added selector, under matching history conditions. Keeping those pairings visible makes it easier to distinguish a stronger action policy from better delivery of an existing demonstration. It also preserves the special retrieval conditions instead of blending them into a general ranking of memory architectures.
This article interprets the project’s published protocol and explanations; it does not report a local reproduction or a robot trial.
Once a robot has used a past demonstration, its execution system still needs to preserve what it actually completed. The guide to long-horizon task memory and recovery explains that operational layer. RoboMME-Interference addresses the earlier information path: selecting a relevant prior experience from accumulated history and delivering enough of it to the policy to guide the current task.