Demo-SCORE uses the outcomes of a robot policy’s own rollouts to select demonstrations for subsequent imitation learning. It first trains on the mixed demonstration set, learns a success-versus-failure classifier from policy experience, and uses that classifier to filter the original data. The selection asks which demonstrated strategies the learner can reproduce reliably, rather than which recording merely looks smooth.
This is useful when a dataset contains several ways to complete a task. A human can recover from an awkward grasp that a learned policy repeatedly mishandles. The entire demonstration can end successfully while teaching a difficult intermediate strategy.
Policy experience supplies the comparison
The RSS paper summary frames the problem as heterogeneous demonstration quality and strategy reliability. The paper’s method section starts with an initial policy trained on the full dataset, then gathers successful and failed rollouts at different checkpoints. A classifier learns from those outcomes before being applied to demonstrations.
Keep the two data roles distinct. Demonstrations teach the initial policy. Rollouts reveal how that policy behaves after learning from them. A classifier trained on rollout outcomes is a selection instrument; its output is not a new physical success measurement for every demonstration it scores.
Consider a hypothetical cup-placement task with rim and side grasps. If the policy often gets stuck while releasing a rim grasp, failed rollouts may reveal that strategy’s difficulty. Filtering can favor a more reproducible approach. But if the side-grasp examples cover only one cup size, removing the others can also narrow the training distribution. Inspect retained coverage alongside the classifier score.
Cross-validation tests more than a familiar rollout set
The method uses rollouts from multiple policy checkpoints to reduce overfitting to one rollout distribution. Classifier selection uses validation experience from a different checkpoint. The paper also describes regularization and the classifier’s proprioceptive inputs. These details matter because the distribution being classified changes as the initial policy improves.
A random frame split from the same rollout would answer a weaker question: whether a classifier recognizes highly related samples. Preserve rollout and checkpoint identities when examining the split. The exact recipe should follow the implementation being reproduced; the principle is to evaluate selection on experience not already used to fit that classifier.
This differs from the SARM progress-weighting workflow. SARM’s guide examines sampled progress predictions and weighting across an action interval. Demo-SCORE’s reader question is how outcome-labeled policy experience can select among demonstrations after capture.
Compare the selected data with what was removed
An audit needs more than the final retained count:
| Retain in the selection record | Why it matters |
|---|---|
| Original dataset and policy revisions | Identifies which learner produced the evidence |
| Rollout outcome and checkpoint | Keeps the classifier labels traceable |
| Classifier split and chosen checkpoint | Separates fitting from validation |
| Source episode or chunk identity | Connects every score to actual demonstration content |
| Coverage before and after filtering | Reveals lost objects, strategies or initial conditions |
The paper permits filtering at episode or chunk level. Choose and record that unit before interpreting the result. Removing one segment and removing its entire demonstration have different consequences for transitions and coverage.
Judge the new policy on fresh executions
Train the filtered-data comparison under a stated budget and evaluate it on fresh task executions. Include the original-data baseline and preserve failures from both. A useful result explains which failure mode became less frequent and which conditions remained difficult; it does not merely report that the classifier preferred the selected data.
The authors’ project illustrates this with real-world tasks and strategy differences. Those examples motivate the method, while a changed robot, object set or task requires its own rollout evidence. No classifier, dataset filtering or robot experiment was executed for this article; the sources were checked October 16, 2026.