SayCan separates two questions that a language-only robot plan can blur: would a skill help with the instruction, and can this robot perform that skill from its current state? A promising sentence answers only the first. SayCan ranks available skills using both language relevance and an estimate of physical feasibility, then executes a selected skill before choosing again.
This makes it a useful concept for diagnosing a robot that proposes sensible steps but cannot carry them out. The first thing to inspect is the skill catalog and its feasibility estimates, not simply the wording of the user’s request.
Two scores meet at the skill boundary
The official SayCan project describes a selection loop over available robot skills. The language model scores how useful each skill description is for the instruction and preceding steps. A value function estimates the skill’s feasibility in the present environment. Their combined score guides the next selection; the chosen skill is executed and added to the sequence before the next decision.
The following numbers are an invented arithmetic example. They are not outputs from SayCan, a reproduced experiment or calibrated probabilities for a real robot. Imagine a request to bring a bottle, with the robot starting too far from it.
| Candidate skill | Illustrative language score | Illustrative feasibility score | Product and interpretation |
|---|---|---|---|
| Pick up the bottle | 0.90 | 0.10 | 0.09: relevant, but currently difficult |
| Move to the bottle | 0.60 | 0.80 | 0.48: less directly completes the request, but is more feasible now |
| Move away from the bottle | 0.10 | 0.90 | 0.09: easy does not make it useful |
Multiplication is easy to reproduce; trustworthy scores are the difficult part. If an affordance model overestimates a blocked grasp, combining it with an excellent language score can still select that grasp. If the useful navigation skill is missing from the catalog, the ranking cannot select it. These are different defects and call for different changes.
Do not interpret the largest product as a guarantee of task completion. A score can rank alternatives while remaining poorly calibrated, and success on the next skill does not establish success on a multi-step instruction.
A skill needs an implementation and a feasibility model
Google Research’s method explanation defines each skill through a short language description, an executable policy and an affordance function. The research system learns policies and value functions using robot data and simulation. Its public release includes a simulation setup; that release is not evidence that a reader’s physical robot already has equivalent skills.
For an integration review, write down the boundary of each candidate skill. A description such as “pick up the bottle” is insufficient unless the action interface identifies the target, the policy accepts the required observations, and the completion condition has a usable meaning. If the target is outside the robot’s reach, a well-written description does not repair that mismatch.
Consider two apparently similar catalog entries that use different cameras or target-coordinate conventions. Comparing their numerical feasibility scores before checking those interfaces risks treating an input error as a planning preference. Start with the observation and action contracts, then assess whether the scores are meaningful under the same operating conditions.
This is also why adding a more capable language model is not a substitute for adding a missing robot behavior. A language model may describe a helpful action in detail while the available policy library cannot execute it. The implementation work belongs at the skill boundary.
Pre-action feasibility does not verify the outcome
The SayCan project explicitly notes a feedback limitation: environmental information enters through value functions at the current decision step, so failures or changes may not provide all the feedback needed to alter the plan. A reader should not silently attribute a complete post-action verification and recovery system to the original method.
Use separate records for the selected skill, its two input scores, the observed outcome and the next decision. In a hypothetical grasp failure, these records help distinguish three possibilities: the feasibility estimate was wrong, the skill policy failed despite a reasonable estimate, or the next planning step did not account for the failure. A single “task failed” label conceals that distinction.
The EmbodiedSkills planner and verifier guide examines the adjacent problem of checking execution outcomes. That is a useful next topic after SayCan’s selection rule, because deciding what can plausibly work and observing what actually worked require different evidence.
When assessing a SayCan-style design, begin with one concrete failure and inspect the available skills, their inputs and both scores at that moment. Then check the outcome independently. This article explains the published architecture and proposes diagnostic steps; it does not report a SayCan deployment, a robot trial or a comparison of language models on a new benchmark.