A vision-language model, or VLM, interprets images and language and returns a digital answer such as text, labels, coordinates or a structured description. A vision-language-action model, or VLA, must connect similar inputs to an action that a robot can execute. That extra letter changes the engineering problem from describing the world to changing it.
A VLM may correctly identify a cup and explain that it sits beside a plate. A VLA must select the intended cup, choose a reachable grasp, produce a motion, monitor contact and respond if the object slips. The result depends on camera calibration, robot geometry, controller timing and safety behavior as well as semantic understanding.
The boundary is architectural rather than absolute. A robot system can use a VLM for perception and a separate planner for action, while another model produces actions end to end. Compare systems by their actual inputs, outputs and feedback loop instead of assuming that every product using the VLA label has the same capabilities.
A VLM produces information about a scene
A VLM combines visual representations with language so it can answer questions, caption an image, identify objects or reason about spatial relationships. In robotics, those outputs can support perception and planning. A point, bounding region or task description can tell another component what object or location matters.
The output remains informational unless it is connected to a robot control process. Correctly naming an object does not specify how to approach it, which gripper pose is stable, how fast to move or what to do after unexpected contact.
A VLA produces an action for an embodied system
A VLA model maps visual observations and language instructions to robot actions. Depending on the system, an action may be a target pose, waypoint, joint command, gripper command or chunk of motion. The action interface determines what the model must learn and what remains for a conventional controller.
Because the output moves hardware, evaluation must include physical consequences. Latency can destabilize a correction, a small coordinate error can miss an object, and an ambiguous instruction can select the wrong target. The system needs feedback and an independent way to slow, stop or request help.
The output changes the complete stack
The same image and instruction can enter either model class, but the expected output changes the data, architecture and test. A VLM can be evaluated against an answer. A VLA must be evaluated through the robot, because identical command tokens can produce different motion on different bodies or under different calibration.
The card summarizes the practical transition from scene interpretation to closed-loop action. It also shows why adding an action head to a VLM is not enough by itself: robot state, timing, embodiment and recovery must be represented somewhere in the system.
Action representation determines what the model controls
An action can be expressed in joint angles, joint velocities, end-effector poses, gripper states or learned tokens. Absolute targets provide a destination, while delta actions describe a change from the current state. Short action chunks reduce repeated inference but can make mid-course correction slower.
A comparison is incomplete unless the action space and control rate are stated. A model producing 6D waypoints at a low rate relies on a downstream controller differently from a model producing high-rate joint commands. The two systems may both be called VLA while carrying very different risk and hardware dependence.
| Model output | What it represents | What remains downstream |
|---|---|---|
| Text or labels | Objects, relations or task description | Planning and control |
| 2D or 3D points | Grounded target locations | Pose estimation and motion generation |
| End-effector pose | Desired tool position and orientation | Inverse kinematics and joint control |
| Joint action chunk | Sequence of robot-specific commands | Tracking, safety filtering and feedback |
Feedback turns an action into a policy
A useful robot policy observes the result after acting. If the gripper closes without securing the object, the next observation should trigger a correction, retry or stop. Open-loop playback can look successful in a fixed scene but fails when object pose or contact changes.
Feedback also exposes uncertainty. The system can compare the expected next state with the measured one, monitor confidence and escalate when the difference becomes too large. This recovery behavior is a stronger deployment signal than a single successful trajectory.
Spatial grounding must become executable geometry
A language instruction such as ‘put the marker in the tray’ is underspecified for a robot. The model must identify the marker, choose a grasp point, account for obstacles and produce a route that the arm can reach. A spatially correct answer may still be mechanically impossible.
The Google Gemini Robotics documentation shows spatial points and trajectory-style outputs grounded in robot scenes. These examples illustrate the bridge between VLM-style reasoning and action, while the real controller and robot still determine execution quality.
VLA training needs synchronized robot experience
VLM pretraining can use large collections of image-text pairs. VLA training additionally needs observations aligned with executable actions and outcomes. The robot action data guide explains why video alone is insufficient: state, commands, timing, calibration and success labels must share a timeline.
Google DeepMind’s RT-2 work describes translating vision and language into robot actions, while Open X-Embodiment and RT-X address learning across diverse robot datasets. Cross-robot data increases scale but also makes action normalization and embodiment metadata critical.
Evaluation must separate understanding from execution
A model can understand the instruction and still fail because of perception noise, motion constraints or contact. Report semantic target accuracy separately from grasp success, task completion and intervention. This reveals whether a change improved language understanding or only the downstream controller.
Evaluation should also identify unseen conditions. New instructions, objects, backgrounds and robot bodies test different forms of generalization. A single average hides which layer failed, so use the evidence table to preserve the distinction.
| Evaluation layer | Example metric | Failure it isolates |
|---|---|---|
| Language grounding | Correct object or region | Instruction misunderstood |
| Action prediction | Trajectory or command error | Wrong motion representation |
| Physical execution | Grasp and task success | Control, calibration or contact failure |
| Operational behavior | Intervention and recovery rate | Policy cannot handle exceptions |
A VLA is one component of a robot system
A broad robot foundation model may support VLA behavior across tasks, but the deployed system still needs state estimation, controllers, safety functions, logging and operations. Some of those components can be learned and others can remain conventional.
This system view prevents two opposite mistakes: dismissing a useful VLM because it is not end to end, and assuming a VLA label means the complete robot is autonomous. Architecture should be judged by task performance and failure containment, not by how few boxes appear in a diagram.
How to compare two robot models
Record each model’s visual inputs, language inputs, robot-state inputs, action representation, control rate, training data and supported bodies. Then record the physical evaluation: number of trials, unseen conditions, interventions, recovery and safety limits. This creates a comparable specification even when vendors use different labels.
Finally ask what happens after an error. A model that detects uncertainty and hands control to a safe recovery process may be more useful than one with higher benchmark accuracy but no escalation. The best comparison follows the full loop from observation through action to the measured result.
- Identify the exact output: text, point, pose, joint action or action chunk.
- List the robot state and calibration supplied to the model.
- Separate semantic accuracy from physical task success.
- Count intervention, recovery and safety events.
Frequently asked questions
Does a VLM become a VLA when it is installed on a robot?
No. A VLM can provide perception or reasoning inside a robot, but a VLA specifically connects observations and language to an action representation used by the robot.
Is a VLA always end to end?
No. Some models predict low-level actions, while others produce waypoints or plans that downstream controllers execute. The action interface must be stated.
Does a VLA make a robot fully autonomous?
No. Autonomy also depends on sensing, control, recovery, safety, task coverage and when people intervene. The model name alone does not answer those questions.
Why does a VLA need more than robot video?
Video shows appearance and motion but may omit joint state, commands, timing, calibration and intervention. Training an executable policy needs those signals aligned.
How should VLA and VLM performance be compared?
Separate language and visual grounding from action prediction, physical task success, intervention and recovery. They do not share one universal metric.
Model Names Do Not Prove Autonomy
VLA, VLM and robot foundation model are broad labels. Verify the published architecture, action interface, evaluation protocol and operating limits before comparing capability claims.