Robot edge computing is a scheduling and data-movement problem before it is a processor-shopping problem. Cameras, decoding, perception, planning, communication and joint control run at different rates and have different deadlines. The useful platform is the one that completes the whole chain with bounded data age under the robot’s real power and thermal conditions.
A CPU is suited to irregular control flow, operating-system services, middleware, state machines and many planning tasks. A GPU is efficient for large parallel workloads and flexible neural models. An NPU can execute supported neural operations at attractive power efficiency. None replaces a dedicated MCU or real-time controller when a hard control or protective deadline requires isolation.
Read this with the robot inference-latency budget and ROS 2 real-time control guide. Measure on the deployable software image and sensor configuration; a vendor peak number or isolated model benchmark cannot establish robot responsiveness.
Start from physical deadlines and update rates
List every recurring function, its trigger, nominal rate, worst acceptable completion time and maximum input age. A 30 Hz camera may feed a 15 Hz policy while a state estimator runs at 200 Hz and a motor controller at 1 kHz. These are separate timing contracts, not one average frame rate.
Assign what must happen after a missed deadline. Perception may drop an old frame, planning may preserve the last valid trajectory briefly, and a controller may hold or decelerate. The fallback needs independent bounds so an overloaded accelerator cannot silently replay stale commands.
Trace one observation to one applied command
Timestamp physical exposure, sensor transfer, decode, preprocessing, inference enqueue and completion, postprocessing, planning, message delivery and command application. Carry one sequence identity across stages. Host clocks, device clocks and sensor hardware clocks need a documented synchronization method before timestamps can be compared.
Report median, P95, P99, maximum observed delay, sample count and input age at actuation. Throughput answers how much work finishes; latency answers how long one item waits. A highly pipelined system can report high FPS while the robot acts on an old scene.
| Stage | Typical owner | Primary measurement | Failure signal |
|---|---|---|---|
| Capture and transfer | Sensor and driver | Exposure-to-host time | Dropped or late frame |
| Preprocess | CPU or GPU | Queue plus execution | Growing buffer |
| Inference | GPU or NPU | Enqueue-to-output time | Fallback partition |
| Planning | CPU or GPU | State-to-command time | Missed horizon |
| Application | Controller | Command age at use | Stale rejection |
Use CPUs for irregular work and system coordination
CPUs handle drivers, ROS 2 executors, state machines, communications, exception paths and algorithms with branches or small changing workloads. They also launch accelerator work and manage memory. A low average utilization does not prove headroom when one busy core or lock determines a deadline.
Separate hard or safety-relevant loops from page faults, file logging, dynamic allocation and best-effort services. Apply affinity and scheduling only after tracing contention. A pinned thread can still wait for memory, a driver, an interrupt or a shared mutex.
Use GPUs for broad parallel workloads
GPUs are effective for image transforms, feature extraction, point processing and neural networks that expose substantial parallel work. Their mature libraries make them useful during rapid model iteration. Performance depends on tensor shapes, precision, kernel selection and how much work is grouped into each launch.
Small-batch robotics often values single-request latency more than maximum throughput. Include host enqueue, synchronization, output copies and competing kernels in the measurement. A faster kernel can make no difference if the pipeline waits longer for format conversion or memory ownership.
Use NPUs only after checking the compiled graph
An NPU can execute supported operators repeatedly at low power, but model conversion can change precision, shapes or graph structure. Unsupported operators may run on the CPU or split the graph across devices. The transition and copy cost can erase the expected gain.
Inspect compiler reports and compare output accuracy against the reference model. Test dynamic inputs, preprocessing and postprocessing as deployed. A device that advertises more TOPS can be slower for a particular model when operator coverage, memory or software maturity differs.
| Workload | First candidate | Why | What to verify |
|---|---|---|---|
| Drivers and state machines | CPU | Irregular control and I/O | Core and lock contention |
| Large vision tensors | GPU | Parallel flexible compute | Copies and launch overhead |
| Fixed supported network | NPU | Power-efficient inference | Operator fallback |
| Hard joint loop | MCU or RT controller | Deterministic period | Isolation and safe reaction |
| Mixed pipeline | Heterogeneous | Concurrent strengths | Synchronization cost |
Memory bandwidth can dominate accelerator math
Multiple cameras, depth maps, point clouds, maps and large weights can saturate memory before arithmetic units are busy. Shared-memory hardware does not guarantee zero copy: software may allocate new buffers, convert color formats or change tensor layouts at every boundary.
Record buffer ownership, allocation, read and write traffic and queue depth. Verify any zero-copy claim with addresses and traces. Test the complete sensor set together because simultaneous bursts and cache pressure reveal problems that one-camera benchmarks hide.
Power mode and cooling are part of the configuration
Battery robots operate inside a combined electrical and thermal budget. Compute peaks can overlap with actuator peaks, causing voltage sag or power limiting. A sealed enclosure, dust filter, quiet fan policy or hot warehouse can reduce sustained clocks long after a short benchmark finishes.
Fix power mode, clocks, fan policy and ambient conditions in every comparison. Run for long enough to reach thermal equilibrium and log rail power, temperature, frequency and latency. Select a sustained operating point with margin rather than the first-minute maximum.
Compare boards with a reproducible workload
TOPS values may use different precision, sparsity assumptions and operation definitions. Compare the same model, inputs, software versions, accuracy target, sensor load and power mode. Measure end-to-end latency, sustained throughput, memory headroom, energy per task and recovery from overload.
Board selection also includes camera and network interfaces, storage, boot time, security, long-term supply, update process and diagnostic tools. A slightly slower platform can reduce lifecycle cost if it is reproducible and maintainable across the fleet.
Profile contention and failure, not just nominal operation
Run mapping, logging, visualization, networking and update checks while perception and planning operate. Inject dropped frames, burst traffic, storage pressure, thermal throttling and a hung accelerator job. Observe whether queues remain bounded and whether stale outputs are rejected.
Correlate CPU scheduling, accelerator timelines, memory traffic, power and robot state on one clock. Average utilization can hide short blocking events. Store traces for successful and failed runs so performance regressions can be tied to a software or firmware change.
One SoC and distributed compute have different failure modes
A single SoC can reduce wiring and serialization but concentrates heat, power and software failure. Distributed controllers can isolate fast loops and provide fault containment, yet add network delay, clock synchronization, protocols and recovery states. Neither topology is automatically more deterministic.
Place large raw data near its sensor and send compact results when that preserves the task evidence. Define what happens when a link disappears or one node restarts. The interface must include timestamps, validity, sequence and health, not only a payload.
Keep the deployment profile versioned
Archive board revision, firmware, kernel, drivers, runtime, model artifact, compiler options, power profile and thermal setup with every benchmark. NVIDIA’s current Jetson software documentation exposes several release branches, so cite the exact release tested. Platform documentation describes capabilities, not the outcome of a specific robot workload.
Repeat the acceptance workload after model, middleware, driver or enclosure changes. Use the NVIDIA Nsight Systems documentation where applicable and equivalent vendor tools for other accelerators. Accept an optimization only when system timing and task performance improve together.
- Define rates, deadlines and command-age limits.
- Trace capture through command application.
- Inspect compiled graphs and memory copies.
- Stress power, heat and concurrent workloads.
- Version the exact hardware and software profile.
Frequently asked questions
Can a robot run without a CPU?
Most practical robots still need a CPU for drivers, communication, state machines and exception handling. GPU and NPU devices accelerate selected workloads.
Does higher TOPS mean lower robot latency?
No. Precision, operator support, memory movement, batching, power mode, cooling and queues all affect end-to-end timing.
Should GPU and NPU run at the same time?
They can, especially for independent workloads, but graph splitting and buffer transfers must be measured because synchronization may cost more than it saves.
Is unified memory automatically zero copy?
No. APIs and ownership changes can still allocate or copy buffers. Confirm the actual path with profiling and address-level evidence.
How long should a board benchmark run?
Long enough to reach the worst expected temperature and expose throttling, memory growth and queue instability while every production sensor and service is active.
Onboard Compute Evidence Boundary
Compute performance depends on the exact board, software release, model, sensor load, power mode and thermal environment. Validate the complete robot pipeline before making a deployment claim.