PerAct uses a fixed voxel grid, while Act3D concentrates spatial search around selected regions. Compare their workspace coverage and precision only after checking the coordinates entering the representation. In our CPU test of PerAct’s official coordinate helper, a point outside the upper workspace boundary returned the same voxel index as a point on that boundary. That result is an input-validation warning, not a comparison of the two models’ task success.
The PerAct project describes a language-conditioned policy over RGB-D observations represented as voxels. Its outputs include translation, rotation, gripper state and collision-related action information. The Act3D project lifts visual features into 3D using depth and predicts a keypose through adaptive spatial attention. Both require an execution layer that can turn the predicted action into robot motion.
A voxel size is a task assumption
A fixed grid divides a bounded workspace into cells. Narrowing the workspace while retaining the same number of cells makes each cell smaller. Expanding it makes each cell larger. That means the workspace bounds are part of the action representation, not just a convenient visualization setting.
As a numerical illustration, 100 cells across a one-meter axis give a cell width of 10 millimeters. The same 100 cells across half a meter give 5 millimeters. These are grid calculations, not measured PerAct positioning errors. Calibration, observation quality, rotation representation and execution can introduce additional error even when the target cell is correct.
PerAct’s project explains a 100 × 100 × 100 voxel input, grouped into 5 × 5 × 5 patches, yielding 8,000 patch embeddings processed with a Perceiver architecture. That mechanism makes the large spatial input manageable; it does not remove the underlying resolution and workspace tradeoff. A small feature near a workspace boundary may be poorly represented for reasons a headline task score cannot reveal.
A valid-looking voxel index can hide an invalid coordinate
Before changing the model’s spatial resolution, check what reaches the voxel conversion. In an AI-assisted CPU check, we ran PerAct’s unchanged point_to_voxel_index helper on synthetic coordinates on October 4, 2026, using CPU Python 3.12.14 and NumPy 2.3.5. The tested function at revision 02fb876 applies a floor operation and an upper cap; it does not reject points outside the workspace. The repository’s demonstration-action conversion calls this helper. We executed the helper alone, without loading that training pipeline, either model, or a simulator.
Our test workspace was a synthetic cube from 0 to 1 meter on every axis, with 100 cells per axis. We varied x while keeping y at 0.251 and z at 0.371 meters. These bounds are test inputs, not PerAct’s default workspace.
| Input x, meters | Returned x index | Our input guard |
|---|---|---|
| −0.001 | −1 | Reject |
| 0 | 0 | Accept |
| 0.500 | 49 | Accept |
| 0.501 | 50 | Accept |
| 1 | 99 | Accept |
| 1.001 | 99 | Reject |
The upper-face point and the point 1 millimeter beyond it returned the same index. Checking only whether the returned index lies between 0 and 99 would miss that overshoot. Below the lower face, the helper returned a negative index instead. The exact internal boundary at 0.500 mapped to 49 in this float64 test because of the implementation’s small epsilon terms before flooring; assuming ideal decimal bin boundaries would give the wrong expected result for this version.
We also passed the intended point (0.501, 0.251, 0.371) in millimeters as (501, 251, 371), leaving the meter-based bounds unchanged. The result saturated at (99, 99, 99). Explicitly dividing those known millimeter inputs by 1,000 restored (50, 25, 37). A larger voxel grid would not correct the unit mismatch.
Our wrapper checks shape, finite values, and the original point against the configured bounds before calling the helper. It rejected overshoots at all six faces, NaN, infinity, and a two-element vector. It preserved the official outputs for 101 valid x positions spanning both endpoints. This test used a closed workspace interval, accepting the exact upper bound; define that boundary convention explicitly in your own integration.
To reproduce the table without installing PerAct, save the pinned utils.py, inspect lines 84–96, and run the snippet below beside that file in a Python environment with NumPy. It verifies the downloaded file and executes only the selected function, leaving the module’s robotics imports and other code unexecuted.
import ast
import hashlib
from pathlib import Path
import numpy as np
data = Path("utils.py").read_bytes()
expected = ("efab9ff8c2757726cd99bd6645446336e"
"b19f4a019563778fea2125d0372114c")
assert hashlib.sha256(data).hexdigest() == expected
tree = ast.parse(data.decode("utf-8"))
fn = next(n for n in tree.body if isinstance(n, ast.FunctionDef)
and n.name == "point_to_voxel_index")
scope = {"np": np}
exec(compile(ast.Module(body=[fn], type_ignores=[]), "helper", "exec"), scope)
voxel_index = scope["point_to_voxel_index"]
bounds = np.array([0., 0., 0., 1., 1., 1.])
def checked(point):
p = np.asarray(point, dtype=np.float64)
if p.shape != (3,) or not np.isfinite(p).all():
raise ValueError("expected three finite coordinates")
if np.any(p < bounds[:3]) or np.any(p > bounds[3:]):
raise ValueError("outside workspace")
return voxel_index(p, 100, bounds)
for x in [-.001, 0., .5, .501, 1., 1.001]:
p = np.array([x, .251, .371])
raw = voxel_index(p, 100, bounds).tolist()
try:
checked(p)
decision = "accepted"
except ValueError:
decision = "rejected"
print(x, raw, decision)
mm = np.array([501., 251., 371.])
print("mm unconverted", voxel_index(mm, 100, bounds).tolist())
print("mm converted", checked(mm / 1000).tolist())
Rejecting an out-of-bounds point is a diagnostic, not a repaired pose. Inspect the units, frame transform, and intended workspace before retrying. A wrong-frame point that happens to fall inside the box still passes this check. These results establish the behavior of one coordinate helper and our wrapper; they do not measure model accuracy, robot safety, or an advantage over Act3D. Keep this input check separate from the choice between a fixed grid and adaptive spatial search.
Adaptive search moves the precision decision
Act3D samples locations across the workspace, then refines the search around promising regions. Its relative spatial attention operates on 3D features rather than requiring the entire workspace to be represented at the finest resolution from the start. The later search can focus on a small interaction area while an earlier stage supplies wider coverage.
The engineering question then changes: did the coarse stage retain the right region? If it focuses on the wrong handle, adding precision around that handle will not repair the selection. A fixed-grid system and a coarse-to-fine system can therefore miss the same target for different reasons. One may lack local resolution; the other may allocate its fine search to an incorrect location.
| Design choice | PerAct-style fixed voxel representation | Act3D-style adaptive spatial search |
|---|---|---|
| Spatial coverage | Set by the workspace bounds and grid | Starts broadly, then refines selected regions |
| Precision decision | Cell size couples workspace size and resolution | Fine precision depends on where refinement occurs |
| Diagnostic to retain | Target cell and physical distance to the desired pose | Selected region at each refinement stage |
| Shared integration need | Calibrated observations and a defined action-to-motion interface | Calibrated observations and a defined action-to-motion interface |
The table compares design mechanisms. It is not an independently measured speed or success ranking.
Compare complete actions, not just translation points
A good translation prediction can still produce a failed manipulation if rotation, gripper state or collision handling is wrong. Inspect how each implementation encodes and decodes those fields, how demonstrations define action keyframes, and what the planner receives. Do not silently change the execution primitive while evaluating the representation.
For a hypothetical button task, a position close to the button may be adequate only when the tool approaches along the correct normal. For a handle, the same position can require a different orientation and subsequent motion. Keep position error, orientation error and final task outcome separate so that the action component causing failure remains visible.
The Octo observation/action binding guide covers a related integration principle: observation and action conventions must match the model and robot. Here the additional issue is how a policy chooses a spatial target before that binding is applied.
Build an equal-condition comparison
Use the same tasks, demonstrations, camera views, depth processing, workspace bounds and success rule when comparing implementations. Record hardware, memory use and the full inference path. If a backbone or camera setup differs, name that difference rather than attributing every outcome to voxelization versus adaptive search.
Separate the cost of visual feature extraction, spatial prediction and motion planning. Report whether a failure came from missing the correct region, inaccurate local pose, or execution. A model with cheaper spatial prediction can still have an expensive visual backbone; a precise target can still be unusable if the downstream planner rejects it.
The PerAct paper and Act3D paper provide the respective experiments. Act3D’s project presents separate single-task and multi-task evaluations, so numbers from those settings should not be combined into a single ranking. Neither paper establishes the best design for an untested workspace or robot.
Start with the failure you need to fix. If a fixed grid cannot resolve a small target within the required workspace, inspect its spatial budget. If adaptive search repeatedly selects the wrong region, inspect coarse-stage grounding before increasing fine-stage precision. No model, benchmark or robot execution was run for this comparison.