In openpi’s documented remote-inference example, the client prepares images and the server handles state normalization. Check that division before moving a policy off the robot. The remote-inference guide supplies an example contract; the served checkpoint still determines which inputs and transforms are appropriate.
A small local check illustrates why image preprocessing needs its own test: a float RGB image already at the requested size passed through openpi’s resize helper, while a wider float RGB image failed. Converting a known 0–1 image to uint8 before resizing resolved that failure. The executed checks and their exact limits are below.
Freeze the observation contract at the client-server boundary
The sample uses resize_with_pad and convert_to_uint8 for images, alongside observation/state and a task prompt. State is sent unnormalized. For a custom checkpoint, compare its policy configuration with the client before deciding which component owns each transform. Do not infer expected state order or image dimensions from a successful connection.
| Boundary field | Record in the interface manifest | Silent mismatch to catch |
|---|---|---|
| Camera input | Camera name, orientation, color order, resize-and-pad rule, output shape and uint8 range | A correctly shaped image with different crop, channel order or camera identity |
| Robot state | Joint and gripper order, physical units, frame, missing-value rule and normalization owner | Client and server both normalize, or neither does |
| Prompt and checkpoint | Prompt construction, policy configuration and checkpoint identity | A valid request sent to a policy trained for another instruction format |
| Action chunk | Horizon, action dimension, units, reference frame and how many steps the client executes before requesting again | A valid array applied to the wrong joints or at the wrong cadence |
The table is an integration checklist, not an automatic openpi check. Record a known observation, then compare its serialized client values with the policy input after server preprocessing. This can reveal shifted padding, exchanged cameras or degrees passed as radians. Keep normalization statistics with the checkpoint and note who applies them. A custom policy’s input contract takes precedence over the sample guide.
The related Octo observation and action binding guide covers mapping policy fields to a robot in general. Remote openpi inference adds a network boundary: an adapter can be internally consistent while disagreeing with the server’s preprocessing and checkpoint configuration.
Test pixel range before resizing an image
Two CPU-only checks were executed with AI assistance on 30 September 2026, using Windows 11, Python 3.12.14, NumPy 2.3.5 and Pillow 12.3.0. They ran the official openpi-client image helper at commit 215abfb on deterministic synthetic arrays. No camera, policy server, model or robot was used.
The first check varied the representation of four intended byte intensities: [0, 1, 128, 255]. Passing a uint8 array to convert_to_uint8 preserved them. A float32 array made by dividing those bytes by 255 also recovered the same four values. But casting the bytes to float32 without rescaling produced [0, 255, 128, 1] after conversion. The function multiplies floating values by 255; it does not detect whether an upstream camera adapter already uses byte-scale floats. In this run, the originally white value 255 became 1 without a range-validation error.
The second check used constant 0.5 RGB arrays and a target of 224 × 224 pixels:
| Input and operation | Observed result | Practical implication |
|---|---|---|
| Float32, 224 × 224 × 3, resize directly | Returned the original float32 array | A same-size test can bypass the Pillow resize path entirely. |
| Float32, 120 × 240 × 3, resize directly | Pillow raised TypeError for the three-channel float array |
Passing the same-size check did not establish support for a resized float RGB input. |
| The same wide array, convert to uint8 first, then resize | Uint8, 224 × 224 × 3; center RGB [127, 127, 127] |
Conversion first resolved this specific failure for the known 0–1 input. |
| Inspect that successful output | 56 zero-valued rows above and below a 112-row image | The wide image retained its aspect ratio through padding. |
To reproduce the checks without installing a robot stack, save the linked pinned file as image_tools.py in a scratch directory and use an environment with NumPy and Pillow. Import numpy as np and image_tools as it.
For the range check, create pixels = np.array([0, 1, 128, 255], dtype=np.uint8). Compare it.convert_to_uint8(pixels.astype(np.float32)) with it.convert_to_uint8(pixels.astype(np.float32) / 255).
For resizing, create wide = np.full((120, 240, 3), 0.5, dtype=np.float32). Compare it.resize_with_pad(wide, 224, 224) with it.resize_with_pad(it.convert_to_uint8(wide), 224, 224), then repeat with a 224 × 224 × 3 input. Record shape, dtype, center pixel and the nonzero row range; the successful wide result occupied rows 56–167 inclusive.
For your camera adapter, establish the documented input range before choosing a conversion. Keep existing uint8 pixels as bytes; for floats, distinguish normalized 0–1 values from byte-scale 0–255 values. Reject unexpected ranges or nonfinite values instead of using an unconditional conversion as validation. The division-by-255 alternative above was verified only for the four byte-scale sample values. These checks establish helper behavior in the stated environment, not image quality on real camera frames, checkpoint compatibility or robot performance.
Treat a returned chunk as a schedule, not a stream of fresh decisions
The guide returns an action chunk shaped (action_horizon, action_dim) and describes executing several steps open-loop between requests. For a moving object, the initial observation can become stale before the final action is applied. Choose the number of consumed steps from the task’s change rate and measured request-to-actuation delay, then record that choice. Do not silently execute a full chunk merely because the server returned it.
Measure separately when the camera frame was captured, when the request left the client, when the response arrived, and when each action reached the controller. The openpi WebSocket server implementation attaches server_timing.infer_ms to the response and may report a previous total server time. These server values are useful diagnostics, but they cannot measure client image preparation, network transit or actuator delay. A local end-to-end timestamp trace has to provide those missing parts.
For a hypothetical pick-and-place cell, try a reproducible object pose and vary network delay while keeping the checkpoint and image transform fixed. Compare requested actions, action age and task outcome. This is a proposed experiment, not an openpi result or a test performed here. If the robot must react to a person or moving obstacle, keep local controller and safety functions independent of remote inference; the guide does not certify real-time behavior or safe stopping.
Recheck the living contract when the server changes
The cited guide and server file are on openpi’s main branch, which can change. Pin the client package, policy config, checkpoint, server revision and observation schema together for an evaluation run. On every upgrade, replay the known observation and compare the returned chunk’s shape, units and selected values within a task-appropriate tolerance before using hardware. A health endpoint or successful WebSocket connection only establishes that the server can answer; it does not verify semantic compatibility. The local checks above exercised only image preprocessing; model inference, network delay and robot behavior remain untested.