Moving a wrist-mounted camera in several straight directions can produce a dataset that looks varied while leaving its mounting offset undetermined. The problem is not necessarily a large equation error. Different offsets can satisfy exactly the same relative-motion equations, even when the camera-to-gripper rotation is held fixed.
A low residual alone is therefore not a sufficient check when planning a calibration capture session. Check whether changing the candidate offset changes the equations for the motion types you intend to collect.
Consider three synthetic camera offsets, expressed in gripper coordinates: (80,−30,120), (−40,60,200) and (80,−30,200) mm. Give all three the same camera-to-gripper rotation: X15°, then Y−20°, then Z30°, applying the rightmost rotation first in the matrix product Rz(30°) Ry(−20°) Rx(15°).
For the three pure relative translations in this example, all three offsets satisfy the same equations. They represent different absolute camera locations. What coincides is the result of the two transform chains within each candidate, not the camera locations across candidates.
The offset cancels out
OpenCV’s hand-eye documentation defines the camera-to-gripper transform and the relative-motion relation AX=XB. In the eye-in-hand convention used here, A is a gripper relative transform, B is the corresponding camera relative transform, and X is the unknown camera-to-gripper transform. A and B must have consistent directions; these are relative transforms, not interchangeable raw pose arrays.
Write their rotations as RA, RB and RX, and their translations as tA, tB and tX. The translation part of the equation is:
RA·tX + tA = RX·tB + tX
For pure translations, RA and RB are identity matrices. The two copies of tX cancel, leaving tA=RX·tB. No choice of translation direction restores tX to that equation. More exact samples of only this motion type cannot select one offset from the alternatives.
The inspected OpenCV translation-equation construction uses the corresponding coefficient RA−I. That source inspection supports the equation; it is not a report of running its solver. The examined code is a pinned 4.x branch revision.
This does not make every rotation unknown
The remaining equation still relates the translation vectors through RX. With exact, correctly paired metric data, two noncollinear translation directions can constrain a proper rotation: their mapped directions and cross product fix a right-handed basis. One direction alone leaves freedom to rotate about that direction.
The standalone controls demonstrate both cases. Two independent translations recover the chosen common rotation by basis construction, while an alternative rotation preserves the first translation direction but fails the second. Neither result reveals the camera offset, and neither establishes that a particular OpenCV algorithm will calibrate from translation-only input.
One rotation axis leaves another ambiguity
Add a prescribed 40° gripper-frame Z rotation while retaining the same reference-generated A and B for every candidate. The second offset now produces a 102.606 mm translation-equation discrepancy. But the third offset—an 80 mm shift along Z—still satisfies that motion. Rotating about Z does not change a displacement along Z.
A second control rotates −35° about gripper X. The Z-shift candidate then produces a 48.113 mm discrepancy. These are differences between known synthetic transform chains, not measured calibration error or expected hardware accuracy.
For the fixed common RX in this construction, the stacked translation coefficients have rank 0 under pure translations, rank 2 with the Z rotation and rank 3 after the X rotation is added. This explains the missing directions; it is not a robust calibration solution. OpenCV’s documentation discusses nonparallel rotation axes for the full hand-eye transformation and recommends more than the minimum poses. This small algebra example does not replace those data-collection or method requirements.
The following standard-library code reproduces the translation-equation gaps. The common rotation is supplied, and rotation-control camera translations are generated from the first offset. No fitting or calibration routine is called.
from math import sin, cos, radians, dist
def turn(v, axis, degrees):
i, j = {'x': (1, 2), 'y': (2, 0), 'z': (0, 1)}[axis]
c, s = cos(radians(degrees)), sin(radians(degrees))
out = list(v)
out[i], out[j] = c*v[i]-s*v[j], s*v[i]+c*v[j]
return out
def R(v):
return turn(turn(turn(v, 'x', 15), 'y', -20), 'z', 30)
columns = [R(v) for v in [(1, 0, 0), (0, 1, 0), (0, 0, 1)]]
def RT(v):
return [sum(a*b for a, b in zip(column, v)) for column in columns]
offsets = [(0.08, -0.03, 0.12), (-0.04, 0.06, 0.20), (0.08, -0.03, 0.20)]
motions = [('pure x', None, 0, (0.1, 0, 0)),
('pure y', None, 0, (0, 0.08, 0)),
('oblique', None, 0, (0.03, -0.02, 0.05)),
('z rotation', 'z', 40, None), ('x rotation', 'x', -35, None)]
for name, axis, angle, camera_translation in motions:
RA = (lambda v: list(v)) if axis is None else (lambda v: turn(v, axis, angle))
if axis is None:
robot_translation = R(camera_translation)
else:
robot_translation = (0.04, 0.01, -0.02)
t0 = offsets[0]
camera_translation = RT([a+b-c for a, b, c in
zip(RA(t0), robot_translation, t0)])
gaps = []
for offset in offsets:
left = [a+b for a, b in zip(RA(offset), robot_translation)]
right = [a+b for a, b in zip(R(camera_translation), offset)]
gaps.append(dist(left, right)*1000)
print(f'{name}: translation-equation gaps_mm={[round(g, 3) for g in gaps]}')
if axis is None:
assert max(gaps) < 1e-9
The AI-operated Blender 5.2.2 LTS experiment on October 5, 2026 placed 60 × 40 × 30 mm camera proxies and finite landmark triads under both transform chains for all fifteen motion/candidate pairs. An independent audit recovered rotation and translation from their evaluated vertices, checked the full proxy geometry and both composition orders, and reproduced the rank pattern above. The nine pure-translation pairs matched; the Z-shift candidate had a native X-rotation chain gap of 48.113 mm. These are prescribed relative matrices and proxy placements, not robot logs or camera observations.
The documentation URL resolved to OpenCV 4.13.0; the inspected 4.x implementation revision is separately pinned above. No OpenCV solver, PnP method or ROS node was run. Numerical vertex comparisons used 3 micrometers; aggregate transform comparisons used 0.00005 and the measured coefficient-rank check used a 0.0001 threshold. Those are audit guards for this floating-point construction, not calibration accuracy. Lens calibration, target detection, robot accuracy, noise sensitivity and a physical mount were not assessed.
For capture planning, test the offset-sensitive directions before increasing the sample count. Repeating translations cannot reveal a missing offset direction, and one rotation axis still leaves its axial shift unconstrained. The full eye-in-hand and eye-to-hand guide covers frame contracts, observation quality, solver comparison and holdout validation. Those checks remain necessary after the dataset can constrain the intended transform.