Bin picking is the task of selecting an object from a container where parts overlap, occlude one another and change position after every action. The robot must reconstruct enough of the scene to propose a grasp, verify reachability and collisions, execute with feedback and decide whether the outcome requires a retry.
AI is useful because the number of appearances and object arrangements can make hand-written rules brittle. Learning can segment instances, estimate pose or score grasp candidates. The rest of the system remains physical: camera calibration, gripper geometry, robot kinematics, collision models and force limits determine whether a promising prediction becomes a safe pick.
Bin picking is one component of a warehouse picking system, and it also appears in manufacturing part feeding. The evaluation must match the use case. Fulfillment emphasizes item variety and verification, while manufacturing may emphasize precise orientation and machine loading.
Clutter hides both objects and free space
Only part of an object may be visible, and repeated identical parts make instance boundaries ambiguous. Reflective metal, black surfaces and transparent packaging degrade depth sensing. The bin wall creates occlusion and limits the approach angle, while a deep pile increases collision risk.
The system does not need a perfect model of every pixel. It needs enough task-relevant geometry to choose an action with acceptable risk. Sometimes moving an easy top object is better than attempting the requested item immediately because the first action creates visibility and space for the next.
Sensing and calibration create the robot's scene
RGB cameras provide texture and color; depth cameras or stereo provide geometry; structured light and time-of-flight have different failure modes. The camera-to-robot transform must place the sensed points in the manipulator frame. Small calibration errors can shift a fingertip or suction cup enough to miss.
Measure depth noise, missing regions and calibration drift under production lighting. Mount stability, vibration and protective windows matter. Recalibration should be a documented maintenance action, and the system should detect when sensor quality falls outside the trained or validated range.
Object separation can be instance-based or grasp-first
An instance-segmentation pipeline tries to assign visible pixels or points to individual objects, then estimates pose. A grasp-first system predicts contact candidates directly from the scene without requiring a complete object identity. Known-part manufacturing can use CAD models, while mixed inventory may rely on category or shape features.
Choose the representation from the downstream requirement. Exact pose is valuable for assembly and oriented placement. A stable suction point may be enough for order fulfillment when a barcode is verified after the pick. Offline segmentation accuracy is not a substitute for physical task success.
Grasp generation needs geometry and task value
A grasp candidate contains contact location, approach direction, hand opening and sometimes expected force. The scorer should consider surface quality, object stability, collision clearance, reachability and what the next process needs. The highest geometric score may be useless if it leaves the item in an unscannable orientation.
Candidate diversity improves recovery. If one suction point fails, a side pinch or alternate face may remain. The planner should avoid repeatedly choosing nearly identical failed grasps. Failure memory and a rescan can update priorities after the pile moves.
Collision checking includes the gripper and carried item
The arm, wrist, fingers, hoses and tool body can collide with bin walls or neighboring objects. The path into the bin may be feasible while the exit path with a larger carried item is not. Approximate geometry speeds planning but needs a conservative margin grounded in calibration error.
A safety or motion controller may slow, stop or modify the path after the AI selects a grasp. Report these rejected candidates and stops because they affect cycle time. A perception model cannot claim system throughput without the downstream feasibility filter.
| Stage | Output | Physical constraint | Metric |
|---|---|---|---|
| Sense | Calibrated RGB/depth scene | Noise and occlusion | Valid depth and drift |
| Separate | Objects or graspable regions | Overlap and repeated shapes | Task-level coverage |
| Plan | Ranked grasp and path | Reach and collisions | Feasible candidates |
| Execute | Monitored contact | Slip, saturation and latency | First-attempt success |
| Verify | Identity and outcome | Double pick or drop | Accepted correct picks |
The gripper changes which AI predictions matter
Vacuum needs a sealable surface and enough exposed area. Fingers need clearance around contact points. Magnetic tools work only with compatible materials and may lift multiple thin parts. A tool-changer increases action options but adds selection, maintenance and cycle-time decisions.
The broader guide to robot hand difficulty explains why more dexterity also brings sensing and durability costs. Bin picking often benefits from a simpler tool matched to the material distribution. Train and evaluate the model with the actual end-effector geometry.
Verification and retry define autonomy
After closing or lifting, the robot should confirm that an item was acquired, only one item was acquired and the identity matches the task. Vacuum pressure, finger position, wrist force, weight, barcode or vision can contribute. An uncertain result should not silently enter the downstream process.
Retry policy matters as much as first-attempt success. The robot may regrasp, shake, rescan, place the item at a verification station or request help. Report eventual success and time-to-success separately so repeated slow attempts do not inflate capability.
The five-decision loop makes failures diagnosable
Separate scene construction, candidate separation, grasp ranking, collision-checked execution and verified retry. When a pick fails, label the earliest stage that created the bad decision and preserve the sensor and controller logs. This produces targeted data rather than one generic failure class.
The card can become an operational dashboard. Track which item families and pile conditions fail at each stage. A rise in sensing failures may indicate dirty optics; a rise in execution failures may indicate gripper wear or calibration drift rather than model regression.
Manufacturing and fulfillment optimize different outcomes
Manufacturing bin picking often handles known parts and must present a precise orientation to a fixture or machine. Fulfillment handles wider packaging variation and may accept arbitrary orientation if identity and destination are verified. Their benchmark objects, tolerances and cycle definitions should differ.
ABB describes its Robotic Item Picker as using AI and vision for varied warehouse items. Vendor percentages should be interpreted with the tested assortment, pick definition, operating conditions and exclusions. Reproduce the metric on the site’s inventory before extrapolating.
| Use case | Priority | Typical output | Hard exception |
|---|---|---|---|
| Order fulfillment | Coverage and identity | Item in order tote | Deformable or reflective package |
| Machine tending | Pose and reliability | Oriented part in fixture | Entangled or nested part |
| Kitting | Correct set and quantity | Verified kit slot | Look-alike parts |
| Depalletizing | Throughput and layer access | Case on conveyor | Damaged or unstable case |
A production benchmark needs scene and item variation
Build a test set from the real object and pile distribution, including difficult surfaces and partially damaged packaging. Hold out items or arrangements from training. Run enough repeated scenes to measure confidence intervals, recovery and degradation as the bin empties or becomes densely packed.
Report correct verified picks per hour, first-attempt and eventual success, interventions, damage and unpickable cases. Save failure images and system state for later improvement. A benchmark that contains only easy top objects measures a different operating envelope from a production bin.
- Use calibrated production sensors.
- Test the actual gripper geometry.
- Hold out object and pile variation.
- Measure verification and retry.
- Preserve failures by decision stage.
Frequently asked questions
Are bin picking and piece picking the same?
They overlap, but bin picking emphasizes objects mixed in a container. Piece picking can also start from organized shelves, trays or conveyors.
Can bin picking work without AI?
Yes, especially with known parts, fixtures or geometric rules. AI becomes useful when object and scene variation makes rigid programming expensive or brittle.
Which objects are hardest?
Transparent, reflective, dark, deformable, entangled, porous or visually similar objects are often difficult, but the actual result depends on sensors, gripper and pile conditions.
What is a good bin-picking success rate?
No universal rate is meaningful without the object set, denominator, verification rule, retry policy, cycle time, damage and intervention definition.
Why does performance change as the bin empties?
Visibility and free space may improve, but remaining objects can be awkwardly oriented or trapped against walls. Test the full depletion cycle.
System Comparison Note
Bin-picking products and performance claims change. Verify the current sensor, gripper and software configuration, then reproduce results on representative objects and pile states.