Eliminating Double Picks in Robotic Parcel Handling Cells

In our last blog, we took an initial look at robotic parcel induction cells that are making their way into modern logistics sorting and distribution centers. We covered the process of parcel induction and sorting and specifically looked at why speed, essentially robot cycle time, is so important. I recommend that you read through that blog together with this one to get a fuller understanding of this application and the role of machine vision in the system. 

Having already examined speed, today, we will examine the flip side of the coin: the importance of the performance and reliability of the data produced by the robot cell’s vision system. As always, it is good to examine some challenges and problems associated with the topic. We will tie this subject in with a critical pain in robotic parcel induction: the dreaded double pick. 

Table of contents

1. What is a double pick?
2. The pain of picking errors
3. How it works
4. Segmentation and singulation software performance
5. Delivering 2D images your AI loves
6. Completeness over correctness for the 3D data
7. Do all this with one camera
8. Conclusion

What is a "double pick"?

In the parcel induction process, we looked at previously, we saw that parcels and packages arrive as large unordered piles of parcels, packages, and boxes. The initial process that we are looking at in the induction process is known as singulation. Singulation is the process of transitioning a big pile of items into a singular flow of items so that sorting systems downstream can process the incoming stream of items in an efficient and orderly way. For this to happen, the job of the singulation cell is to pick and place items singularly at an even rate onto a conveyor that takes the singulated packages to the sorting stage. A double pick is when the robot cell mistakenly picks 2 items simultaneously and places them on the conveyor. As the system is picking, it is updating the backend system as to the nature of the package, is it a small box, is it a black poly mailer pack, is it a package covered in bubble wrap, etc. 

Unsorted packages arrive in bulk to be singulated for downstream sorting.

The pain of picking errors 

The pain and knock-on effects of double picks are considerably problematic. As you can imagine, there are now 2 items on the conveyor headed downstream to the sorting stage, and the overall system believes there is only one. As those items approach the appropriate sortation bin they will be directed into it for further processing. We now have an item in the sortation bin that should not be there. The next stage of the sortation process may not be designed to deal with this ‘intruder’ package. Consequently, that next stage may be forced to stop and require an operator to go and address the problem, remove the package and take it back for reprocessing. Possibly even worse, the intruder package may make its way through and end up in entirely the wrong part of the facility undetected. Clearly, this kind of problem is very hard to mitigate and can lead to stoppage of the line and lost productivity. 

How it works

The parcel induction cell makes use of both 2D images and 3D point clouds to achieve its objectives. The camera makes a 2D acquisition(s), which is then sent for immediate processing. The 2D data is the basis for segmentation software to identify separate items. This data is typically available after approximately 5 to 20 milliseconds. 3D point cloud processing is more computationally complex and time-consuming than that of 2D images and thus takes a longer period. However, once the 2D data is available the AI-based segmentation software can begin to work with it and begin identifying items. 3D cameras such as the Zivid 2+ R-series have pipelined data and make the 2D data available as soon as it is ready. Once the 3D point cloud is ready, the system can start calculating candidate pick poses for the robot as the parcel segmentation is most likely already complete. This parallel approach to 2D and 3D data processing can significantly affect the robot cell’s performance by being ‘hidden’ inside the cycle time budget of the robot. 

The parcel segmentation and pick process (simplified).

Capture calculator in Zivid knowledge Base.

Segmentation and singulation software performance 

Segmentation and singulation software for parcel induction is a relatively new field that leverages the recent advancement in AI modeling to achieve its purpose. Companies such as Fizyr in The Netherlands are leading the way with state-of-the-art pick-and-place solutions that use Zivid 3D cameras for a range of industries, including logistics parcel induction. The very best systems can segment and calculate a pick for the robot in a few hundred milliseconds. As with any system using AI, segmentation software is dependent on the data it receives. The system uses 2 core data components for different purposes:

• 2D image data – Used for item detection and item segmentation
• 3D point cloud data – Used for depth sensing and pick pose estimation

Parcel segmentation software typically has those stages: 

• Image acquisition and processing
• Deep learning algorithm processing
• Parcel identification and separation
• Feature extraction and classification
• Quality control and damage detection
• Integration and output

The software begins by acquiring high-quality 2D and 3D images of the parcel scene. These images and point clouds are the data that underpin the success of the robot cell’s performance and reliability. 

The software processes the 2D and 3D to understand the scene identifying individual parcels even when they are overlapping or in complex arrangements. These neural networks are trained on vast datasets that enable the software to recognize parcels in various conditions, including different shapes, sizes, colors, and materials. 

The core functionality of the segmentation software lies in its ability to distinguish individual parcels. The software should be able to segment items even when they overlap or in challenging lighting conditions. The software classifies parcels based on their characteristics, which enables different picking strategies that are most suitable for the package. Another feature beyond segmentation is quality control and damage detection, which can be flagged to the wider system for countermeasures. 

The system then integrates with the wider system to instruct the robotic picking cell on which candidate pick poses are available and how they should be executed.  

Successful segmentation and candidate pick poses generated on different, and challenging items (Fizyr).

Delivering 2D images your AI loves 

As we have shown, the 2D image data is a critical component of the vision system in parcel picking. Aside from the speed of data delivery, the consistency and stability of this data will have a fundamental impact on the performance of your AI segmentation software. The things that really matter are:

• Consistent data, regardless of ambient light interference
• Crisp edge detail without blooming, color shift, or other artifacts
• Crisp, detailed edges without aliasing effects

The AI models used in segmentation solutions do not need very high resolution; often, around 1.5 megapixels is sufficient. However, they do need high levels of correctness and stability. A camera such as the Zivid 2+ MR130 with very high-fidelity 2D output can also downsample the number of pixels to be processed. As the inherent quality of each pixel’s data remains unaffected by the downsampling, this can offer a considerable reduction in overall processing time for parcel segmentation. 

Completeness over correctness for the 3D data

When it comes to 3D data, the demands flip over somewhat. Now, we are less concerned about the absolute accuracy of the point cloud. Parcel induction is carried out with robots using suction end effectors with a reasonable degree of compliance, typically around a few centimeters. This compliance means the end effector can overshoot a small amount without creating any issues. 

Point cloud comparison of popular 3D cameras under strong ambient lighting conditions.

Watch the full comparison video →

But with the 3D point cloud what really matters is the completeness, that we have a scene full of points without holes and missing areas. If the point cloud is missing areas, then it becomes difficult for the system to identify a good candidate pick point on the parcel, typically the center. With standard manilla-type packages and parcels, complete point clouds can be achieved by most 3D cameras. The challenge is the rapidly increasing number of plastic poly mailer packs that are being used in logistics. These mailer packs have shiny reflective surfaces that are very challenging to reproduce point clouds of. The choice of 3D camera you make on this criterion will be fundamental to the adaptability and reliability of your parcel induction cell. Choosing a top-end camera such as the Zivid 2+ MR130 will allow you to process any parcel currently in use and will also enable the processing of parcels wrapped in bubble wrap and other transparent materials. We have a detailed 3D camera comparison for parcel induction tasks here. The alternative will be the headache of pre-routing packages the cell is able to deal with and routing challenging poly mailer packs to human sorters. 

Zivid 2+ MR130 can deliver complete point clouds on any item encountered in logistics. 

Clear edge separation, especially on thin, overlapping envelopes. 

Do all this with one camera

The new Zivid 2+ R-series has taken both 2D and 3D imaging to new heights. The MR130 has been designed to meet all of the requirements we have covered in this blog in a single product, and has plenty of capability to adapt to changes in how parcels and packages are shipped. If you want to induct any parcel, then the MR130 is likely the 3D camera for you. 

All-in-one robot vision with MR130 removes the need for extra cameras and lighting. 

Conclusion

Double picks are a critical pain point in robotic parcel induction, causing mis-sorts, system stoppages, and lost productivity. By leveraging high-fidelity 2D and 3D vision systems, induction cells can accurately segment and singulate parcels—even challenging items like reflective items. Industrial cameras like the Zivid 2+ MR130 offer complete, high-quality data that ensures reliable pick pose estimation and minimizes errors. With the right vision technology, parcel handling cells can maintain high-speed, accurate operations while adapting to evolving logistics demands. 

But don’t take our word for it! We have compared the performance of the MR130 with 2 well-known competitor cameras and analyzed the results: