8 Machine Vision Challenges Affecting Picking System Throughput

We have come a long way in automation, but ‘‘the simple’’ task of picking in robotics has caused us to still use manual labour for repetitive tasks which require cognitive ability, for example, goods-to-man or order fulfilment in logistics and e-commerce. To cope with the increasing demands in manufacturing, logistics and fulfilment industries, better imitation of human-like movements is required. Simply put, we want robots to work smarter, faster, and more flexibly.

On the surface of it, picking is as simple as detect, pick, and place. However, in each phase some error is introduced accumulating an overall high uncertainty in picking an object. This uncertainty can be reduced manifolds by using a 3D vision system that replicates reality as it actually is. As a sales engineer leading business development in Europe, I have come across 8 challenges that industries are facing when it comes to choosing the right machine vision technology for their automation solutions.

1. Precision of 3D points

Precision of vision systems is an integral part of picking applications. We want robots to pick accurately by looking at the objects. This is where the ‘’detect’’ part of automated system is crucial. Higher precision gives you higher confidence to detect objects to be picked with all the details. Precision of a 3D system impacts how and where an object will be picked.

2. Trueness of represented reality

Precision is one of the determining factors for localizing pick points, however what most people do not pay attention to is the trueness of represented reality. A vision system can be very precise but does not necessarily depict true reality causing uncertainty in performing automated tasks. For example, if we are to pick objects from a bin, precision would give the ability to ‘detect’, however trueness will determine the reality of object in space coordinates providing confidence to ‘pick’, lack of which poses significant challenge in output of picking systems.

3. Speed of acquisition

As discussed above, we want the robots to compete with human capacity, which means to perform tasks as good as humans to achieve a return on investment (ROI). With increased production, warehouse logistics, and order fulfilment, speed is one of the crucial factors that determines if automated picking system is worth the investment or not. Vision system should be able to support at least 600 picks/hour or more in order to compete with the current solutions otherwise the high quality of data is of no use.

4. Colour dimension

Today many of the picking systems are not utilizing colour dimension for more accurate picking because most 3D vision systems are not providing that information. One method is to add an additional 2D colour camera to get this information, but this just creates more problems than it solves. For example, the pain of calibration. Think of it this way, if you are supposed to pick different objects, colour adds value by introducing another dimension. The same goes for robots. Colour adds more trueness to the replicated reality, which can decrease the uncertainty of picking points.

5. High Dynamic Range (HDR)

Moving to more automated picking systems, including random picking etc, such vision systems are required that provide high dynamic range. Meaning, picking objects from a pile where you have the whole range of contrast spectrum requires vision system to provide data on all of them for accurate picking. This is achievable if the vision system used provides data over the whole range instead of only providing data on part of the contrast spectrum. For example, mix dark and bright/shiny parts in a bin and try to get data on all of them in single shot, you will face major challenges because of two ends of the spectrum, unless the vision system can provide that feature. An important factor to consider when you work in logistics and e-commerce.

6. Handling reflective material

Metallic and shiny objects are still a major challenge in the picking domain. The reflections caused by metallic/shiny objects cause a lot of error in determining where to pick the object. For starters, these reflections create ghost points in air, which the robot still sees as an object and tries to avoid during motion planning. Tackling reflection is a feature that can solve detection challenge in metallic/shiny/reflective parts.

7. Form factor

Robotics is a domain where size matters! Contrary to the general belief, the smaller the better. Having a large baseline of vision system not only introduces occlusion, contributing to the error, but also limits the use of vision system in different scenarios. For example, robot mounted scenario. In this scenario, large baseline will not only introduce error in data but also requires care to avoid collision of vision system with surroundings while moving the robot, especially in bin/piece picking where items are in a container.

8. System stability

Expansion of robotics, in all kinds of industries, has introduced operating conditions which are ‘’harsh’’ and deteriorate the quality of 3D data over time, provided by vision system. This is a major challenge automated picking tasks have been facing. 3D vision systems are highly impacted by environmental conditions over time because of the optics involved in them. A 3D system, that can adapt according to external conditions and still provides confident picking tasks for long term, is the need of the hour. In short, vision system should be industrial-grade.

In summary, these machine vision challenges have previously limited the use of automated picking systems but not anymore! We, at Zivid, have solved these challenges to give developers of picking system a painless and smooth 3D machine vision experience. In my next article I will explain how our upcoming product has tackled all the above challenges for smooth and accurate picking experience.