SDK 2.17: Full Zivid 3 Support and New Barcode Reader

We have just successfully launched the new Zivid 3 XL250, a 3D camera for big robots and extra-large workspaces. Overall, it has been quite a year for us. We have launched two product lines, consisting of four separate cameras, to cover the scope of robotic picking and manipulation across multiple industries. We have released SDK 2.17 with some exciting new things. The main headlines are the support for the new Zivid 3 XL250 and... barcode reading for all Zivid cameras! Watch the short video on this new SDK or dive into the longer blog post below.

Table of contents

• Support for the new Zivid 3 XL250
  Application presets for Zivid 3
  Increased resilience to ambient light
  Color-separated unorganized point cloud stitching
• Barcode reading!
• Zivid Studio updates
  New Z-range filter slider
  Hand-Eye GUI - more feature support
• Knowledge Base articles 
• Summary

Support for the new Zivid 3 XL250 

We have just launched our 6th-generation 3D camera, the Zivid 3 XL250. This camera extends the use cases for Zivid cameras to applications that involve big robots working on large workspaces. Two areas we expect to have an immediate impact are palletizing-related tasks in warehouses and logistics operations, and factory bin-picking in manufacturing enterprises. It is an astonishingly good 3D camera that has seen Zivid take the best of what we already do and push the envelope even further for performance and reliability. 

Of course, a camera is hardware, and it needs to be supported with a software to perform. As always with Zivid, we deliver full and complete software support at launch. No waiting around with a bare-bones set of features with the promise of more complete support down the line. No, our cameras launch ready to do what all our other cameras can do without exception.

Application presets for Zivid 3 

We have added some new presets for Zivid 3 that are tailored to its strengths and focus on the types of tasks we know our customers want to use it for.  

For robotic depalletizing, we have 2 presets: One for the highest capture quality and one for the fastest capture speed. The highest quality preset is for the scenes that may have strong ambient light and mixed SKUs of varied and challenging items, and ambient light. For example, these might be transparent bottles, shiny black polybags, and reflective soda cans. This preset makes multiple acquisitions to ensure complete coverage of any item, yet still comes at a capture time of around 1 second. 

The fastest capture speed preset is still excellent for coverage on most items and can probably be used for most use cases. This setting has a very fast <500 ms capture time. We’d recommend trying this preset first; if there are some stubborn items, you can easily move to the higher quality setting. 

Robotic manufacturing tasks can have differing considerations for depalletizing. Most items to be handled and manipulated are usually made of some sort of metallic material. This brings its own challenges with interreflections between the parts themselves, with bin walls, and ambient light effects. We have optimised these presets to be especially performant for typical bin-picking challenges in factories.

We have 3 manufacturing presets for Zivid 3: Specular, Small details, and Diffuse.  

• The specular preset is designed for the shiniest metal parts, such as chrome tubes and reflective plates. It is very effective at combating the reflection artifacts that are often encountered in such scenes. The capture time for this preset is typically 1000 - 1500 ms on a good computer.  

• The small details preset excels at rendering small holes, raised areas, and intricate detail with very high accuracy and precision. This can be especially important, for example, if CAD-matching is being used to identify parts, and such features of an item need to be clearly represented for the matching algorithm to work reliably. This preset also takes about 1000 - 1500 ms on a strong computer.  

• The diffuse setting is the fastest preset for manufacturing, needing typically 600-900 ms, and is ideal for more matte surfaces that do not give off very specular reflections, for example, when picking steel billets from a large factory bin.  

If you are using a slower machine, such as an integrated Intel GPU, the capture time can typically be 2-3 times slower. 

Shiny metallic parts with holes and small features are reliably captured with Manufacturing – Specular and Manufacturing – Small details presets. 

The Manufacturing - Diffuse preset is ideal for capturing steel billets in factory bin-picking. 

Increased resilience to ambient light 

2D images play a key role in robotic automation. 2D images are commonly used to identify individual items in a scene. For this reason, 2D image stability and consistency are critical. Zivid cameras provide their own illumination source using structured light; however, other light sources are also usually present, and we want to suppress their effects as much as possible. Zivid has a range of techniques that enable this, and this feature is an additional one. With this feature, a sampling interval suppresses the effects of flicker and light level fluctuation that can be seen with supply lighting that operates at 50 Hz or 60 Hz. This feature ensures consistent quality and color for 2D images. Currently, this is only supported for Zivid 3 XL250, but we expect to roll this feature out across our other cameras in the future.

Color-separated, unorganized point cloud stitching 

Stitching multiple point clouds together is often done in robotic automation. It is not always possible to get a complete point cloud as is required by just capturing from one viewpoint. This means an item will be captured from various viewpoints to get a more complete point cloud. The challenge here is that the coordinate system of these separate point clouds will differ from each other, because they have been taken from a different viewpoint, either we have moved the 3D camera, or the position of the item itself. What we must do now is transform the coordinate systems to a unified system so that common points are aligned. 

The stitched point clouds can be viewed in the Hand-Eye GUI tool and evaluated. This new feature helps with this evaluation process. If there are multiple point clouds and they are all represented in a single color, it can soon become quite difficult to tell the point clouds apart. So now each point cloud can set its own color. After the stitching process, the different point clouds are easily distinguishable from each other when the transformation is complete. This makes evaluating the quality of the transformation and stitching process a lot faster and reduces frustration. 

Point cloud stitching using several point clouds marked up in different colors. 

Barcode reading!

This is an exciting new feature for the detection of 1D and 2D barcodes. Many robotic picking cells incorporate some form of barcode reading, and this is especially true in warehousing and logistics applications. With this new feature, all Zivid cameras support barcode detection and reading. As the barcode reading happens in 2D, its performance is directly related to the size of the barcode being read and the FOV of the camera. The pixel coverage of the barcode will determine if it can be read. Zivid cameras with more pixels per region will therefore be able to read smaller barcodes, while cameras with larger FOV will be able to read somewhat larger barcodes. A range of common barcodes is currently supported in this SDK. This feature will be developed further in upcoming SDKs. Barcode detection is supported in Zivid Studio by selecting the Barcode preset, then the barcode can be viewed in the GUI, where the code is shown with the associated barcode. 

Barcode numbers are displayed with their associated barcode in Zivid Studio. 

There are some aspects around barcode reading and performance that should be considered. The size of the barcode and its condition will have an impact. The camera sensor and the camera’s FOV will determine how many pixels per millimeter are available to render the barcode clearly. We recommend using the barcode reading presets in Zivid Studio for optimal results. The Zivid 2+ MR60 has the best barcode reading capability as it has the most pixels per millimeter of our cameras. It can reliably read 10 mil barcodes, and it is possible to read as small as 7 and 8 mil barcodes. As cameras with larger FOV are used, the barcode reading works with progressively larger barcodes. 

This new feature has just been introduced, and we will work on continuous improvement for all cameras so that the barcode reading capabilities get progressively better and more capable over time. 

Overview of some supported barcode formats.

The barcode reader is available for testing through Zivid Studio. Evaluation and development licenses are free of charge, and licenses for commercial deployment require a commercial agreement. 

Reach out to your Zivid contact or sales@zivid.com for more information.

Zivid Studio updates

We have made some improvements to Zivid Studio in this release to add more functionality and improve ease-of-use: 

Z-range filter slider

The Z-range slider feature is useful for simplifying point cloud evaluation. This allows a user to select the depth, or Z-range, that is viewable in Zivid Studio. This means that areas of interest in the point cloud can be isolated without the clutter of the entire scene. Previously, the range selection was done via a text box selector. Now this tool can be easily snapped to the desired range using a mouse to control the range in the depth map window. 

Additionally, Ethernet link speed with ‘connected’ status is added in Cameras and Cameras Network configuration dialogs.

Hand-eye GUI – more feature support

The Zivid Hand-Eye GUI has been available for a year now and is a great visual tool for speeding up your calibration and verification tasks. If you haven’t checked it out, it is well worth a look. You can find out more about it here. With SDK 2.17, we have added more SDK features, such as point cloud stitching, to this tool. 

The Hand-Eye GUI runs independently of Zivid Studio and requires the installation of Python 3.0 or later. 

Hand-Eye GUI tool.

Knowledge Base articles 

We have updated the Zivid Knowledge Base with several supporting articles for this SDK release: 

• Depalletization and palletization tutorial
• Barcode detection
• Zivid 3 Quick Mount
• New APIs
• Zivid 3 PSU
  

Summary

To wrap up, our new Zivid 3 XL250 camera for big robots and big workspaces has full SDK support for customers to start developing their applications immediately. We have also introduced barcode reading, which offers great opportunities to streamline and simplify picking systems design and cost. Customers can begin evaluating this feature at no upfront cost to find out if it suits their demands. Zivid Studio and the Hand-Eye GUI have also been upgraded to offer a range of improved options and even better ease of use. We hope you like what we’ve brought to you this time around. Now, it’s back to work for us to bring you an exciting SDK 2.18.