SDK 2.11: Simple Networking and Pipeline Optimization for Speed Boost

SDK 2.11 has an emphasis on making network connectivity with Zivid 3D cameras simple and efficient. It is also about making the process of data transfer as efficient as possible to keep driving those capture times down. As always there are some other nice things to make you smile, so let’s jump in. 

No time to read? Just check our summary video:

Plug-and-play networking for Zivid ethernet devices

Networking issues with hidden devices or unconnectable devices can be a frustrating headache. You want to just connect your device to the network, find it, and connect to it. So, we’ve introduced some more finesse to the networking with Zivid ethernet cameras. Now when the Zivid 3D camera is connected to the network it will be found and visible. The 3D camera is identified as connectable, or unconnectable. In the case that it is unconnectable this is often due to a duplicate IP or being on a different subnet. 

Example of detecting a wrongly configured Zivid 2+ camera in Zivid Studio 

This upgrade allows you to see the camera and then reconfigure it so that it is connectable. This functionality is present in both the SDK and Zivid Studio. The NetworkCameraConfigurator tool allows IP reconfiguration and can now accept a camera’s serial number or IP as an identifier. This makes network management of Zivid cameras simple, even for cameras that are on a different subnet. The mDNS (multicast DNS) protocol is used to resolve IP addresses and hostnames and has the advantage of not needing a DNS server, and very simple setup. 

Configure inaccessible Zivid camera in the CLI tool by serial number 

Major speed improvements with 4x4 subsampling

At Zivid we put a lot of effort into offering you a broad range of camera capture strategies for 2D and 3D that match your application's needs. In our previous release of SDK 2.10, we introduced a subsampling feature that offered a considerable reduction in overall capture time by using fewer points and consequently less network bandwidth and memory usage. When using this subsampling feature, the number of points in the point cloud is reduced without affecting the precision or accuracy of any individual point. You simply trade the number of points for speed.

Here is an example of how fast the Zivid 2+ can be thanks to 4x4 subsampling:

We made this feature based on feedback from customers in e-commerce and logistics that demanded even faster captures from Zivid 2+ without the need for very high-density point clouds. In this release, we’ve taken this a step further by adding 4x4 subsampling: 

• The maximum point cloud capture rate possible with a Zivid 2+ is almost doubled (from ~100ms to 50-60ms). 
• Point cloud captures with compute-heavy settings (like Omni engine) on weak GPUs are up to 3x faster in 4x4 mode compared to 2x2 mode. 
• The Zivid 2+ settings presets called ParcelsFast, ParcelsQuality, and ConsumerGoodsFast have been modified to take advantage of the 4x4 mode. 

Below is an overview of capture time benchmarks on different GPUs. 

Capture time improvements from SDK 2.10 to SDK 2.11 

Point cloud with full resolution (2448 x 2048 pixels)

Point cloud with 2x2 subsampling (1224 x 1024 pixels)

Point cloud with 4x4 subsampling (612 x 512 pixels) 

Faster across the full vision pipeline

Multiple actions and operations happen while making 2D and 3D captures. At Zivid we have always had a focus on driving down overall capture times. To do this we look at every operation that is happening and try to find individual areas to optimize that will reduce capture time. By making data available as soon as it is ready, other processes can begin working with it as fast as possible, thus making the system faster and reducing the time budget allocation required for machine vision-related tasks. 

In this SDK we have been paring away at the time budget by making concurrent image acquisition, data transfer, and processing possible together with several firmware optimizations. 

When doing 2D + 3D captures, the SDK will trigger the initiation of 3D point cloud acquisition concurrently with the transfer and processing of a 2D capture. Previously 3D processing would begin only once 2D capture was complete. This new feature improves overall capture time by 15-20 milliseconds on high-end GPUs (Nvidia), and up to 50 milliseconds and mid-range GPUs (Intel). A further range of firmware improvements are implemented that shave off another 5 milliseconds in total capture time, these changes do not introduce any changes to the API. The scheduling, pipelining, and timing are exemplified in the figure below where the capture time for 2D and 3D imaging is measured for a parcel handling application on a high-end NVIDIA GPU and compared between SDK 2.10 and 2.11.  

Pipeline speed improvements in SDK 2.11 

Improved 2D image quality with glare reduction

There is now the option to choose between linear RGB, and sRGB for 8-bit images. This is designed to aid capture assessment when using Zivid Studio. The color space mapping sRGB is standard for the web and monitors. Using the sRGB color space means displayed images are more realistic and brighter.  

sRGB is a very common format for 2D cameras and this feature will help align and improve performance with software that is designed to give the best results with the sRGB format. This will be beneficial to many AI algorithms that are already trained with sRGB images. It is also expected that this feature will help to reduce glare overall in 2D images. 

2D image with Linear RGB vs sRGB

2D image in Linear RGB color space

2D image in sRGB color space

Summary

Capture speed and ease-of-use improvement is a journey that is never over for us at Zivid, there is always something more that can be done to improve both important aspects. Consequently, you will see these two appear frequently in our SDK releases. But behind these two high-level customer goals, there is a broad scope of deep improvement going on, with our engineers attacking the challenge on multiple fronts. The feature improvements in SDK 2.11 have the potential to improve capture times by 10-300%, which is, we believe, a welcome Xmas present for our customers. 

Merry Christmas! And happy developing with Zivid.