2.1

LumiScan Technology

The triangulation principle is still widespread in today’s sensor technology, such as passive stereo sensors or active systems with structured illumination. These systems have frequent problems with complex objects, due to shading effects or in the case of metallically shiny surfaces. Here, the assignment of the corresponding image areas as the basis of triangulation cannot be done correctly. In order to avoid such errors, these procedures are often complex in the setup or parameterization.

Lichtfeld imaging and the LumiScan Technology in particular offers completely new possibilities. The light field contains the complete optical information of a scene. By measuring the light field, the exact reconstruction of this scene, such as object shape and position, as well as surface color and gloss, is possible. This is done highly robustly of any interference. Due to these advantages, the consistent use of the LumiScan is predestined to meet the increasingly higher requirements for optical sensors in productive environments in the long term.

The patented pending LumiScan technology from HD Vision Systems GmbH teaches a robot to see, much like humans themselves. When people take a component into their hand for further processing in production, they check whether the component meets the desired quality: Is the colour correct? Are scratches visible? Is the correct serial number present? Is the form correct? The size and visual appearance? All these are questions that are largely unconscious in humans during the manufacturing process. This processing is now also open to robots, which can also take on complicated tasks.

As a spin-off of the IWR of the University of Heidelberg, HD Vision Systems GmbH has a technology that reflects the latest research results. The novel, light field-based sensor technology is a hardware/software combination. This allows to measure the object geometry, object color and gloss properties with only one sensor even on complex and glossy surfaces. At the same time, Deep Learning approaches are applied to 2D and 3D data. This allows for the first time to solve many tasks of automation under the umbrella of Industry 4.0 in a highly accurate, easy to use and flexible manner, with only one sensor system.

The technology of HD Vision Systems GmbH is easy to use and simple to implement. The team has extensive expertise in all areas of industrial image processing. As such, integration of the technology is offered in addition to the LumiScan products themselves, either by HD Vision Systems GmbH or by their partners. Via Profinet, OPC-UA and GenICam, the systems implement standard interfaces and can thus be seamlessly integrated into the existing infrastructure and machine control. The software can be extended modularly via a simple to use API. The customer is offered the entire process chain from raw 3D point clouds to application-related parameters. This includes object and layer detection, the handle in the box, barcode and OCR for traceability and IO/NIO values for deviations from CAD models. Customer-specific solutions are also developed by HD Vision Systems GmbH.

HD Vision Systems GmbH successfully implements projects with strong partners, leading suppliers of the automotive industry, the packaging industry and in the manufacturing industry. As a result, the LumiScan products are well proven for production requirements under challenging environments.

2.2

LumiScan Products

HD Vision System’s LumiScan products sample the plenoptic function with discrete cameras. The term plenoptic was derived from the word roots plen- (plenus) and opti- (optos), which means full/complete and eye/view, respectively. The plenoptic function was coined by Bergen and Anderson to describe the intensity of each light ray in the world as a function of visual angle, wavelength, time, and viewing position. It captures everything that can potentially be seen by an optical device and is related to previous concept of J. J. Gibson’s “the structure of ambient light” and Leonardo da Vinci’s “visual pyramid.” The advantage of using discrete cameras for measuring the plenoptic function is the flexibility and high resolution of this setup. The first product, primarily tailored towards the robotics and automation market, is the LumiScanX, shown in Figure 1.

Figure 1.

LumiScanX, a sensor system consisting of 13 discrete industrial-grade cameras. The footprint of the system is 112mm x 122mm x 52mm.

Other hardware configurations are presented in Figure 2. They consist of single camera gantry setups or linear arrays. All these configurations have different strengths. For example, with a single camera, a temporal multiplexing of the plenoptic function is performed. This facilitates high quality captures with a high-resolution camera. Only static objects can be captured in this fashion due to a significant longer capturing process as compared to multi-camera setups. All the configuration shown in Figure 2 have been produced for customers and are currently used for different aspects of their automation or quality inspection tasks.

Figure 2.

The LumiScan technology is independent of a specific hardware. Different system layouts such has single camera.

3.1

Bin-Picking

Figure 3.

Sample bin-picking application. The general setup and an image of the objects in a bin are shown in the top row. The point cloud and detected object candidates for performing the pick process below.

The task of Bin Picking is a core problem in robotics. The goal is to have the robot to pick-up known objects with random poses out of a bin. This process, if robust, facilitates the handling in automatic production tremendously. The tasks is made difficult due to occlusions of the objects in the bin and to identifying objects, the robot can reach without collisions or singularities of the required poses in reaching the object.

The LumiScanX sensor and it’s accompanying software suite has been specifically designed for this tasks. It is fully operational at customers and can be easily integrated into present processes. The technology is robust against occlusions, making the process easier to handle than with current sensors. Objects can be taught-in via CAD data or directly through examples. LumiScan is easy to install and set up. Communication with all common robot types and image processing libraries is done by default via OPC-UA, Profinet or Gen-TL.

3.2

Quality Inspection

Figure 5.

Results of deep-learning-based defect classification and localization. In this sample, scratches are detected on metal parts.

Besides the localization and gripping of objects, quality inspection is a vital task in fully automated production systems. To this end, HD Vision Systems offers Deep Learning-based approaches. Defects classes are simple to teach-in. The quality inspection tasks can be done in conjunction with handling by the robot. Thus, no or only minimal additions to cycle times are required.