Model-based machine learning for model variance detection, recognition, and classification in dynamic quality assurance systems

Example of the application of object detection in quality control. The actual position of individual components is detected and compared with the nominal position obtained from the CAD model.
Example of the application of object detection in quality control. The actual position of individual components is detected and compared with the nominal position obtained from the CAD model.
Beispiel für den Aufbau eines Qualitätskontrollsystems mit den im Rahmen von MODAL entwickelten Kameras. Die 3D-Kameras erfassen den Ist-Zustand der Baugruppe auf dem Förderband. In diesem Beispiel kann sich ein Werker die Ergebnisse des Soll-Ist-Abgleiches via Augmented Reality visualisieren lassen.
Example of the setup of a quality control system with the cameras developed within MODAL. The 3D cameras record the actual state of the assembly on the conveyor belt. In this example, a worker can visualize the results of the target-actual comparison via augmented reality.

MODAL is a publicly funded project of the Federal Ministry of Economics and Technology (BMWi) in cooperation with Optonic, a company based in Freiburg im Breisgau. The aim is to optimize and increase the efficiency of 3D cameras for recording 3D data in the application area of industrial quality control by means of automatic object detection. In automatic object detection, a 3D image of an industrially manufactured object is captured and compared with the available CAD model. The challenge is to enable the cameras used in this process as compact systems to perform this process independently and resource-efficiently by developing and integrating a machine learning (ML) system using augmented reality as a hardware module. A corresponding workflow combines both the preparatory processing of CAD geometries for the necessary ML learning processes and corresponding defect classifications with a fast setup of the inspection system for optical inspection on dynamic production lines.


Automatic object detection

Building on the existing data library, highly efficient ML procedures will be developed in MODAL so that training with rendered imagery can be activated concurrently with the AR application based on a hardware-assisted pre-processing step. For this purpose, the workspace in which object recognition is used is optimally reduced. Here, the creation of a semi-automated workflow is necessary to train the ML processes with synthetic and real images of the components to be inspected, which are then recognizable in a complex shoring condition and can be matched with a 3D specification as product data structure.


Neural networks facilitate point cloud matching

The synthetically generated images of components have to be processed for the ML methods afterwards and the neural networks have to be trained in such a way that a fast object recognition and classification in real product structures is possible. The main challenge here is the matching with preprocessed/segmented point clouds, the recognition of partial structures in point clouds and the combination of corresponding methods on optoelectronic level for temporal optimization. The improvement of these approaches to industrial application is to be seen in a safe handling of background noise as well as in the implementation for the use for products of different construction states and variants. The data process should be as simple as possible for the user and should not require any input other than the CAD data.

Learn how MODAL works in this video:

Privacy warning

With the click on the play button an external video from is loaded and started. Your data is possible transferred and stored to third party. Do not start the video if you disagree. Find more about the youtube privacy statement under the following link:

Funded by