Optimal Spatial Registration of SLAM for Augmented Reality

Augmented reality (AR) signifies a paradigm aimed at merging a person’s perceived, real environment with digital information placed within the 3D space. Typically, this involves embedding in real time virtual 3D graphics in images captured by a moving camera or directly in the user’s field of vision using transparent optical displays. Displaying these objects in the correct location and in accordance with the user’s perspective requires a solution to various static and dynamic registration problems in order to give the impression that the virtual and real worlds are seamlessly connected to one another.

© Fraunhofer IGD
Folker Wientapper (left) with his adviser Prof. Dieter W. Fellner (right), director of Fraunhofer IGD.

During his time at the “Virtual & Augmented Reality” Competence Center at Fraunhofer IGD, Wientapper worked extensively with tracking technologies, which are needed for AR applications. The progress made in the last decade in the field of simultaneous localization and mapping (SLAM) has made an important contribution to this technology. With SLAM, it is possible to reconstruct a  real environment while simultaneously capturing the dynamic movements of a camera from its images without having to prepare the environment in advance. As part of his dissertation, however, Wientapper determined that SLAM solves just one part of the overall registration problem: The generated 3D spatial geometry and the computed motion path are only placed in space in relation to a freely selected coordinate system.

Without a corresponding comparison of coordinate systems, a human observer’s real world remains forever disconnected from the virtual world. Existing approaches at solving this problem either require a virtual 3D model, which needs to reflect the real object with sufficient accuracy (model-based tracking) or they rely on application-specific assumptions and additional sensor data (like GPS signals or the “Manhattan world” assumption), which limits the overall applicability of these methods. The fact that automated registration is desired but not always possible creates the need for methods with which a user can specify connections between the real and virtual worlds while setting up AR applications in order to be able to monitor and control registration. These methods require the help of numerical algorithms that maximize the information in order to achieve accurate registration results.

The public defense of the dissertation on "Optimal Spatial Registration of SLAM for Augmented Reality" took place on March 15 at Fraunhofer IGD in Darmstadt. The advisers were Dr. Arjan Kuijper (TU Darmstadt), Dr. Dieter W. Fellner (TU Darmstadt) and Dr. Didier Stricker (Kaiserslautern).