• Publikationen
Show publication details

Shahid, Muhammad Ali; Jesche, Benjamin-Paul; Olbrich, Manuel; Graf, Holger; Franek, Andreas; Kuijper, Arjan; Bockholt, Ulrich; Schmitt, Michael

Measurement Based AR for Geometric Validation Within Automotive Engineering and Construction Processes


Virtual, Augmented and Mixed Reality. Industrial and Everyday Life Applications

International Conference Virtual Augmented and Mixed Reality (VAMR) <12, 2020, Copenhagen, Denmark>

Lecture Notes in Computer Science (LNCS), 12191

We look at the final stages of the automobile design process, during which the geometric validation process for a design, in particular for the vehicle front end, is examined. A concept is presented showing how this process can be improved using augmented reality. Since the application poses high accuracy requirements the augmented reality also needs to be highly accurate and of measurable quality. We present a Measurement Based AR approach to overlaying 3D information onto images, which extends the existing process and is particularly suited to the application in question. We also discuss how the accuracy of this new approach can be validated using computer vision methods employed under the appropriate conditions. The results of an initial study are presented, where the overlay accuracy is expressed in image pixels as well as millimeters followed by a discussion on how this validation can be improved to meet the requirements posed by the application.

Show publication details

Wientapper, Folker; Schmitt, Michael; Fraissinet-Tachet, Matthieu; Kuijper, Arjan

A Universal, Closed-form Approach for Absolute Pose Problems


Computer Vision and Image Understanding

We propose a general approach for absolute pose problems including the well known perspective-n-point (PnP) problem, its generalized variant (GPnP) with and without scale, and the pose from 2D line correspondences (PnL). These have received a tremendous attention in the computer vision community during the last decades. However, it was only recently that efficient, globally optimal, closed-form solutions have been proposed, which can handle arbitrary numbers of correspondences including minimal configurations as well as over-constrained cases with linear complexity. We follow the general scheme by eliminating the linear parameters first, which results in a least squares error function that only depends on the non-linear rotation and a small symmetric coefficient matrix of fixed size. Then, in a second step the rotation is solved with algorithms which are derived using methods from algebraic geometry such as the Gröbner basis method. We propose a unified formulation based on a representation with orthogonal complements which allows to combine different types of constraints elegantly in one single framework. We show that with our unified formulation existing polynomial solvers can be interchangeably applied to problem instances other than those they were originally proposed for. It becomes possible to compare them on various registrations problems with respect to accuracy, numerical stability, and computational speed. Our compression procedure not only preserves linear complexity, it is even faster than previous formulations. For the second step we also derive an own algebraic equation solver, which can additionally handle the registration from 3D point-to-point correspondences, where other rotation solvers fail. Finally, we also present a marker-based SLAM approach with automatic registration to a target coordinate system based on partial and distributed reference information. It represents an application example that goes beyond classical camera pose estimation from image measurements and also serves for evaluation on real data.

Show publication details

Piero, Nils; Schmitt, Michael

Virtual Commissioning of Camera-based Quality Assurance Systems for Mixed Model Assembly Lines


Procedia Manufacturing

International Conference on Flexible Automation and Intelligent Manufacturing (FAIM) <27, 2017, Modena, Italy>

We propose a new computer vision based technique that does not use data collected from real images to teach a decision-making algorithm. It uses CAD-Models which is already available from the product planning phase to check all product variants on the assembly line for misplaced or wrong components. This solution can hence be used already in the ramp-up phase of new models in which problems are more frequent. Our method can also easily adapt to changes of production, where existing methods need to redo the whole teach-in process, we just need to set the new nominal position of the part in the reference coordinate system, which can even be done semi automatically upfront the changes are initiated in the assembly line.

Show publication details

Fraissinet-Tachet, Matthieu; Schmitt, Michael; Kuijper, Arjan; Wen, Zhuoman

Multi-Camera Piecewise Planar Object Tracking with Mutual Information


Journal of Mathematical Imaging and Vision

Real-time and robust tracking of 3D objects based on a 3D model with multiple cameras is still an unsolved problem albeit relevant in many practical and industrial applications. Major problems are caused by appearance changes of the object. We present a template-based tracking algorithm for piecewise planar objects. It is robust against changes in the appearance of the object (occlusion, illumination variation, specularities). The version we propose supports multiple cameras. The method consists in minimizing the error between the observed images of the object and the warped images of the planes. We use the mutual information as registration function combined with an inverse composition approach for reducing the computational costs and get a near-real-time algorithm. We discuss different hypotheses that can be made for the optimization algorithm.

Show publication details

Engelke, Timo; Keil, Jens; Rojtberg, Pavel; Wientapper, Folker; Schmitt, Michael; Bockholt, Ulrich

Content First - A Concept for Industrial Augmented Reality Maintenance Applications using Mobile Devices


MMSys '15

ACM Multimedia Systems Conference (MMSys) <6, 2015, Portland, OR, USA>

Although AR has a long history in the area of maintenance and service-support in industry, there still is a lack of lightweight, yet practical solutions for handheld AR systems in everyday workflows. Attempts to support complex maintenance tasks with AR still miss reliable tracking techniques, simple ways to be integrated into existing maintenance environments, and practical authoring solutions, which minimize costs for specialized content generation. We present a general, customisable application framework, allowing to employ AR and VR techniques in order to support technicians in their daily tasks. In contrast to other systems, we do not aim to replace existing support systems such as traditional manuals. Instead we integrate well-known AR- and novel presentation techniques with existing instruction media. To this end practical authoring solutions are crucial and hence we present an application development system based on web-standards such as HTML,CSS and X3D.


Show publication details

Behr, Johannes; Mouton, Christophe; Parfouru, Samuel; Champeau, Julien; Jeulin, Clotilde; Thöner, Maik; Stein, Christian; Schmitt, Michael; Limper, Max; Sousa, Miguel de; Franke, Tobias; Voss, Gerrit

webVis/instant3DHub - Visual Computing as a Service Infrastructure to Deliver Adaptive, Secure and Scalable User Centric Data Visualisation


Proceedings Web3D 2015

International Conference on 3D Web Technology (WEB3D) <20, 2015, Heraklion, Crete, Greece>

This paper presents the webVis / instant3DHub platform, which combines a novel Web-Components based framework and a Visual Computing as a Service infrastructure to deliver an interactive 3D data visualisation solution. The system focuses on minimising resource consumption, while maximising the end-user experience. It utilises an adaptive and automated combination of client, server and hybrid visualisation techniques, while orchestrating transmission, caching and rendering services to deliver structural and semantically complex data sets on any device class and network architecture. The API and Web Component framework allow the application developer to compose and manipulate complex data setups with a simple set of commands inside the browser, without requiring knowledge about the underlying service infrastructure, interfaces and the fully automated processes. This results in a new class of interactive applications, built around a canvas for real-time visualisation of massive data sets.


Show publication details

Keil, Jens; Engelke, Timo; Schmitt, Michael; Bockholt, Ulrich; Pujol, Laia

Lean In or Lean Back? Aspects on Interactivity & Mediation in Handheld Augmented Reality in the Museum


GCH 2014. Short Papers - Posters

Eurographics Symposium on Graphics and Cultural Heritage (GCH) <12, 2014, Darmstadt, Germany>

With the idea to design augmented reality experiences that attract and inform, and which are also seamlessly incorporated into interactive museum narratives, this paper explores finding the appropriate balance between attraction, interactivity and information mediation from the user's point of view. Within the scope of research project CHESS, we've implemented techniques that fuse interaction and mediation to enrich visits to cultural institutions by visual means and AR specific interactions. While it is AR's wow-effect that attracts, our findings show that users far too often struggle to cope with the system and interactivity instead of focusing on the information presentation in AR. We discuss our results of finding the right balance between interactive (lean-in) and non-interactive (lean-back) presentation and interaction techniques in AR.

Show publication details

Fraissinet-Tachet, Matthieu; Kuijper, Arjan [Supervisor]; Schmitt, Michael [Supervisor]

Mutual Information-Based Piecewise Planar Object Tracking


Darmstadt, TU, Master Thesis, 2014

This master thesis deals with a template based tracking algorithm for piecewise planar objects. It is robust against changes in the appearance of the object (occlusion, illumination variation, specularities). The version that we propose supports multiple cameras. The method consists in minimizing the error between the observed images of the object and the warped images of the planes. We use for that an estimation of the pose of the object, which is to say a rigid 3D transformation. The robustness is obtained by using the mutual information as registration function. The main drawback of the mutual information is that it has a high computation complexity. We use an inverse composition approach for the warp update, so that pre-computations can be done and it decreases the complexity of the algorithm. We develop a way of computing the warp update and analyse the impact of this initiative on the optimization process. We also determine the optimal parameters for running the algorithms.

Show publication details

Keil, Jens; Pujol, Laia; Roussou, Maria; Engelke, Timo; Schmitt, Michael; Bockholt, Ulrich; Eleftheratou, Stamatia

A Digital Look at Physical Museum Exhibits


2013 Digital Heritage International Congress. Volume 2

Digital Heritage International Congress (DigitalHeritage) <2013, Marseille, France>

In this paper we present the design of handheld Augmented Reality (AR) experiences that are seamlessly incorporated into interactive museum narratives, specifically for the Acropolis Museum. The experiences start by forming a visitor profile that later dynamically adapts the narrative, including the AR activities, to the user's behaviour. In this cohesive narrative context, the AR activities provide four ways to digitally look at the exhibits: virtual reconstruction of the original aspect; placement in the original location; visual highlighting of interesting details and annotations; and recreation of mythological appearances. The challenges of this design are presented, concluding with a discussion and lessons learned.

Show publication details

Keil, Jens; Zöllner, Michael; Engelke, Timo; Wientapper, Folker; Schmitt, Michael

Controlling and Filtering Information Density with Spatial Interaction Techniques via Handheld Augmented Reality


Virtual Augmented and Mixed Reality. Designing and Developing Augmented and Virtual Environments. Proceedings Part I

International Conference Virtual Augmented and Mixed Reality (VAMR) <5, 2013, Las Vegas, NV, USA>

In our paper we are proposing a method for contextual information filtering based on the user's movement and location in order to enable the intuitive usage of an "internet of things" via augmented reality (AR) without information overload. Similar to Ray & Charles Eames' "Power of Ten" and Jef Raskin's "Zooming Interface" we are displaying seamless information layers by simply moving around a Greek statue or a miniature model of an Ariane-5 space rocket. Therefore we are employing concepts of camera- and motion-based interaction techniques and use the metaphors of "investigation" and "exploration" to control the way augmented and visually superimposed elements are presented in order to mediate information in an enhanced and engaging manner with aspects of digital storytelling techniques.

Show publication details

Schmitt, Michael; Engelke, Timo [Betreuer]; Goesele, Michael [Betreuer]

Estimation of Matching Probabilities for Histogrammed Intensity Patches


Darmstadt, TU, Master Thesis, 2012

This work presents a way to estimate the matching probabilities for histogrammed intensity patches [41] by approximating the robustness and distinctiveness of a feature with beta distributions. We propose a simplified analytical approach to calculate the distinctiveness and show that it produces at least as good results as a specifically trained neural network. The matching probability correlates stronger with the inlier ratio than the dissimilarity score does. We then evaluate a basic tracking by detection pipeline where no longer the closest but the most probable match is chosen. Our tests show, that this improves the average performance of the system in terms of tracking rate.

Show publication details

Schmitt, Michael; Kirschner, Matthias [Betreuer]

Evaluation statistischer Verfahren zur Modellierung von Formvariation


Darmstadt, TU, Bachelor Thesis, 2009

Statistische Modelle werden in vielen Bereichen der medizinischen Bildverarbeitung eingesetzt, aber auch zur Lösung von allgemeinen Aufgaben in Computer Vision und Computer Graphik, wie etwa bei der Gesichtserkennung, der Entrauschung von Bildern oder der Segmentierung. In der Theorie wurden mehrere statistische Methoden zur Verwendung bei den genannten Problemen entwickelt, die aber im praktischen Einsatz nur geringen Anklang finden. So wird im Kontext der statistischen Formmodelle größtenteils die Hauptkomponentenanalyse angewandt. In dieser Arbeit untersuche ich statistische Modelle, die im Rahmen des Active Shape Models zur Bildsegmentierung eingesetzt werden können. Das Ziel der Arbeit ist es, durch diese Untersuchung herauszufinden, worin die Diskrepanz zwischen Theorie und Praxis begründet ist. Ich suche also nach Ursachen dafür, dass zwar verschiedenste Technikenbekannt, in der Wirklichkeit aber nur die Hauptkomponentenanalyse eingesetzt wird. Dafür wurden zwei weitere wichtige Vertreter statistischer Modelle ausgewählt, welche im Folgenden vorgestellt und anhand ihrer Leistungsfähigkeit miteinander und mit der Hauptkomponentenanalyse verglichen werden.