Multi-Camera Piecewise Planar Object Tracking with Mutual Information
Journal of Mathematical Imaging and Vision, Vol.56 (2016), 3, pp. 591-602
Real-time and robust tracking of 3D objects based on a 3D model with multiple cameras is still an unsolved problem albeit relevant in many practical and industrial applications. Major problems are caused by appearance changes of the object. We present a template-based tracking algorithm for piecewise planar objects. It is robust against changes in the appearance of the object (occlusion, illumination variation, specularities). The version we propose supports multiple cameras. The method consists in minimizing the error between the observed images of the object and the warped images of the planes. We use the mutual information as registration function combined with an inverse composition approach for reducing the computational costs and get a near-real-time algorithm. We discuss different hypotheses that can be made for the optimization algorithm.
Content First - A concept for Industrial Augmented Reality Maintenance Applications using Mobile Devices
ACM SIGMM: MMSys '15 : Proceedings of the 6th ACM Multimedia Systems Conference. New York: ACM, 2015, pp. 105-111
Multimedia Systems Conference (MMSys) <6, 2015, Portland, OR, USA>
Although AR has a long history in the area of maintenance and service-support in industry, there still is a lack of lightweight, yet practical solutions for handheld AR systems in everyday workflows. Attempts to support complex maintenance tasks with AR still miss reliable tracking techniques, simple ways to be integrated into existing maintenance environments, and practical authoring solutions, which minimize costs for specialized content generation. We present a general, customisable application framework, allowing to employ AR and VR techniques in order to support technicians in their daily tasks. In contrast to other systems, we do not aim to replace existing support systems such as traditional manuals. Instead we integrate well-known AR- and novel presentation techniques with existing instruction media. To this end practical authoring solutions are crucial and hence we present an application development system based on web-standards such as HTML,CSS and X3D.
webVis/instant3DHub - Visual Computing as a Service Infrastructure to Deliver Adaptive, Secure and Scalable User Centric Data Visualisation
ACM SIGGRAPH: Proceedings Web3D 2015 : 20th International Conference on 3D Web Technology. New York: ACM, 2015, pp. 39-47
International Conference on 3D Web Technology (WEB3D) <20, 2015, Heraklion, Crete, Greece>
This paper presents the webVis / instant3DHub platform, which combines a novel Web-Components based framework and a Visual Computing as a Service infrastructure to deliver an interactive 3D data visualisation solution. The system focuses on minimising resource consumption, while maximising the end-user experience. It utilises an adaptive and automated combination of client, server and hybrid visualisation techniques, while orchestrating transmission, caching and rendering services to deliver structural and semantically complex data sets on any device class and network architecture. The API and Web Component framework allow the application developer to compose and manipulate complex data setups with a simple set of commands inside the browser, without requiring knowledge about the underlying service infrastructure, interfaces and the fully automated processes. This results in a new class of interactive applications, built around a canvas for real-time visualisation of massive data sets.
Lean In or Lean Back? Aspects on Interactivity & Mediation in Handheld Augmented Reality in the Museum
Klein, Reinhard (Ed.) et al.: GCH 2014. Short Papers - Posters : Eurographics Workshop on Graphics and Cultural Heritage. Goslar: Eurographics Association, 2014, pp. 17-20
Eurographics Symposium on Graphics and Cultural Heritage (GCH) <12, 2014, Darmstadt, Germany>
With the idea to design augmented reality experiences that attract and inform, and which are also seamlessly incorporated into interactive museum narratives, this paper explores finding the appropriate balance between attraction, interactivity and information mediation from the user's point of view. Within the scope of research project CHESS, we've implemented techniques that fuse interaction and mediation to enrich visits to cultural institutions by visual means and AR specific interactions. While it is AR's wow-effect that attracts, our findings show that users far too often struggle to cope with the system and interactivity instead of focusing on the information presentation in AR. We discuss our results of finding the right balance between interactive (lean-in) and non-interactive (lean-back) presentation and interaction techniques in AR.
Mutual Information-Based Piecewise Planar Object Tracking
Darmstadt, TU, Master Thesis, 2014
This master thesis deals with a template based tracking algorithm for piecewise planar objects. It is robust against changes in the appearance of the object (occlusion, illumination variation, specularities). The version that we propose supports multiple cameras. The method consists in minimizing the error between the observed images of the object and the warped images of the planes. We use for that an estimation of the pose of the object, which is to say a rigid 3D transformation. The robustness is obtained by using the mutual information as registration function. The main drawback of the mutual information is that it has a high computation complexity. We use an inverse composition approach for the warp update, so that pre-computations can be done and it decreases the complexity of the algorithm. We develop a way of computing the warp update and analyse the impact of this initiative on the optimization process. We also determine the optimal parameters for running the algorithms.
A Digital Look at Physical Museum Exhibits: Designing Personalized Stories with Handheld Augmented Reality in Museums
2013 Digital Heritage International Congress. Volume 2 : DigitalHeritage. New York: The Institute of Electrical and Electronics Engineers (IEEE), 2013, pp. 685-688
Digital Heritage International Congress (DigitalHeritage) <2013, Marseille, France>
In this paper we present the design of handheld Augmented Reality (AR) experiences that are seamlessly incorporated into interactive museum narratives, specifically for the Acropolis Museum. The experiences start by forming a visitor profile that later dynamically adapts the narrative, including the AR activities, to the user's behaviour. In this cohesive narrative context, the AR activities provide four ways to digitally look at the exhibits: virtual reconstruction of the original aspect; placement in the original location; visual highlighting of interesting details and annotations; and recreation of mythological appearances. The challenges of this design are presented, concluding with a discussion and lessons learned.
Controlling and Filtering Information Density with Spatial Interaction Techniques via Handheld Augmented Reality
Shumaker, Randall (Ed.): Virtual Augmented and Mixed Reality. Designing and Developing Augmented and Virtual Environments. Proceedings Part I : VAMR 2013. Berlin, Heidelberg, New York: Springer, 2013. (Lecture Notes in Computer Science (LNCS) 8021), pp. 49-57
International Conference Virtual Augmented and Mixed Reality (VAMR) <5, 2013, Las Vegas, NV, USA>
In our paper we are proposing a method for contextual information filtering based on the user's movement and location in order to enable the intuitive usage of an "internet of things" via augmented reality (AR) without information overload. Similar to Ray & Charles Eames' "Power of Ten" and Jef Raskin's "Zooming Interface" we are displaying seamless information layers by simply moving around a Greek statue or a miniature model of an Ariane-5 space rocket. Therefore we are employing concepts of camera- and motion-based interaction techniques and use the metaphors of "investigation" and "exploration" to control the way augmented and visually superimposed elements are presented in order to mediate information in an enhanced and engaging manner with aspects of digital storytelling techniques.
Estimation of Matching Probabilities for Histogrammed Intensity Patches
Darmstadt, TU, Master Thesis, 2012
This work presents a way to estimate the matching probabilities for histogrammed intensity patches  by approximating the robustness and distinctiveness of a feature with beta distributions. We propose a simplified analytical approach to calculate the distinctiveness and show that it produces at least as good results as a specifically trained neural network. The matching probability correlates stronger with the inlier ratio than the dissimilarity score does. We then evaluate a basic tracking by detection pipeline where no longer the closest but the most probable match is chosen. Our tests show, that this improves the average performance of the system in terms of tracking rate.
Evaluation statistischer Verfahren zur Modellierung von Formvariation
Darmstadt, TU, Bachelor Thesis, 2009
Statistische Modelle werden in vielen Bereichen der medizinischen Bildverarbeitung eingesetzt, aber auch zur Lösung von allgemeinen Aufgaben in Computer Vision und Computer Graphik, wie etwa bei der Gesichtserkennung, der Entrauschung von Bildern oder der Segmentierung. In der Theorie wurden mehrere statistische Methoden zur Verwendung bei den genannten Problemen entwickelt, die aber im praktischen Einsatz nur geringen Anklang finden. So wird im Kontext der statistischen Formmodelle größtenteils die Hauptkomponentenanalyse angewandt. In dieser Arbeit untersuche ich statistische Modelle, die im Rahmen des Active Shape Models zur Bildsegmentierung eingesetzt werden können. Das Ziel der Arbeit ist es, durch diese Untersuchung herauszufinden, worin die Diskrepanz zwischen Theorie und Praxis begründet ist. Ich suche also nach Ursachen dafür, dass zwar verschiedenste Technikenbekannt, in der Wirklichkeit aber nur die Hauptkomponentenanalyse eingesetzt wird. Dafür wurden zwei weitere wichtige Vertreter statistischer Modelle ausgewählt, welche im Folgenden vorgestellt und anhand ihrer Leistungsfähigkeit miteinander und mit der Hauptkomponentenanalyse verglichen werden.