• Vita
  • Publikationen
  • Vorlesungen
  • Forschungsthemen
Show publication details

Wuest, Harald; Engelke, Timo; Schmitt, Florian; Keil, Jens

From CAD to 3D Tracking - Enhancing & Scaling Model-Based Tracking for Industrial Appliances

2016

Mayol-Cuevas, Walterio (Ed.) et al.: 2016 IEEE International Symposium on Mixed and Augmented Reality : ISMAR 2016. Los Alamitos, Calif.: IEEE Computer Society, 2016, 2 p.

IEEE International Symposium on Mixed and Augmented Reality (ISMAR) <15, 2016, Merida, Mexico>

For Augmented Reality to succeed in industrial appliances, industries demand not only robust and reliable tracking techniques, but also a scalable and performant pipeline, that is easy-to-integrate within the existing data- and content environment and that enables vendors to create tracking solutions on their own. In our demo, we present recent advances of our model tracking pipeline and tracking technology, which on the one hand is easy use and easy to integrate, while on the other hand robust enough during difficult environmental conditions and which delivers high accuracy for the industrial domain. We showcase our results inside an AR-manual scenario.

Show publication details

Graf, Holger; Keil, Jens; Pagano, Alfonsina; Pescarin, Sofia

A Contextualized Educational Museum Experience: Connecting Objects, Places and Themes through mobile Virtual Museums

2015

Guidi, Gabriele (Ed.) et al.: 2015 Digital Heritage International Congress. Volume 1. New York: The Institute of Electrical and Electronics Engineers (IEEE), 2015, pp. 337-340

Digital Heritage International Congress (DH) <2015, Granada, Spain>

This paper presents the results of ongoing work realizing a mobile virtual museum that enables the connection between locally dispersed museums, collections and artefacts. The mobile application interconnects venues, objects and stories for an enhanced user experience and increased learnability inside the museum. It contextualizes information and reveals missing and hidden links between the digital replicas of the objects displayed in four different venues under one storyline. Within this context, we also provide a preliminary evaluation on how the mobile virtual museum could be used as support to indirect formal learning of young visitors, improving memorability.

Show publication details

Engelke, Timo; Keil, Jens; Rojtberg, Pavel; Wientapper, Folker; Schmitt, Michael; Bockholt, Ulrich

Content First - A concept for Industrial Augmented Reality Maintenance Applications using Mobile Devices

2015

ACM SIGMM: MMSys '15 : Proceedings of the 6th ACM Multimedia Systems Conference. New York: ACM, 2015, pp. 105-111

Multimedia Systems Conference (MMSys) <6, 2015, Portland, OR, USA>

Although AR has a long history in the area of maintenance and service-support in industry, there still is a lack of lightweight, yet practical solutions for handheld AR systems in everyday workflows. Attempts to support complex maintenance tasks with AR still miss reliable tracking techniques, simple ways to be integrated into existing maintenance environments, and practical authoring solutions, which minimize costs for specialized content generation. We present a general, customisable application framework, allowing to employ AR and VR techniques in order to support technicians in their daily tasks. In contrast to other systems, we do not aim to replace existing support systems such as traditional manuals. Instead we integrate well-known AR- and novel presentation techniques with existing instruction media. To this end practical authoring solutions are crucial and hence we present an application development system based on web-standards such as HTML,CSS and X3D.

Show publication details

Olbrich, Manuel; Keil, Jens; Makiela, Thomas

Heritage Move: A Natural & Lightweight Navigation Schema for Low-Cost, Non-Stationary Immersive Virtual Environments

2015

Guidi, Gabriele (Ed.) et al.: 2015 Digital Heritage International Congress. Volume 1. New York: The Institute of Electrical and Electronics Engineers (IEEE), 2015, pp. 289-292

Digital Heritage International Congress (DH) <2015, Granada, Spain>

Show publication details

Schaller, Andreas; Biedenkapp, Tim; Keil, Jens; Fellner, Dieter W.; Kuijper, Arjan

Immersive Interaction Paradigms for Controlling Virtual Worlds by Customer Devices Exemplified in a Virtual Planetarium

2015

Antona, Margherita (Ed.) et al.: Universal Access in Human-Computer Interaction. Proceedings Part IV : Access to the Human Environment and Culture. Springer International Publishing, 2015. (Lecture Notes in Computer Science (LNCS) 9178), pp. 74-86

International Conference on Universal Access in Human-Computer Interaction (UAHCI) <9, 2015, Los Angeles, CA, USA>

This work provides an insight into the basics of 3D applications in conjunction with various customer devices. In this case, the application is a 3D planetarium of our solar system for a museum. The aim is to create a concept for intuitive and immersive navigation through the virtual planetarium using inexpensive Customer Devices. Visitors should be able to move freely and easily in the solar system. Here, the visitor should be able to focus on the simulation and not quickly lose interest in the complex control application. For this similar approaches and previous research are examined and a new approach is described. As low-cost customer devices, the controller of the Nintendo Wii (Wiimote) and current smartphones are considered in this work. A detailed analysis of these devices is an integral part of this work. Based on the selected devices, there are various possibilities for interaction and resulting interaction concepts. For each device, a concept will be developed to meet the identified needs.

Show publication details

Vayanou, Maria; Katifori, Akrivi; Karvounis, Manos; Kourtis, Vassilis; Kyriakidi, Marialena; Roussou, Maria; Tsangaris, Manolis; Ioannidis, Yannis; Balet, Olivier; Prados, Thibaut; Keil, Jens; Engelke, Timo; Pujol, Laia

Authoring Personalized Interactive Museum Stories

2014

Mitchell, Alex (Ed.) et al.: Interactive Storytelling : 7th International Conference on Interactive Digital Storytelling. ICIDS 2014. Berlin, Heidelberg, New York: Springer, 2014. (Lecture Notes in Computer Science (LNCS) 8832), pp.37-48

Joint International Conference on Interactive Digital Storytelling (ICIDS) <7, 2014, Singapore>

CHESS is a research prototype system aimed at enriching museum visits through personalized interactive storytelling. Aspiring to replace traditional exhibit-centric descriptions by story-centric cohesive narrations with carefully-designed references to the exhibits, CHESS follows a plot-based approach, where the story authors create stories around pre-selected museum themes. In this paper we place the CHESS system within the Interactive Digital Narrative field, describing the main objectives and requirements addressed. We present the system's architecture and outline its overall functionality. We describe the underlying storytelling model using examples from the stories authored using the CHESS Authoring Tool. Finally, we report key results focusing on the authors' perspective for the creation of personalized stories.

Show publication details

Katifori, Akrivi; Karvounis, Manos; Kourtis, Vassilis; Kyriakidi, Marialena; Roussou, Maria; Tsangaris, Manolis; Vayanou, Maria; Ioannidis, Yannis; Balet, Olivier; Prados, Thibaut; Keil, Jens; Engelke, Timo; Pujol, Laia

CHESS: Personalized Storytelling Experiences in Museums

2014

Mitchell, Alex (Ed.) et al.: Interactive Storytelling : 7th International Conference on Interactive Digital Storytelling. ICIDS 2014. Berlin, Heidelberg, New York: Springer, 2014. (Lecture Notes in Computer Science (LNCS) 8832), pp. 232-235

Joint International Conference on Interactive Digital Storytelling (ICIDS) <7, 2014, Singapore>

In this work, we present the CHESS research prototype system which offers personalized, interactive digital storytelling experiences to enhance museum visits, demonstrating the authoring and visiting experiences.

Show publication details

Keil, Jens; Engelke, Timo; Schmitt, Michael; Bockholt, Ulrich; Pujol, Laia

Lean In or Lean Back? Aspects on Interactivity & Mediation in Handheld Augmented Reality in the Museum

2014

Klein, Reinhard (Ed.) et al.: GCH 2014. Short Papers - Posters : Eurographics Workshop on Graphics and Cultural Heritage. Goslar: Eurographics Association, 2014, pp. 17-20

Eurographics Symposium on Graphics and Cultural Heritage (GCH) <12, 2014, Darmstadt, Germany>

With the idea to design augmented reality experiences that attract and inform, and which are also seamlessly incorporated into interactive museum narratives, this paper explores finding the appropriate balance between attraction, interactivity and information mediation from the user's point of view. Within the scope of research project CHESS, we've implemented techniques that fuse interaction and mediation to enrich visits to cultural institutions by visual means and AR specific interactions. While it is AR's wow-effect that attracts, our findings show that users far too often struggle to cope with the system and interactivity instead of focusing on the information presentation in AR. We discuss our results of finding the right balance between interactive (lean-in) and non-interactive (lean-back) presentation and interaction techniques in AR.

Show publication details

Wientapper, Folker; Engelke, Timo; Keil, Jens; Wuest, Harald; Mensik, Johanna

User Friedly Calibration and Tracking for Optical Stereo See-Through Augmented Reality

2014

Julier, Simon (Ed.) et al.: IEEE International Symposium on Mixed and Augmented Reality - Science & Technology 2014 : ISMAR 2014. Piscataway, NJ: IEEE Service Center, 2014, pp. 385-386

IEEE International Symposium on Mixed and Augmented Reality (ISMAR) <13, 2014, Munich, Germany>

Optical see through head mounted displays (OST-HMD) are ever since the first days of Augmented Reality (AR) in focus of development and in nowadays first affordable and prototypes are spread out to markets. Despite common technical problems, such as having a proper field of view, weight, and other problems concerning the miniaturization of these systems, a crucial aspect for AR relies also in the calibration of such a device with respect to the individual user for proper alignment of augmentations. Our demonstrator shows a practical solution for this problem along with a fully featured example application for a typical maintenance use case based on a generalized framework for application creation. We depict the technical background and procedure of the calibration, the tracking approach considering the sensors of the device, user experience factors, and its implementation procedure in general. We present our demonstrator using an Epson Moverio BT-200 OST-HMD.

Show publication details

Keil, Jens; Pujol, Laia; Roussou, Maria; Engelke, Timo; Schmitt, Michael; Bockholt, Ulrich; Eleftheratou, Stamatia

A Digital Look at Physical Museum Exhibits: Designing Personalized Stories with Handheld Augmented Reality in Museums

2013

2013 Digital Heritage International Congress. Volume 2 : DigitalHeritage. New York: The Institute of Electrical and Electronics Engineers (IEEE), 2013, pp. 685-688

Digital Heritage International Congress (DigitalHeritage) <2013, Marseille, France>

In this paper we present the design of handheld Augmented Reality (AR) experiences that are seamlessly incorporated into interactive museum narratives, specifically for the Acropolis Museum. The experiences start by forming a visitor profile that later dynamically adapts the narrative, including the AR activities, to the user's behaviour. In this cohesive narrative context, the AR activities provide four ways to digitally look at the exhibits: virtual reconstruction of the original aspect; placement in the original location; visual highlighting of interesting details and annotations; and recreation of mythological appearances. The challenges of this design are presented, concluding with a discussion and lessons learned.

Show publication details

Olbrich, Manuel; Graf, Holger; Kahn, Svenja; Engelke, Timo; Keil, Jens; Rieß, Patrick; Webel, Sabine; Bockholt, Ulrich; Picinbono, Guillaume

Augmented Reality Supporting User-centric Building Information Management

2013

The Visual Computer, Vol.29 (2013), 10, pp. 1093-1105

The rapid development of geo-referenced information changed the way on how we access and interlink data. Smartphones as enabling devices for information access are main driving factor. Thus, the hash key to information is the actual position registered via camera and sensory of the mobile device. A rising technology in this context is Augmented Reality (AR) as its fuses the real world captured with the smartphone camera with geo-referenced data. The technological building blocks analyse the intrinsic sensor data (camera, GPS, inertial) to derive a detailed pose of the smartphone aiming to align geo-referenced information to our real environment. In particular, this is interesting to applications where 3D models are used in planning and organization processes as, e.g., facility management. Here, Building Information Models (BIM) were established in order to hold "as built" information, but also to manage the vast amount of additional information coming with the design, such as building components, properties, maintenance logs, documentation, etc. One challenge is to enable stakeholders involved in the overall building lifecycle to get mobile access to the management system within onsite inspections and to automatise feedback of newly generated information into the BIM. This paper describes a new AR framework that offers on-site access to BIM information and user centric annotation mechanism.

Show publication details

Kahn, Svenja; Keil, Jens; Müller, Benedikt; Bockholt, Ulrich; Fellner, Dieter W.

Capturing of Contemporary Dance for Preservation and Presentation of Choreographies in Online Scores

2013

2013 Digital Heritage International Congress. Volume 1 : DigitalHeritage. New York: The Institute of Electrical and Electronics Engineers (IEEE), 2013, pp. 273-280

Digital Heritage International Congress (DigitalHeritage) <2013, Marseille, France>

In this paper, we present a generic and affordable approach for an automatized and markerless capturing of movements in dance, which was developed in the Motion Bank / The Forsythe Company project (www.motionbank.org). Thereby within Motion Bank we are considering the complete digitalization workflow starting with the setup of the camera array and ending with a web-based presentation of "Online Scores" visualizing different elements of choreography. Within our project, we have used our technology in two modern dance projects, one "Large Motion Space Performance" covering a large stage in solos and trios and one "Restricted Motion Space Performance" that is suited to be captured with range cameras. The project is realized in close cooperation with different choreographers and dance companies of modern ballet and with multi-media artists forming the visual representations of dance.

Show publication details

Engelke, Timo; Keil, Jens; Rojtberg, Pavel; Wientapper, Folker; Webel, Sabine; Bockholt, Ulrich

Content First - A concept for Industrial Augmented Reality Maintenance Applications Using Mobile Devices

2013

IEEE Computer Society Visualization and Graphics Technical Committee (VGTC): 12th IEEE International Symposium on Mixed and Augmented Reality 2013. : ISMAR 2013. Los Alamitos, Calif.: IEEE Computer Society, 2013, pp. 251-252

IEEE International Symposium on Mixed and Augmented Reality (ISMAR) <12, 2013, Adelaide, SA, Australia>

Although AR has a long history in the area of maintenance and service-support in industry, there still is a lack of lightweight, yet practical solutions for handheld AR systems in everyday workflows. Attempts to support complex maintenance tasks with AR still miss reliable tracking techniques, simple ways to be integrated into existing maintenance environments, and practical authoring solutions, which minimize costs for specialized content generation. We present a general, customisable application framework, allowing to employ AR and VR techniques in order to support technicians in their daily tasks. In contrast to other systems, we do not aim to replace existing support systems such as traditional manuals. Instead we integrate well-known AR- and novel presentation techniques with existing instruction media. To this end practical authoring solutions are crucial and hence we present an application development system based on web-standards such as HTML,CSS and X3D.

Show publication details

Keil, Jens; Zöllner, Michael; Engelke, Timo; Wientapper, Folker; Schmitt, Michael

Controlling and Filtering Information Density with Spatial Interaction Techniques via Handheld Augmented Reality

2013

Shumaker, Randall (Ed.): Virtual Augmented and Mixed Reality. Designing and Developing Augmented and Virtual Environments. Proceedings Part I : VAMR 2013. Berlin, Heidelberg, New York: Springer, 2013. (Lecture Notes in Computer Science (LNCS) 8021), pp. 49-57

International Conference Virtual Augmented and Mixed Reality (VAMR) <5, 2013, Las Vegas, NV, USA>

In our paper we are proposing a method for contextual information filtering based on the user's movement and location in order to enable the intuitive usage of an "internet of things" via augmented reality (AR) without information overload. Similar to Ray & Charles Eames' "Power of Ten" and Jef Raskin's "Zooming Interface" we are displaying seamless information layers by simply moving around a Greek statue or a miniature model of an Ariane-5 space rocket. Therefore we are employing concepts of camera- and motion-based interaction techniques and use the metaphors of "investigation" and "exploration" to control the way augmented and visually superimposed elements are presented in order to mediate information in an enhanced and engaging manner with aspects of digital storytelling techniques.

Show publication details

Webel, Sabine; Olbrich, Manuel; Franke, Tobias; Keil, Jens

Immersive Experience of Current and Ancient Reconstructed Cultural Attractions

2013

2013 Digital Heritage International Congress. Volume 1 : DigitalHeritage. New York: The Institute of Electrical and Electronics Engineers (IEEE), 2013, pp. 395-398

Digital Heritage International Congress (DigitalHeritage) <2013, Marseille, France>

Manual and automatic reconstruction of cultural assets is well-established fields of research. Especially with the recent introduction of methods such as KinectFusion even low cost sensors can be employed to create digital copies of the environment. However, experiencing these 3D models is still an active issue considering the wide variety of interaction devices for immersive Virtual Reality. The balance between system mobility and freedom of interaction is still a challenge: either the user's view is totally covered through digital data, what complicates the intuitive interaction with the virtual world enormously, or projection-based installations are used which are usually stationary. In our paper we focus on creating a low-cost, fully immersive and non-stationary Virtual Reality setup that allows the user to intuitively experience cultural heritage artifacts. For this purpose we explore and analyze recent devices such as the Oculus Rift HMD, the Microsoft Kinect and the Leap Motion controller.

Show publication details

Keil, Jens; Engelke, Timo; Pujol, Laia

Interactive Adaptive Storytelling with AR at Acropolis Museum

2013

Pescarin, Sofia et al.: 2013 Digital Heritage International Congress. Volume 3 : E-Catalogue of the Digital Heritage 2013 Expo. Roma: CNR - ITABC edizioni, 2013, pp. 56-57

Digital Heritage International Congress (DigitalHeritage) <2013, Marseille, France>

In this paper we present the design of handheld Augmented Reality (AR) experiences that are seamlessly incorporated into interactive museum narratives, specifically for the Acropolis Museum. The experiences start by forming a visitor profile that later dynamically adapts the narrative, including the AR activities, to the user's behaviour. In this cohesive narrative context, the AR activities provide four ways to digitally look at the exhibits: virtual reconstruction of the original aspect; placement in the original location; visual highlighting of interesting details and annotations; and recreation of mythological appearances. The challenges of this design are presented, conluding with a discussion and lessons learned.

Show publication details

Engelke, Timo; Becker, Mario; Wuest, Harald; Keil, Jens; Kuijper, Arjan

MobileAR Browser - A Generic Architecture for Rapid AR-multi-level Development

2013

Expert Systems with Applications, Vol.40 (2013), 7, pp. 2704-2714

We present our novel generic approach for interfacing web components on mobile devices in order to rapidly develop Augmented Reality (AR) applications using HTML5, JavaScript, X3D and a vision engine. A general concept is presented exposing a generalized abstraction of components that are to be integrated in order to allow the creation of AR capable interfaces on widely available mobile devices. Requirements are given, yielding a set of abstractions, components, and helpful interfaces that allow rapid prototyping, research at application level, as well as commercial applications. A selection of various applications (also commercial) using the developed framework is given, proving the generality of the architecture of our MobileAR Browser. Using this concept a large number of developers can be reached. The system is designed to work with different standards and allows for domain separation of tracking algorithms, render content, interaction and GUI design. This can potentially help groups of developers and researchers with different competences creating their application in parallel, while the declarative content remains exchangeable.

Show publication details

Olbrich, Manuel; Franke, Tobias; Keil, Jens

Skeletal Input for User Interaction in X3D

2013

Posada, Jorge (General Chair) et al.: Proceedings Web3D 2013 : 18th International Conference on 3D Web Technology. New York: ACM Press, 2013, pp. 143-146

International Conference on 3D Web Technology (WEB3D) <18, 2013, San Sebastian, Spain>

Recent developments in depth sensor technology enable developers to use skeletal input in interactive 3D environments with high user fluctuation like museum exhibits. However, the question of how to use natural user input and body movement to control rich interactive and immersive environments remains and demands more intuitive solutions due to the new audience. In this paper, we document different forms of interaction that where designed to be used by untrained visitors and regard hand and body gestures form a user centric angle instead of a technological point-of- view. We thereby show how we cope with untrained user expectations and how we intuitively guide them to use the systems. In addition, we discuss how the results could be improved based on an informal evaluation and observations made on-site.

Show publication details

Engelke, Timo; Keil, Jens

WebApp Development for Enhanced Cultural Heritage Experience through mobile Augmented Reality

2013

2013 Digital Heritage International Congress. Volume 2 : DigitalHeritage. New York: The Institute of Electrical and Electronics Engineers (IEEE), 2013, p. 767

Digital Heritage International Congress (DigitalHeritage) <2013, Marseille, France>

The use of mobile devices in cultural heritage is still a new field of research, while the amount of potential applications grows from day to day. Due to the enormous increase of technological capabilities, mobile devices are suitable for performing Augmented Reality in cultural heritage applications. The mobile application framework "instantAR" for iOS and Android devices enables high-level development of Augmented Reality solutions. It provides an efficient environment for the development of AR applications on mobile devices based on common standards like HTML, Javascript and X3D. While providing convenient interfaces and abstractions it maintains great flexibility in order to also realise advanced interaction paradigms for AR. The full day tutorial will introduce into the concepts of the "instantAR" framework, its provided development environment, discuss the possibilities for deploying standards in AR-Systems, demonstrate its capabilities and present the full range of potentials for a deployment in research and commercial environments targeting diverse stakeholders in the cultural heritage domain.

Show publication details

Kahn, Svenja; Olbrich, Manuel; Engelke, Timo; Keil, Jens; Rieß, Patrick; Webel, Sabine; Graf, Holger; Bockholt, Ulrich; Picinbono, Guillaume

Beyond 3D "as-Built" Information using Mobile AR Enhancing the Building Lifecycle Management

2012

Kuijper, Arjan (Ed.) et al.: 2012 International Conference on Cyberworlds. Proceedings : Cyberworlds 2012. Los Alamitos, Calif.: IEEE Computer Society Conference Publishing Services (CPS), 2012, pp. 29-36

International Conference on Cyberworlds (CW) <11, 2012, Darmstadt, Germany>

The success of smartphone technologies changed the way information is processed as more and more geo-referenced structures are emerging linking information to specific locations within our environment. Thereby, Augmented Reality has become a key technology as it analyses the smartphone sensor data (camera, GPS, inertial) to derive the detailed pose of the smartphone, with the aim to correlate our real environment to the geo-referenced information space. In particular, this is relevant for application fields where 3Dmodels are used in planning and organization processes as e.g. facility management. In facility construction and management Building Information Model (BIM) was established as a standard that not only holds the 3D-building-geometry but encompasses pipe/electrical systems as well as semantic building information, for example properties and conditions of building components. With this motivation our work integrates BIM and Augmented Reality.

Show publication details

Kahn, Svenja; Keil, Jens; Zöllner, Michael; Müller, Benedikt

Towards an Affordable Markerless Acquisition of Intangible Contemporary Dance Choreographies at Large-Scaled Stages

2012

Arnold, David (Ed.) et al.: VAST 2012. Short and Project Papers. Goslar: Eurographics Association, 2012, pp. 33-36

International Symposium on Virtual Reality, Archaeology and Cultural Heritage (VAST) <13, 2012, Brighton, UK>

While the documentation and preservation of rigid cultural heritage objects has become much easier with technologies such as 3D scanning or photogrammetry technologies, the digitalization of 3D intangible moving content is still a major issue. This concerns also the in situ creation of digital dance representations and the question of how to preserve and disseminate dance performances. In this paper, we present a generic and affordable approach for an automatized and markerless capturing of the movements of dancers, which was developed in the Motion Bank research project, as well as first application examples which analyse and visualize the captured dance data. The captured data is stored in a cloud based service and is thus made available for online and offline processing.

Show publication details

Jung, Yvonne; Keil, Jens; Behr, Johannes

X3DOM - Declarative (X)3D in HTML5: Demo

2012

Behr, Johannes (Ed.) et al.: Dec3D : Declarative 3D for the Web Architecture. [cited 19 August 2012] Available from: http://ceur-ws.org/Vol-869/, 2012. (CEUR Workshop Proceedings 869), 2 p.

International Workshop on Declarative 3D for the Web Architecture (Dec3D) <1, 2012, Lyon, France>

In this demo description we present X3DOM, which is an open source framework and runtime system to support the ongoing discussion in the Web3D and W3C communities how an integration of HTML5 and declarative 3D graphics can look like by including X3D elements as part of the HTML5 DOM tree. The goal here is to have a live X3D scene-graph integrated into the HTML DOM, which allows manipulating the 3D content by only adding, removing, or changing the corresponding DOM elements. No specific plugin or plugin interface, such as the X3D-specific SAI, is required. X3DOM also supports CSS integration as well as standard HTML events like "onclick" on 3D objects.

Show publication details

Keil, Jens; Bockholt, Ulrich (Betreuer)

Conveying Information for Cultural Heritage with Mobile Augmented Reality: Concepts for Mixing Realities on Smartphones

2011

Darmstadt, Hochschule, Master Thesis, 2011

Augmented Reality (AR) technology is stepping into mass markets and everyday usage. While the technology evolves from lightweight to sophisticated technical approaches running on mobile devices, this thesis questions and explores how to perceive information with AR and how to interact with reality on mobile devices, mediated by it. The thesis analyzes recent smartphone-based approaches of video-see-through AR and proposes a categorization that shapes AR's nature. Thereby it shows how AR can sustainably add value for conveying information. It gives an overview of information mediation and interaction techniques that are suitable for mobile AR on smartphones and recent tablets in the context of cultural heritage. Examples and small experiments give an impression of the state-of-the-art and discuss what might still be missing. A practical exploration and demonstration is given in the context of Fraunhofer IGD's Olbrich project. Here, computer vision-based AR is employed outdoors in order to illustrate the history of House of Olbrich - one of Darmstadt's sights in the famous Artists' Colony. By the video see-through effect users can jump back in time visually with a smartphone: AR enriches a user's view on reality with a reconstruction based on historical blueprints. It reveals the building's original appearance in a sketch?like manner and communicates architectural characteristics in a reduced visual fashion. The presented AR techniques overcome current lightweight-AR and demonstrate how AR can add value to users through compelling visual depictions and how it extends user experience through enhanced interaction.

Show publication details

Webel, Sabine; Bockholt, Ulrich; Keil, Jens

Design Criteria for AR-Based Training of Maintenance and Assembly Tasks

2011

Shumaker, Randall (Ed.): Virtual and Mixed Reality - New Trends: Part I : International Conference, Virtual and Mixed Reality 2011. Berlin, Heidelberg, New York: Springer, 2011. (Lecture Notes in Computer Science (LNCS) 6773), pp. 123-132

International Conference on Virtual and Mixed Reality (VMR) <4, 2011, Orlando, FL, USA>

As the complexity of maintenance tasks can be enormous, the efficient training of technicians in performing those tasks becomes increasingly important. Maintenance training is a classical application field of Augmented Reality explored by different research groups. Mostly technical aspects (e.g tracking, 3D augmentations) have been in focus of this research field. In our paper we present results of interdisciplinary research based on the fusion of cognitive science, psychology and computer science. We focus on analyzing the improvement of AR-based training of maintenance skills by addressing also the necessary cognitive skills. Our aim is to find criteria for the design of AR-based maintenance training systems. A preliminary evaluation of the proposed design strategies has been conducted by expert trainers from industry.

Show publication details

Schaller, Andreas; Biedenkapp, Tim; Keil, Jens (Betreuer); Kuijper, Arjan (Betreuer)

Immersive Interaktionsparadigmen zur Steuerung virtueller Welten durch Customer Devices

2011

Darmstadt, TU, Bachelor Thesis, 2011

Diese Arbeit gibt einen Einblick in die Grundlagen der 3D-Anwendungen im Zusammenspiel mit verschiedenen Customer Devices. In diesem Fall ist die Anwendung ein 3D-Planetarium unseres Sonnensystems für ein Museum. Ziel ist es, ein Konzept zur intuitiven und immersiven Navigation durch das virtuelle Planetarium mittels preiswerte Customer Devices zu erstellen. Die Besucher sollen sich frei und ohne großen Aufwand im Sonnensystem bewegen können. Dabei soll sich der Besucher auf die Simulation konzentrieren können und nicht durch komplexe Steuerung schnell das Interesse an der Anwendung verlieren. Dafür werden vergleichbare Ansätze und der bisherige Forschungsstand untersucht und ein neuer Ansatz beschrieben. Als kostengünstige Customer Devices werden in dieser Arbeit die Controller der Nintendo Wii (Wiimote) und aktuelle Smartphones betrachtet. Eine genaue Analyse dieser Geräte ist Bestandteil dieser Arbeit. Ausgehend von den gewählten Geräten gibt es verschiedene Interaktionsmöglichkeiten und daraus resultierende Interaktionskonzepte. Für jedes Gerät soll ein Konzept entwickelt werden, das die gestellten Ansprüche erfüllt.

Show publication details

Keil, Jens; Zöllner, Michael; Becker, Mario; Wientapper, Folker; Engelke, Timo; Wuest, Harald

The House of Olbrich - An Augmented Reality Tour through Architectural History

2011

IEEE Computer Society Visualization and Graphics Technical Committee (VGTC): 10th IEEE International Symposium on Mixed and Augmented Reality - Arts, Media, and Humanities : ISMAR-AMH 2011. The Institute of Electrical and Electronics Engineers (IEEE), 2011, pp. 15-18

IEEE International Symposium on Mixed and Augmented Reality (ISMAR) <10, 2011, Basel, Switzerland>

With "House of Olbrich" we present an iPhone Augmented Reality (AR) app that visualizes the compelling history of Darmstadt's unique Jugendstil (Art Nouveau) quarter with video-see through Augmented Reality. We propose methods for enabling high performance computer vision algorithms to deploy sophisticated AR visuals on current generation Smartphones by outsourcing resource intensive tasks to the cloud. This allows us to apply methods on 3D feature recognition for even complex tracking situations outdoors, where lightning conditions change and tracked objects are often occluded. By taking a snapshot of the building, the user learns about the architect, design and history of the building. Historical media, like old photographs and blueprints are superimposed on the building's front, depicting the moved history of the famous House of Olbrich, which was destroyed during World War II and has been only rudimentary restored. Augmented Reality technology allows tourists to jump back in time visually by using their Smartphones: Mixing Realities emphasizes the user's experience and leads his attention to the impressive historical architecture of the Art Nouveau. In addition, we ease interaction means by superimposing snapshots. Tourists may view and read information also in a relaxed position without the need to front-up their mobiles all the time.

Show publication details

Behr, Johannes; Jung, Yvonne; Keil, Jens; Drevensek, Timm; Zöllner, Michael; Eschler, Peter; Fellner, Dieter W.

A Scalable Architecture for the HTML5 / X3D Integration Model X3DOM

2010

ACM SIGGRAPH: Proceedings Web3D 2010 : 15th International Conference on 3D Web Technology. New York: ACM Press, 2010, pp. 185-193

International Conference on 3D Web Technology (WEB3D) <15, 2010, Los Angeles, USA>

We present a scalable architecture, which implements and further evolve the HTML/X3D integration model X3DOM introduced in [Behr et al. 2009]. The goal of this model is to integrate and update declarative X3D content directly in the HTML DOM tree. The model was previously presented in a very abstract and generic way by only suggesting implementation strategies. The available opensource x3dom.js architecture provides concrete solutions to the previously open points and extents the generic model if necessary. The outstanding feature of the architecture is to provide a single declarative interface to application developers and at the same time support of various backends through a powerful fallback-model. This fallback-model does not provide a single implementation strategy for the runtime and rendering module but supports different methods transparently. This includes native browser implementations and X3D-plugins as well as a WebGL-based scene-graph, which allows running the content without the need for installing additional plugins on all browsers that support WebGL. The paper furthermore discusses generic aspects of the architecture like encoding and introspection, but also provides details concerning two backends. It shows how the system interfaces with X3D-plugins and WebGL and also discusses implementation specific features and limitations.

Show publication details

Zöllner, Michael; Becker, Mario; Keil, Jens

Snapshot Augmented Reality - Augmented Photography

2010

Artusi, Alessandro (Ed.) et al.: VAST 2010. Short and Project Papers Proceedings. Goslar: Eurographics Association, 2010, pp. 53-56

International Symposium on Virtual Reality, Archaeology and Cultural Heritage (VAST) <11, 2010, Paris, France>

With the current generation of smartphones augmented reality (AR) finally gets in the hands of end users. This is a giant leap for cultural heritage presentation. But due to software and hardware limitations of consumer devices the AR experience is still lacking the quality we have seen in research projects over the last years. In this paper we are proposing a scalable method for high quality AR presentations for cultural heritage on a wide range of consumer devices: Snapshot Augmented Reality. Instead of a live video stream superimposed with jittering annotations we are "freezing" the scene and enabling Augmented Reality Photography. The result is an interactive scene superimposed on a still image taken by a visitor. In order to outsource processing power and deliver content for a wide range of smartphones most of the sophisticated software works in the cloud. We are describing a reliable and scalable server infrastructure for tracking objects and environments and delivering context aware content to the visitors' devices.

Show publication details

Zöllner, Michael; Keil, Jens; Wuest, Harald; Pletinckx, Daniël

An Augmented Reality Presentation System for Remote Cultural Heritage Sites

2009

Debattista, Kurt (Ed.) et al.: VAST 2009. VAST-STAR, Short and Project Papers Proceedings. Msida: University of Malta, 2009, pp. 112-116

International Symposium on Virtual Reality, Archaeology and Cultural Heritage (VAST) <10, 2009, St. Julians, Malta>

Museums often lack the possibilitiy to present archaeological or cultural heritage sites in a realistic and interesting way. Thus are proposing a new way to show augmented reality applications of cultural heritage sites at remote places like museums. In the exhibition space large wall-filling photographs of the real site are superimposed with interactive contextual annotations like 3D reconstructions, images and movies. Therefore we are using two different hardware setups for visualization: Standard UMPCs and a custom made revolving display. The setup has been installed and tested at SIGGRAPH 2008, Allard Pierson Museum in Amsterdam and CeBIT 2009. Museum visitors could experience Forum Romanum and Saticum in an informative and intuitive way by pointing the video see through devices on different areas of the photographs. The result is a more realistic and entertaining way for presenting cultural heritage sites in museums. Furthermore our solution is less expensive than comparable installations regarding content and hardware.

Show publication details

Zöllner, Michael; Keil, Jens; Drevensek, Timm; Wuest, Harald

Cultural Heritage Layers: Integrating Historic Media in Augmented Reality

2009

Sablatnig, Robert (Ed.) et al.: 15th International Conference on Virtual Systems and Multimedia. Proceedings : VSMM 2009. Los Alamitos, Calif.: IEEE Computer Society, 2009, pp. 193-196

International Conference on Virtual Systems and MultiMedia (VSMM) <15, 2009, Vienna, Austria>

In this paper we are presenting Cultural Heritage Layers, an approach that enables the visualization of historic media like drawings, paintings and photographs of buildings and historic scenes seamlessly superimposed on reality via video see through using X3D. This enables simple, inexpensive and sustainable Augmented Reality applications in the cultural heritage and architectural area based on industry standards. The main idea is to use existing historic media from archives and superimpose them seamlessly on reality at the right spot. These locative layers are context sensitively telling the location's history and create the impression of a virtual time journey. The registration of the virtual objects in the video images is provided by a robust 6DOF tracking framework based on two technologies that work in tandem: an initialization step based on Randomized Trees and a frame-to-frame tracking phase based on KLT. The entire application runs in real time on current Ultra Mobile PCs and MIDs.

Show publication details

Jung, Yvonne; Keil, Jens; Wuest, Harald; Engelke, Timo; Rieß, Patrick; Behr, Johannes

Knowledge at Your Fingertips: Multi-touch Interaction for GIS and Architectural Design Review Applications

2009

Institute for Systems and Technologies of Information, Control and Communication (INSTICC): VISIGRAPP 2009. Proceedings : International Joint Conference on Computer Vision and Computer Graphics Theory and Applications [CD-ROM]. INSTICC Press, 2009, GRAPP, pp. 387-392

International Conference on Computer Graphics Theory and Applications (GRAPP) <4, 2009, Lisboa, Portugal>

This paper introduces novel techniques of interacting and controlling 3D content using multi-touch interaction principles for navigation and virtual camera control. Based on applications from GIS and for the architectural design review process, implementation and usage of these interaction techniques are illustrated. A comprehensive hardware and software setup is used, which not only includes tracking, but also an X3D based layer to simplify application development. Therefore it allows designers and other non-programmers to develop multi-touch applications very efficiently, while allowing to focus on user interaction and content.

Show publication details

Jung, Yvonne; Weber, Christine; Keil, Jens; Franke, Tobias

Real-Time Rendering of Skin Changes Caused by Emotions

2009

Ruttkay, Zsófia (Ed.) et al.: Intelligent Virtual Agents : 9th International Conference, IVA 2009. Berlin; Heidelberg; New York: Springer, 2009. (Lecture Notes in Artificial Intelligence (LNAI) 5773), pp. 504-505

International Conference on Intelligent Virtual Agents (IVA) <9, 2009, Amsterdam, The Netherlands>

In this paper a framework for dynamically parameterizing emotional skin changes like blushing and pallor is outlined. The parameterization is based on physiological and psychological knowledge. Furthermore, an on-surface droplet flow simulation was integrated for simulating a weeping character. The skin color changes and the droplet flow simulation can operate in real-time and thus provide an animation, which is consistent with the facial expression of the virtual character.

Show publication details

Jung, Yvonne; Keil, Jens; Behr, Johannes; Webel, Sabine; Zöllner, Michael; Engelke, Timo; Wuest, Harald; Becker, Mario

Adapting X3D for Multi-touch Environments

2008

Spencer, Stephen N. (Ed.): Proceedings WEB3D 2008 : 13th International Symposium on 3D Web Technology. New York: ACM Press, 2008, pp. 27-30

International Conference on 3D Web Technology (WEB3D) <13, 2008, Los Angeles, CA, USA>

Multi-touch interaction on tabletop displays is a very active field of todays HCI research. However, most publications still focus on tracking techniques or develop a gesture configuration for a specific application setup. Very few explore generic high level interfaces for multi-touch applications. In this paper we present a comprehensive hardware and software setup, which includes an X3D based layer to simplify the application development process. We present a robust FTIR based optical tracking system, examine in how far current sensor and navigation abstractions in the X3D standard are useful and finally present extensions to the standard, which enable designers and other non-programmers to develop multi-touch applications very efficiently.

Show publication details

Zöllner, Michael; Keil, Jens; Behr, Johannes; Gillich, Jan; Gläser, Sebastian; Schöls, Erich

Coperion 3D - A Virtual Factory on the Tabletop

2008

5th Intuition 2008. Proceedings : Virtual Reality in Industry and Society: From Research to Application. Turin, 2008, 7 p.

Intuition International Conference <5, 2008, Turin, Italy>

This paper describes the development of a virtual factory application based on multi-touch interaction, high resolution projection technology and industry standards like X3D. The application is a presentation tool to describe hidden complex physical processes inside a plant for bulk material handling. We are covering the project objectives, the scene architecture and the CAD and physical simulation production pipelines.

Show publication details

Hodgson, Benjamin; Megliola, Maurizio; Toffolo, Silvana; Zöllner, Michael; Keil, Jens

Encouraging Cultural Exploration: A Dynamic Tour Guide Based on Personalised Contextual Information and Ad-hoc Planning

2008

Cunningham, Paul (Ed.) et al.: Collaboration and the Knowledge Economy: Issues, Applications, Case Studies. Part 2. Amsterdam; Berlin: IOS Press, 2008. (Information and Communication Technologies and the Knowledge Economy 5), pp. 1331-1338

eChallenges Conference <18, 2008, Stockholm, Sweden>

Because of the difficulties in navigating public transport networks and finding out about available activities, cultural tourists tend to follow a pre-planned itinerary and to visit a fairly small set of well known attractions. An analogous situation emerges at the cultural heritage sites themselves: while cultural heritage sites typically have a large amount of archive material, only a small amount of information can be presented, therefore visitors tend not to explore the culturalsignificance of exhibits in depth. Our solution represents a complete mobile information system for travellers intended to encourage exploration. The system consists of three modules: firstly location-based information intended to enhance the cultural tourist's understanding and to suggest exploratory activities based on modelling the users interests; secondly ad-hoc itinerary planning functionality to give the user the confidence to explore and a novel augmented reality system for presenting information in a user friendly way using markerless tracking. This paper covers the research and development aspects of iTACITUS, a sixth framework programme project, reporting on both technical and coordination issues, as well as interaction issues encountered during the project trials.

Show publication details

Webel, Sabine; Keil, Jens; Zöllner, Michael

Multi-touch Gestural Interaction in X3D Using Hidden Markov Models

2008

ACM SIGCHI: Proceedings VRST 2008 : ACM Symposium on Virtual Reality Software and Technology. New York: ACM, 2008, pp. 263-264

ACM Symposium on Virtual Reality Software and Technology (VRST) <2008, Bordeaux, France>

Multi-touch interaction on tabletop displays is a very active field of todays HCI research. However, most publications still focus on tracking techniques or develop gesture configurations for a specific application setup using a small set of simple gestures. In this work we present a new approach to easily set up the recognition of even complex gestures for multi-touch applications. Our gesture recognition module is based on Hidden Markov Models, which offer a robust recognition of multiple gestures in real-time. An X3D interface of the recognition module is provided to qualify designers and other non-programmers to apply gesture recognition functionalities to multi-touch applications in an easy and straightforward manner.