• Positionsbeschreibung
  • Publikationen
  • Projekte
  • Forschungsthemen
Show publication details

Behr, Johannes; Blach, Roland; Bockholt, Ulrich; Harth, Andreas; Hoffmann, Hilko; Huber, Manuel; Käfer, Tobias; Keppmann, Felix Leif; Pankratz, Frieder; Rubinstein, Dmitri; Schubotz, René; Vogelgesang, Christian; Voss, Gerrit

ARVIDA-Referenzarchitektur: Ressourcen-orientierte Architekturen für die Anwendungsentwicklung Virtueller Techniken

2017

Schreiber, Werner (Ed.) et al.: Web-basierte Anwendungen Virtueller Techniken: Das ARVIDA-Projekt - Dienste-basierte Software-Architektur und Anwendungsszenarien für die Industrie. Berlin: Springer Vieweg, 2017, pp. 117-191

Die ARVIDA-Referenzarchitektur ist ein zentrales Element und Ergebnis des ARVIDA- Projektes. Sie ermöglicht es, mit etablierten Technologien und Konzepten aus dem Web-Umfeld heterogene VT-Systemlandschaften in integrierten, sehr weitgehend plattformunabhängigen VT-Anwendungen effizient zu nutzen. Die Referenzarchitektur nutzt und adaptiert das Prinzip der RESTful-Web-Services sowie die darauf aufbauenden Linked-Data Konzepte, um interoperable, leicht erweiterbare und modulare VTAnwendungen zu bauen. Die nachfolgenden Abschnitte beschreiben die Grundprinzipien und spezifischen Erweiterungen im Detail.

Show publication details

Alvarado, Pablo; Bockholt, Ulrich; Canzler, Ulrich; Herbort, Steffen; Heuser, Nicolas; Keitler, Peter; Krzikalla, Roland; Olbrich, Manuel; Prager, André; Schröder, Frank; Schwerdt, Jörg; Willneff, Jochen; Zürl, Konrad

ARVIDA-Technologien

2017

Schreiber, Werner (Ed.) et al.: Web-basierte Anwendungen Virtueller Techniken: Das ARVIDA-Projekt - Dienste-basierte Software-Architektur und Anwendungsszenarien für die Industrie. Berlin: Springer Vieweg, 2017, pp. 193-217

Dieses Kapitel beschreibt Technologien im Kontext von ARVIDA, die über die allgemeine Beschreibung von Technologien aus Kap. 2 hinausgehen. Eine der Hürden für den produktiven Einsatz von VT ist der Mangel an robusten, markerlosen Trackingsystemen. Hier wurden im Rahmen des Projektes essentielle Fortschritte gemacht. Auch bei der Gestenerkennung konnten im Rahmen der Interaktion in einer Sitzkiste wesentliche Verbesserungen erzielt werden. Die Vermessung von Geodaten ist eine Grundvoraussetzung für Anwendungen im Digitalen Fahrzeugerlebnis. Hier wurden ebenfalls deutliche Fortschritte erzielt. Schließlich ist zu erwähnen, dass die vorgestellten Technologien als Dienste der Referenzarchitektur bereitgestellt werden, um den Austausch von Technologien einfach zu gestalten.

Show publication details

Bär, Thomas; Bockholt, Ulrich; Hoffmann, Hilko; Jundt, Eduard; Roth, Matthias; Schreiber, Werner; Staack, Ingo; Zimmermann, Peter; Zürl, Konrad

Das Verbundprojekt ARVIDA

2017

Schreiber, Werner (Ed.) et al.: Web-basierte Anwendungen Virtueller Techniken: Das ARVIDA-Projekt - Dienste-basierte Software-Architektur und Anwendungsszenarien für die Industrie. Berlin: Springer Vieweg, 2017, pp. 1-16

Ziel des BMBF-geförderten Verbundprojektes ARVIDA war die anwendungs- und nutzerorientierte Forschung, Entwicklung und Evaluation von zukunftsorientierten Technologien im Kontext der Virtuellen Techniken in Verbindung mit neuen und modernen Ansätzen des Semantic Web, mit dessen Hilfe Virtuelle Techniken über Dienste verbunden werden können, um zukünftig ein großes Maß an Interoperabilität unter Berücksichtigung von Echtzeitanforderungen sicherzustellen.

Show publication details

Fellner, Dieter W.; Baier, Konrad; Ackeren, Janine van; Alexandrin, Max; Barth, Anna; Bockholt, Ulrich; Kopold, Franziska; Löwer, Chris; May, Thorsten; Peters, Wiebke; Wehner, Detlef; Gollnast, Anja; Bumke, Carina

Jahresbericht 2016: Fraunhofer-Institut für Graphische Datenverarbeitung IGD

2017

Darmstadt, 2017

Das Fraunhofer IGD hat seine Forschungsaktivitäten vor Kurzem in vier Leitthemen gebündelt, welche die Basis seiner Arbeit bilden und verschiedene Themen abteilungsübergreifend miteinander verknüpfen. Eines dieser Leitthemen ist "Visual Computing as a Service - Die Plattform für angewandtes Visual Computing". Die Basis dieser universellen Plattform für Visual-Computing Lösungen ist gelegt und wird kontinuierlich erweitert. Dieser technologische Ansatz bildet die Grundlage für die weiteren Leitthemen. In der "Individuellen Gesundheit - Digitale Lösungen für das Gesundheitswesen" werden die Daten betrachtet, die in der personalisierten Medizin anfallen - mithilfe der Visual-Computing-Technologien des Instituts. Im Leitthema "Intelligente Stadt - Innovativ, digital und nachhaltig" ist die Fragestellung, wie man den Lebenszyklus urbaner Prozesse unterstützen kann. Und im Leitthema "Digitalisierte Arbeit - Der Mensch in der Industrie 4.0" geht es erster Linie um die Unterstützung des Menschen in der durch die Digitalisierung veränderten Produktion.

Show publication details

Bockholt, Ulrich; Bochtler, Thomas; Enderlein, Volker; Olbrich, Manuel; Otto, Michael; Prieur, Michael; Sauerbier, Richard; Stechow, Roland; Wehe, Andreas; Wirsching, Hans-Joachim

Motion Capturing

2017

Schreiber, Werner (Ed.) et al.: Web-basierte Anwendungen Virtueller Techniken: Das ARVIDA-Projekt - Dienste-basierte Software-Architektur und Anwendungsszenarien für die Industrie. Berlin: Springer Vieweg, 2017, pp. 219-262

Motion Capture Technologien werden in vielen unterschiedlichen Bereichen eingesetzt, wie zum Beispiel der Filmindustrie, der Sportrehabilitation, der Spieleentwicklung oder im industriellen Umfeld. Die Technologie ermöglicht es, Bewegungen aufzuzeichnen und diese auf eine digitale Repräsentanz (Avatar) eines Menschen (Digital Human Model) zu applizieren. Über ein generisches Menschmodell bzw. mit Hilfe der im Projekt ARVIDA entwickelten Referenzarchitektur ist eine universelle Anwendung möglich und verschiedene Expertensysteme lassen sich derart nutzen, dass ein ganzheitlicher Prozess durchgeführt werden kann. Für Ergonomie-Untersuchungen im Nutzfahrzeug können allgemein gültige digitale Bewegungsbausteine entwickelt werden, die sich nachträglich an neue, ähnliche Szenarien anpassen lassen. In der Produktionsabsicherung wird bereits in den frühen Phasen ein höherer Reifegrad ermöglicht, indem Motion Capturing für die Echtzeitsimulation von manuellen Arbeitsvorgängen in virtuellen Umgebungen eingesetzt wird.

Show publication details

Adams, Oliver; Bockholt, Ulrich; Hildebrand, Axel; Jonescheit, Leiv; Krzikalla, Roland; Olbrich, Manuel; Pankratz, Frieder; Pfützner, Sebastian; Roth, Matthias; Scheer, Fabian; Schwerdtfeger, Björn; Staack, Ingo; Wasenmüller, Oliver

Soll/Ist-Vergleich

2017

Schreiber, Werner (Ed.) et al.: Web-basierte Anwendungen Virtueller Techniken: Das ARVIDA-Projekt - Dienste-basierte Software-Architektur und Anwendungsszenarien für die Industrie. Berlin: Springer Vieweg, 2017, pp. 263-307

Das Ziel des Anwendungsszenarios ist eine präzise Digitalisierung und die Erforschung effizienter Verfahren zur Integration dieser Umgebungsdaten in Mixed Reality Anwendungen. Einsatzgebiete für die Erfassung großer Messvolumina liegen im Bereich der Fabrik-, Produktions- und Montageplanung, bei denen ein realistisches digitales Abbild der vorhandenen Arbeitsumgebung eine wichtige zusätzliche Informationsquelle darstellt. Für kleine Messvolumina liegen die Einsatzbereiche in der Produktabsicherung entlang des gesamten Produktlebenszyklus. Das Anwendungsszenario "Soll/Ist-Vergleich" zeigt das Potential der Referenzarchitektur, weil hier zahlreiche existierende Technologien und ARVIDA-Entwicklungsergebnisse unterschiedlicher Partner über die Referenzarchitektur integriert wurden. Das Teilprojekt betrachtet zum einen die Umfelderkennung und Umfelderfassung in kleinen und großen Messvolumina, zum anderen wurden projektionsbasierte AR-Lösungen realisiert, durch die Verfahren für den Soll/Ist-Abgleich realisiert werden konnten.

Show publication details

Bockholt, Ulrich; Brauns, Sarah; Fluck, Oliver; Harth, Andreas; Keitler, Peter; Koriath, Dirk; Lengowski, Stefan; Olbrich, Manuel; Staack, Ingo; Rautenberg, Ulrich; Widor, Volker

Werkerassistenz

2017

Schreiber, Werner (Ed.) et al.: Web-basierte Anwendungen Virtueller Techniken: Das ARVIDA-Projekt - Dienste-basierte Software-Architektur und Anwendungsszenarien für die Industrie. Berlin: Springer Vieweg, 2017, pp. 309-377

Es werden Szenarien beschrieben, die sich mit Aufgabenstellungen der Mitarbeiterführung und der Mitarbeiter-Assistenz in der Industrie befassen. Im Anwendungsszenario "MR-Engineering" wurden Systemlösungen erforscht, mit denen Planer, Arbeitsvorbereiter und In-Betrieb-Nehmer mit Hilfe von virtuellen Techniken und individuellen Software- Lösungen in Zukunft unterstützt werden können. Im Szenario "3D-Bauplanung" wird die automatische Verknüpfung von Generalplan und Daten zur Auftragsabwicklung dargestellt. Ziel ist es, die Arbeit des Planers mit effizienteren und anschaulicheren Folgen von Arbeitsschritten zu vereinfachen. In den weiteren Szenarien "3D-Bauanleitung", "Intelligentes Schema" und "Fertigen ohne Zeichnung" wird die Planung von Rohrleitungen und die Ableitung von Bauanleitungen unterstützt. Hier werden Daten zusammengeführt und über Augmented Reality visualisiert. Im Unterkapitel "Mobile Projektionsbasierte Assistenzsysteme" werden in vier Anwendungsszenarien Systeme entwickelt und beschrieben, die den Mitarbeiter bei der Bewältigung verschiedener Arbeitsaufgaben durch Augmented Reality mobil unterstützen. Diese AR Komponenten sind im Wesentlichen Projektionssysteme, die durch optisches Tracking mobilisiert werden. Der Schwerpunkt liegt hier in der Integration von Tracking-Systemen und Datendiensten durch die ARVIDA-Referenzarchitektur. Der Forschungsschwerpunkt im Anwendungsszenario "Instandhaltung und Training" war das schnelle und einfache Trainieren von komplexen wiederkehrenden Arbeitsabläufen. Durch vielfältige und variantenreiche Produkte ist die klassische Erstellung von Trainingsszenarien nicht mehr zu bewältigen. In drei aufeinander aufbauenden Phasen entstand in diesem Teilprojekt eine Trainingsapplikation, mit der es einem Trainer möglich ist, eigenständig und schnell Trainingsmodule ohne Programmier- oder VR-Kenntnisse zu erstellen. Im Training werden dem Trainee die Informationen als sowohl statische wie auch kongruente Überlagerungen mittels Datenbrille direkt im Sichtfeld angezeigt.

Show publication details

Engelke, Timo; Keil, Jens; Rojtberg, Pavel; Wientapper, Folker; Schmitt, Michael; Bockholt, Ulrich

Content First - A concept for Industrial Augmented Reality Maintenance Applications using Mobile Devices

2015

ACM SIGMM: MMSys '15 : Proceedings of the 6th ACM Multimedia Systems Conference. New York: ACM, 2015, pp. 105-111

Multimedia Systems Conference (MMSys) <6, 2015, Portland, OR, USA>

Although AR has a long history in the area of maintenance and service-support in industry, there still is a lack of lightweight, yet practical solutions for handheld AR systems in everyday workflows. Attempts to support complex maintenance tasks with AR still miss reliable tracking techniques, simple ways to be integrated into existing maintenance environments, and practical authoring solutions, which minimize costs for specialized content generation. We present a general, customisable application framework, allowing to employ AR and VR techniques in order to support technicians in their daily tasks. In contrast to other systems, we do not aim to replace existing support systems such as traditional manuals. Instead we integrate well-known AR- and novel presentation techniques with existing instruction media. To this end practical authoring solutions are crucial and hence we present an application development system based on web-standards such as HTML,CSS and X3D.

Show publication details

Gavish, Nirit; Gutiérrez, Teresa; Webel, Sabine; Rodríguez, Jorge; Peveri, Matteo; Bockholt, Ulrich; Tecchia, Franco

Evaluating Virtual Reality and Augmented Reality Training for Industrial Maintenance and Assembly Tasks

2015

Interactive Learning Environments, Vol.23 (2015), 6, pp. 778-798

The current study evaluated the use of virtual reality (VR) and augmented reality (AR) platforms, developed within the scope of the SKILLS Integrated Project, for industrial maintenance and assembly (IMA) tasks training. VR and AR systems are now widely regarded as promising training platforms for complex and highly demanding IMA tasks. However, there is a need to empirically evaluate their efficiency and effectiveness compared to traditional training methods. Forty expert technicians were randomly assigned to four training groups in an electronic actuator assembly task: VR (training with the VR platform twice), Control-VR (watching a filmed demonstration twice), AR (training with the AR platform once), and Control-AR (training with the real actuator and the aid of a filmed demonstration once). A posttraining test evaluated performance in the real task. Results demonstrate that, in general, the VR and AR training groups required longer training time compared to the Control-VR and Control-AR groups, respectively. There were fewer unsolved errors in the AR group compared to the Control-AR group, and no significant differences in final performance between the VR and Control-VR groups, probably due to a ceiling effect created by the use of two training trials in the selected task for participants who were expert technicians. The results suggest that use of the AR platform for training IMA tasks should be encouraged and use of the VR platform for that purpose should be further evaluated.

Show publication details

Muttscheller, Andreas; Goesele, Michael (Betreuer); Bockholt, Ulrich (Betreuer)

Use-Case getriebene Prototypisierung einer Lagernavigationsanwendung

2015

Darmstadt, TU, Bachelor Thesis, 2015

Die Lagerlogistik ist ein schnell wachsender Bereich, der immer optimiertere Prozesse für schnelle Warenbewegungen benötigt, um das Warenaufkommen der heutigen Zeit zu bewältigen. Durch die Veröffentlichung von Datenbrillen wird ein verstärkter Fokus auf Anwendungen mit Smartglasses und Augmented Reality gelegt. Diese Arbeit beschäftigt sich mit der Frage, wie man moderne Technologien wie Augmented Reality oder Indoor Navigation Prozesse innerhalb eines Lagers optimieren kann und die Fehlerrate verringert. Dazu wurde zunächst eine Bestandsaufnahme der aktuellen Prozesse und Tätigkeiten gemacht und mit Logistikern Gespräche geführt, um Probleme und Optimierungspotential zu isolieren. Mit diesen Informationen wurde ein Use-Case basiertes Konzept zu einem Prototypen erstellt, der die Probleme angeht und den Prozess optimieren und Fehler verhindern soll. Zusätzlich wurden die technischen Voraussetzungen festgehalten, die für einen Prototypen notwendig sind. Im weiteren Verlauf wurden verschiedene Local Positioning Systeme (LPS) auf ihre Tauglichkeit in einem Lagerhaus untersucht. Darunter zählt Wifi, Bluetooth Beacons und AR Codes. Außerdem wurden Augmented Reality SDKs auf ihre Funktionalitäten getestet. Mit diesen Informationen wurde ein Prototyp für eine Logistik-Indoor-Navigation (LIN) Anwendung erstellt. Für das User Interface wird das vorher ausgewählte AR SDK genutzt und für die Positionsbestimmung wird ein LPS mit Bluetooth Beacons und AR Codes genutzt. Zum Schluss wird ein Konzept zur Evaluierung des Prototypen vorgestellt und Thesen zu diesem aufgestellt. Bei einer möglichen Evaluierung sollen diese entweder bewiesen oder widerlegt werden. Eine komplette Evaluation ist aufgrund von Zeit- und Kostengründen nicht ausgeführt worden.

Show publication details

Bockholt, Ulrich; Roth, Matthias; Lengowski, Stefan; Olbrich, Manuel; Schwerdtfeger, Björn; Staack, Ingo

Zeichnungslose Fertigung - AR-Überlagerung von PMI (Product and Manufacturing Information) auf die zu fertigenden Baugruppen

2015

Schenk, Michael (Ed.): 18. IFF-Wissenschaftstage 2015. Tagungsband : Digitales Engineering zum Planen, Testen und Betreiben technischer Systeme, pp. 245-251

IFF-Wissenschaftstage <18, 2015, Magdeburg, Germany>

Der Schiffbau ist gekennzeichnet durch die Herstellung von Unikaten mit Prototypcharakter bei sich permanent ändernden Bauzuständen während der Produktentstehung. In diesem Zusammenhang werden pro Schiff etwa 12.000 Standardbauteile bzw. -baugruppen gefertigt, für die individuelle Fertigungszeichnungen erstellt werden. Ein Medienbruch (Übertragung der digitalen Zeichnung auf Papier) bringt immer zum einen eine potentielle Fehlerquelle (etwa wenn die Fertigungszeichnung nicht dem aktuellen Planungsstand entspricht) zum anderen einen erhöhten Arbeitsaufwand mit sich. Deswegen werden im Rahmen des BMBF-Projektes »ARVIDA« unter anderem Augmented-Reality-Assistenzsysteme zur zeichnungslosen Fertigung entwickelt, die die notwendigen Informationen und Arbeitsschritte direkt auf die Bauteile übertragen bzw. projizieren und somit den Werker bei der Herstellung der Werkstücke effektiv unterstützen. Neben der Erforschung von Augmented Reality Methoden wird dabei die vollständige Integration der Verfahren in den im Schiffbau etablierten Workflow und PLM Landschaft betrachtet, d.h.: - Die Fertigungsinformationen werden im CAD System in Form von PMI (Product and Manufacturing Information) erzeugt und dargestellt. - PMI sowie die zugehörigen CAD-Daten (im JT Format) können direkt aus dem PLM-System in das AR-System überführt werden (hier wurde insbesondere ein Transcoding-Dienst zu Integration der AR Technologien in die Teamcenter-Umgebung (Siemens) entwickelt. - CAD-Modelle und PMI werden auf Basis von WebGL im Web-Browser visualisiert. Dadurch ist es nicht erforderlich, eine spezielle Viewer-Software zu installieren. Die 3D-Informationen können auf jeder Plattform (z.B. Standard PC, Smartphone, Tablet) dargestellt werden. Um das Projekt zu erläutern, werden im Folgenden die Aspekte des Projektes in den Abschnitten »PMI - Product and Manufacturing Information«, »Integration in die PDM-Landschaft« und »AR Visualisierung« dargestellt.

Show publication details

Bockholt, Ulrich

Fusing Web Technologies & Augmented Reality

2014

Julier, Simon (Ed.) et al.: IEEE International Symposium on Mixed and Augmented Reality - Science & Technology 2014 : ISMAR 2014. Piscataway, NJ: IEEE Service Center, 2014, 1 p.

IEEE International Symposium on Mixed and Augmented Reality (ISMAR) <13, 2014, Munich, Germany>

Within the German research project ARVIDA a large consortium of industrial Virtual and Augmented Reality users, of technology providing companies and research institutes cooperate on the establishment of highly flexible web-based reference architecture for Augmented Reality applications. The use of web technologies is motivated by modern web standards as WebGL or WebRTC supporting e.g. real time rendering of 3D-content of video streaming within Web-Browsers. Thereby, the use of Web technologies not only offers the possibility to develop applications platform and OS independent but it also facilitates the integration of Augmented Reality into industrial workflows or PDM environments. The developed reference architecture offers RESTful tracking, rendering and interaction services that foster the combination and exchange of different algorithms with the aim to fit the technology to the specific requirements of an AR-applications in an optimal way.

Show publication details

Keil, Jens; Engelke, Timo; Schmitt, Michael; Bockholt, Ulrich; Pujol, Laia

Lean In or Lean Back? Aspects on Interactivity & Mediation in Handheld Augmented Reality in the Museum

2014

Klein, Reinhard (Ed.) et al.: GCH 2014. Short Papers - Posters : Eurographics Workshop on Graphics and Cultural Heritage. Goslar: Eurographics Association, 2014, pp. 17-20

Eurographics Symposium on Graphics and Cultural Heritage (GCH) <12, 2014, Darmstadt, Germany>

With the idea to design augmented reality experiences that attract and inform, and which are also seamlessly incorporated into interactive museum narratives, this paper explores finding the appropriate balance between attraction, interactivity and information mediation from the user's point of view. Within the scope of research project CHESS, we've implemented techniques that fuse interaction and mediation to enrich visits to cultural institutions by visual means and AR specific interactions. While it is AR's wow-effect that attracts, our findings show that users far too often struggle to cope with the system and interactivity instead of focusing on the information presentation in AR. We discuss our results of finding the right balance between interactive (lean-in) and non-interactive (lean-back) presentation and interaction techniques in AR.

Show publication details

Bockholt, Ulrich; Behr, Johannes; Wagner, Sebastian; Limper, Max; Olbrich, Manuel; Franke, Tobias

VR/AR-Technologien im Browser: Effiziente Integration virtueller Technologien ins PDM-Umfeld durch Nutzung von Web-Technologien

2014

Schenk, Michael (Ed.): 17. IFF-Wissenschaftstage 2014. Tagungsband : Digitales Engineering zum Planen, Testen und Betreiben technischer Systeme, pp. 103-108

IFF-Wissenschaftstage <17, 2014, Magdeburg, Germany>

Die Verknüpfung von Web-Technologien und VT-Technologien ermöglicht es, sehr schnell und sehr effizient, schlanke und aufgabenspezifischen Anwendungen zu entwickeln, die über HTML-Interfaces auf PDM-Systeme zugreifen. Diese Anwendungen können gut gewartet, modifiziert und erweitert werden, weil für die Anpassung lediglich HTML-Kenntnisse benötigt werden. Durch die vorgestellten Technologien können Virtual und Augmented Reality Technologien in den Web-Browser gebracht werden und Interfaces können geschaffen werden, durch die VR/AR direkt auf PDM Daten zugreifen kann. Damit können die Workflows in industriellen Anwendungen unterstützt werden und die Akzeptanz der Technologien kann in den Unternehmen gestärkt werden.

Show publication details

Keil, Jens; Pujol, Laia; Roussou, Maria; Engelke, Timo; Schmitt, Michael; Bockholt, Ulrich; Eleftheratou, Stamatia

A Digital Look at Physical Museum Exhibits: Designing Personalized Stories with Handheld Augmented Reality in Museums

2013

2013 Digital Heritage International Congress. Volume 2 : DigitalHeritage. New York: The Institute of Electrical and Electronics Engineers (IEEE), 2013, pp. 685-688

Digital Heritage International Congress (DigitalHeritage) <2013, Marseille, France>

In this paper we present the design of handheld Augmented Reality (AR) experiences that are seamlessly incorporated into interactive museum narratives, specifically for the Acropolis Museum. The experiences start by forming a visitor profile that later dynamically adapts the narrative, including the AR activities, to the user's behaviour. In this cohesive narrative context, the AR activities provide four ways to digitally look at the exhibits: virtual reconstruction of the original aspect; placement in the original location; visual highlighting of interesting details and annotations; and recreation of mythological appearances. The challenges of this design are presented, concluding with a discussion and lessons learned.

Show publication details

Bockholt, Ulrich; Wuest, Harald; Wientapper, Folker; Engelke, Timo; Webel, Sabine

Augmented Reality Assistenzsysteme für Wartung und Service in Industrie, Bau und Gebäudemanagement

2013

Schenk, Michael (Ed.): 16. IFF-Wissenschaftstage 2013. Tagungsband : Digital Engineering zum Planen, Testen und Betreiben technischer Systeme. Stuttgart: Fraunhofer Verlag, 2013, pp. 195-200

IFF-Wissenschaftstage <16, 2013, Magdeburg, Germany>

Für die Nutzbarkeit von Augmented Reality ist die Voraussetzung entscheidend, dass sich der Einsatz von AR im industriellen Kontext insbesondere dann rentiert, wenn sich Skaleneffekte einstellen. Im Bereich von Montage-, Reparatur- und Fehlerdiagnosearbeiten ist dies der Fall, wenn einerseits ein und dieselbe AR-Applikation in hohem Maße mehrfach benutzt werden kann, und andererseits die Erstellung des Trackingmodells hierfür nur einmal durchgeführt werden muss. Mit der Häufigkeit der Anwendung amortisiert sich dann der Aufwand für das einmalige Einrichten der AR-Applikation. Weiterhin ist es für die Akzeptanz von AR-Applikationen vorteilhaft, wenn beim Endanwender keine besonderen Kenntnisse hinsichtlich des Trackings bzw. der Benutzung der AR-Applikation vorausgesetzt werden müssen. Die AR-Applikation sollte den Endanwender bei der Erreichung seiner persönlichen Arbeitsziele unterstützen, ihm jedoch nicht ein zu hohes Maß an zusätzlichem Vorwissen in der Handhabung abverlangen. Seine Aufmerksamkeit sollte nicht darauf ausgerichtet sein, wie er sich verhalten muss, damit das Tracking funktioniert. Vielmehr muss er durch AR schneller, einfacher und intuitiver an die benötigten Informationen gelangen.

Show publication details

Olbrich, Manuel; Graf, Holger; Kahn, Svenja; Engelke, Timo; Keil, Jens; Rieß, Patrick; Webel, Sabine; Bockholt, Ulrich; Picinbono, Guillaume

Augmented Reality Supporting User-centric Building Information Management

2013

The Visual Computer, Vol.29 (2013), 10, pp. 1093-1105

The rapid development of geo-referenced information changed the way on how we access and interlink data. Smartphones as enabling devices for information access are main driving factor. Thus, the hash key to information is the actual position registered via camera and sensory of the mobile device. A rising technology in this context is Augmented Reality (AR) as its fuses the real world captured with the smartphone camera with geo-referenced data. The technological building blocks analyse the intrinsic sensor data (camera, GPS, inertial) to derive a detailed pose of the smartphone aiming to align geo-referenced information to our real environment. In particular, this is interesting to applications where 3D models are used in planning and organization processes as, e.g., facility management. Here, Building Information Models (BIM) were established in order to hold "as built" information, but also to manage the vast amount of additional information coming with the design, such as building components, properties, maintenance logs, documentation, etc. One challenge is to enable stakeholders involved in the overall building lifecycle to get mobile access to the management system within onsite inspections and to automatise feedback of newly generated information into the BIM. This paper describes a new AR framework that offers on-site access to BIM information and user centric annotation mechanism.

Show publication details

Webel, Sabine; Bockholt, Ulrich; Engelke, Timo; Gavish, Nirit; Olbrich, Manuel; Preusche, Carsten

Augmented Reality Training Platform for Assembly and Maintenance Skills

2013

Robotics and Autonomous Systems, Vol.61 (2013), 4, pp. 398-403. First published as Article in Press on 01 November 2012

The training of technicians in the acquisition of new maintenance and assembly tasks is an important factor in industry. As the complexity of these tasks can be enormous, the training of technicians to acquire the necessary skills to perform them efficiently is a challenging point. In this context it is shown that Augmented Reality is a powerful technology to support training in the particular context of industrial service procedures, since instructions on how to perform the service tasks can be directly linked to the machine parts to be processed. Due to the increasing complexity of maintenance tasks, it is not sufficient to train the technicians' execution of these tasks, but rather to train the underlying skills-sensorimotor and cognitive-that are necessary for an efficient acquisition and performance of new maintenance operations. All these facts lead to the need for efficient training systems for the training of maintenance and assembly skills which accelerate the technicians' learning and acquisition of new maintenance procedures. Furthermore, these systems should improve the adjustment of the training process to new training scenarios and enable the reuse of existing training material that has proven its worth. In this context a novel concept and platform for multimodal Augmented Reality-based training of maintenance and assembly skills has been developed, which includes the training of the involved sub-skills and the evaluation of the designed training system. Since procedural skills are considered as the most important skills for maintenance and assembly operations, we focus on the training of these skills, as well as appropriate methods for improving them.

Show publication details

Bockholt, Ulrich; Wientapper, Folker; Wuest, Harald; Fellner, Dieter W.

Augmented-Reality-basierte Interaktion mit Smartphone-Systemen zur Unterstützung von Servicetechnikern

2013

at - Automatisierungstechnik, Vol.61 (2013), 11, pp. 793-799

Smartphonesysteme erfordern neue Interaktionsparadigmen, die die integrierte Sensorik auswerten (GPS, Inertial, Kompass), die aber insbesondere auf die Smartphonekamera aufsetzten, mit der die Umgebung aufgezeichnet wird. In diesem Zusammenhang liefern Augmented Reality Verfahren großes Potential besonders für industrielle Anwendung bei Wartungs- und Instandsetzungsarbeiten.

Show publication details

Kahn, Svenja; Keil, Jens; Müller, Benedikt; Bockholt, Ulrich; Fellner, Dieter W.

Capturing of Contemporary Dance for Preservation and Presentation of Choreographies in Online Scores

2013

2013 Digital Heritage International Congress. Volume 1 : DigitalHeritage. New York: The Institute of Electrical and Electronics Engineers (IEEE), 2013, pp. 273-280

Digital Heritage International Congress (DigitalHeritage) <2013, Marseille, France>

In this paper, we present a generic and affordable approach for an automatized and markerless capturing of movements in dance, which was developed in the Motion Bank / The Forsythe Company project (www.motionbank.org). Thereby within Motion Bank we are considering the complete digitalization workflow starting with the setup of the camera array and ending with a web-based presentation of "Online Scores" visualizing different elements of choreography. Within our project, we have used our technology in two modern dance projects, one "Large Motion Space Performance" covering a large stage in solos and trios and one "Restricted Motion Space Performance" that is suited to be captured with range cameras. The project is realized in close cooperation with different choreographers and dance companies of modern ballet and with multi-media artists forming the visual representations of dance.

Show publication details

Engelke, Timo; Keil, Jens; Rojtberg, Pavel; Wientapper, Folker; Webel, Sabine; Bockholt, Ulrich

Content First - A concept for Industrial Augmented Reality Maintenance Applications Using Mobile Devices

2013

IEEE Computer Society Visualization and Graphics Technical Committee (VGTC): 12th IEEE International Symposium on Mixed and Augmented Reality 2013. : ISMAR 2013. Los Alamitos, Calif.: IEEE Computer Society, 2013, pp. 251-252

IEEE International Symposium on Mixed and Augmented Reality (ISMAR) <12, 2013, Adelaide, SA, Australia>

Although AR has a long history in the area of maintenance and service-support in industry, there still is a lack of lightweight, yet practical solutions for handheld AR systems in everyday workflows. Attempts to support complex maintenance tasks with AR still miss reliable tracking techniques, simple ways to be integrated into existing maintenance environments, and practical authoring solutions, which minimize costs for specialized content generation. We present a general, customisable application framework, allowing to employ AR and VR techniques in order to support technicians in their daily tasks. In contrast to other systems, we do not aim to replace existing support systems such as traditional manuals. Instead we integrate well-known AR- and novel presentation techniques with existing instruction media. To this end practical authoring solutions are crucial and hence we present an application development system based on web-standards such as HTML,CSS and X3D.

Show publication details

Bockholt, Ulrich; Behr, Johannes

instantreality - Virtual and Augmented Reality Applications for Industrial Use

2013

Neugebauer, Reimund (Ed.): VAR² 2013 - Realität Erweitern. Tagungsband : 2. Fachkonferenz zu VR/AR-Technologien an der Professur für Werkzeugmaschinen und Umformtechnik der TU Chemnitz. Auerbach: Wissenschaftliche Scripten, 2013, pp. 125-133

Fachkonferenz zu VR/AR-Technologien in Anwendung und Forschung <2, 2013, Chemnitz, Germany>

Rapid development in processing power, graphic cards and mobile computers open up a wide domain for Mixed Reality applications. Thereby the Mixed Reality continuum covers the complete spectrum from Virtual Reality using immersive projection technology to Augmented Reality using mobile systems like smartphones and tablets. At the Fraunhofer IGD the Virtual and Augmented Reality framework instantreality (www.instantreality.org) has been developed as a single and consistent interface for AR/VR developers. This framework provides a comprehensive set of features to support classic Virtual Reality (VR) as well as mobile smartphone Augmented Reality (AR). The goal is to provide a very simple application interface which includes the latest research results in the fields of computer vision, computer graphics and 3D user interaction. The system design is based on industry standards to facilitate application development and deployment.

Show publication details

Gross, Richard; Bockholt, Ulrich; Biersack, Ernst W.; Kuijper, Arjan

Multimodal Kinect-Supported Interaction for Visually Impaired Users

2013

Stephanidis, Constantine (Ed.) et al.: Universal Access in Human-Computer Interaction: Part I : Design Methods, Tools, and Interaction Techniques for eInclusion. 7th International Conference, UAHCI 2013. Berlin, Heidelberg, New York: Springer, 2013. (Lecture Notes in Computer Science (LNCS) 8009), pp. 500-509

International Conference on Universal Access in Human-Computer Interaction (UAHCI) <7, 2013, Las Vegas, NV, USA>

This paper discusses Kreader, a proof-of-concept for a new interface for blind or visually impaired users to have text read to them. We use the Kinect device to track the user`s body. All feedback is presented with auditory cues, while a minimal visual interface can be turned on optionally. Interface elements are organized in a list manner and placed ego-centric, in relation to the user's body. Moving around in the room does not change the element's location. Hence visually impaired users can utilize their "body-sense" to find elements. Two test sessions were used to evaluate Kreader. We think the results are encouraging and provide a solid foundation for future research into such an interface, that can be navigated by sighted and visually impaired users.

Show publication details

Bockholt, Ulrich; Webel, Sabine; Engelke, Timo; Olbrich, Manuel; Wuest, Harald

Skill Capturing and Augmented Reality for Training

2013

Bergamasco, Massimo (Ed.) et al.: Skill Training in Multimodal Virtual Environments. Boca Raton: Taylor & Francis, CRC Press, 2013, pp. 81-90

AR-based training applications must clearly differ from AR-based guiding applications, as they must really train the user and not only guide the user through the task. AR is a good technology for training, as instructions or location-dependent information can be directly linked and attached to physical objects. Because objects to be maintained usually contain a large number of similar components, the provision of location-dependent information is vitally important for the training. Furthermore, in AR-based training, sessions can be combined with teleconsultation technologies, and the availability of a trainer on-site is not mandatory.

Show publication details

Kahn, Svenja; Bockholt, Ulrich; Kuijper, Arjan; Fellner, Dieter W.

Towards Precise Real-Time 3D Difference Detection for Industrial Applications

2013

Computers in Industry, Vol.64 (2013), 9, pp. 1115-1128

3D difference detection is the task to verify whether the 3D geometry of a real object exactly corresponds to a 3D model of this object. We present an approach for 3D difference detection with a hand-held depth camera. In contrast to previous approaches, with the presented approach geometric differences can be detected in real-time and from arbitrary viewpoints. The 3D difference detection accuracy is improved by two approaches: first, the precision of the depth camera's pose estimation is improved by coupling the depth camera with a high precision industrial measurement arm. Second, the influence of the depth measurement noise is reduced by integrating a 3D surface reconstruction algorithm. The effects of both enhancements are quantified by a ground-truth based quantitative evaluation, both for a time-of-flight (SwissRanger 4000) and a structured light depth camera (Kinect). With the proposed enhancements, differences of few millimeters can be detected from 1 m measurement distance.

Show publication details

Kramer, Michael; Hohl, Kathrin; Bockholt, Ulrich; Schneider, Florian; Dehner, Christoph

Training Effects of Combined Resistance and Proprioceptive Neck Muscle Exercising

2013

Journal of Back and Musculoskeletal Rehabilitation, Vol.26 (2013), 2, pp. 189-199

OBJECTIVES: To investigate training effects of two different resistance and proprioceptive exercising concepts of neck muscles. MATERIAL AND METHOD: Twenty-six healthy women participated in a randomized pilot trial. The test persons were randomized to two different neck-training programs (resistance training (RT) and proprioceptive resistance training (PRT)). They performed a standardized training program for the duration of ten weeks two times weekly. The neck strength, the cross-sectional area of three neck muscle groups (1. sternocleidomastoid muscles; 2. multifidus and semispinalis cervicis muscles; 3. semispinalis capitis and splenius muscles) and the proprioceptive capability evaluated by the dynamic joint repositioning error (DJRE) of the head were assessed pre- and post-intervention. RESULTS: Strength gain did not differ significantly between the two resistance training groups (PRT group: 8.2% to 29.3%; RT group: 1.4% to 19.8%). Change of hypertrophy of all neck muscle groups was significantly (p< 0.001 to p=0.013) greater in the PRT group (18.9% to 32.3%) than in the RT group (1.5% to 12.9%). The DJRE deteriorated with 35% in the RT group and did not change in PRT group (-2.0%). CONCLUSION: In combination with resistance training, proprioceptive training led to a significantly higher muscle hypertrophy and didn't effect a significant deterioration of the proprioceptive capability compared to isolated resistance

Show publication details

Herzig, Pasquale; Englert, Michael; Wagner, Sebastian; Jung, Yvonne; Bockholt, Ulrich

X3D-EarthBrowser: Visualize Our Earth in Your Web Browser

2013

Posada, Jorge (General Chair) et al.: Proceedings Web3D 2013 : 18th International Conference on 3D Web Technology. New York: ACM Press, 2013, pp. 139-142

International Conference on 3D Web Technology (WEB3D) <18, 2013, San Sebastian, Spain>

Earth data covering geodetic, meteorological and oceanographic information provides petabyte volumes of multi-dimensional data that can be accessed via dynamic data queries specifying the data subset to be processed and rendered in real-time. Our proposed X3D-EarthBrowser interfaces these queries that are standardized by the Open Geospatial Consortium (OGC) to real-time visualization technologies based on WebGL and X3D. In this context, we have developed rendering technologies for terrain and raster data to be used in standard web browsers, which have been evaluated in different use cases and run on various operating systems. Besides interfacing these technologies we have developed data streaming components that instantaneously offer pre-renderings in lower LODs of the requested data volume while providing high responsiveness and smooth interaction.

Show publication details

Kahn, Svenja; Bockholt, Ulrich

3D Soll-Ist Abgleich mit Hilfe von Tiefenkameras

2012

Schenk, Michael (Ed.): 15. IFF-Wissenschaftstage 2012. Tagungsband : Digitales Engineering zum Planen, Testen und Betreiben technischer Systeme. Stuttgart: Fraunhofer Verlag, 2012, pp. 195-198

IFF-Wissenschaftstage <15, 2012, Magdeburg, Germany>

Für einen 3D-Soll-Ist Abgleich wird untersucht, ob ein 3D-Modell eines Objektes geometrisch mit dem realen Objekt übereinstimmt. Mit 3D Tiefenkameras ist es möglich, die Oberfläche eines Objektes in Echtzeit zu vermessen. Dadurch können Unterschiede zwischen dem 3D-Modell und dem realen Objekt in Echtzeit erkannt werden. Der Vorteil dieses Ansatzes besteht darin, dass die 3D-Kamera um das Objekt herumbewegt werden kann. Somit können Unterschiede zwischen Soll und IST-Zustand an verschiedenen Stellen des Objektes aus verschiedenen Blickwinkeln vermessen und visualisiert werden. Die große Herausforderung für einen Einsatz dieser Technologie in industriellen Szenarien besteht darin, eine ausreichende Genauigkeit der 3D-Vermessung zu erreichen. Die erreichbare Genauigkeit ist zum einen durch Messfehler der 3D-Kamera beschränkt, und zum anderen durch Ungenauigkeiten in der Bestimmung der Position und Orientierung der 3D-Kamera relativ zum realen Objekt. Um die Genauigkeit des 3D-Abgleiches zu erhöhen, wurde ein Demonstrator entwickelt, bei welchem die Position und Orientierung der Kamera mit einem hochpräzisen Faro-Messarm bestimmt wird. Dies reduziert die Fehler deutlich, die sich aus einer ungenauen Posenschätzung ergeben und ist damit ein wichtiger Schritt dahingehend, einen Echtzeit-3D Abgleich mit einer Tiefenkamera zukünftig auch industriell einsetzbar zu machen.

Show publication details

Kahn, Svenja; Olbrich, Manuel; Engelke, Timo; Keil, Jens; Rieß, Patrick; Webel, Sabine; Graf, Holger; Bockholt, Ulrich; Picinbono, Guillaume

Beyond 3D "as-Built" Information using Mobile AR Enhancing the Building Lifecycle Management

2012

Kuijper, Arjan (Ed.) et al.: 2012 International Conference on Cyberworlds. Proceedings : Cyberworlds 2012. Los Alamitos, Calif.: IEEE Computer Society Conference Publishing Services (CPS), 2012, pp. 29-36

International Conference on Cyberworlds (CW) <11, 2012, Darmstadt, Germany>

The success of smartphone technologies changed the way information is processed as more and more geo-referenced structures are emerging linking information to specific locations within our environment. Thereby, Augmented Reality has become a key technology as it analyses the smartphone sensor data (camera, GPS, inertial) to derive the detailed pose of the smartphone, with the aim to correlate our real environment to the geo-referenced information space. In particular, this is relevant for application fields where 3Dmodels are used in planning and organization processes as e.g. facility management. In facility construction and management Building Information Model (BIM) was established as a standard that not only holds the 3D-building-geometry but encompasses pipe/electrical systems as well as semantic building information, for example properties and conditions of building components. With this motivation our work integrates BIM and Augmented Reality.

Show publication details

Gross, Richard; Bockholt, Ulrich (Betreuer); Kuijper, Arjan (Betreuer)

Multimodal Kinect-supported Interaction for the Visually Impaired

2012

Darmstadt, TU, Master Thesis, 2012

This thesis suggests a new computer interface, specifically targeted at blind and visually impaired people. We use the Microsoft Kinect to track a user's position and have implemented a novel spatial interface to control text-to-speech synthesis of a document. Which actions are executed is solely determined through hand movements in relation to the body. All feedback for the actions is given in auditory form, through synthesized speech or earcons. Earcons are brief, unique sounds that convey information. Visually impaired or blind users do not have to point or remember keyboard commands, but can use their proprioceptive sense to effectively explore documents and execute actions. The test results are encouraging. Even when participants found themselves lost they were always able to find their way back to an interface state they knew how to navigate. Furthermore, most negative feedback can be attributed to the current technical limitations and not the spatial interface itself.

Show publication details

Olbrich, Manuel; Wuest, Harald; Rieß, Patrick; Bockholt, Ulrich

Augmented Reality Pipe Layout Planning in the Shipbuilding Industry

2011

IEEE Computer Society Visualization and Graphics Technical Committee (VGTC): 10th IEEE International Symposium on Mixed and Augmented Reality : ISMAR 2011. The Institute of Electrical and Electronics Engineers (IEEE), 2011, pp. 269-270

IEEE International Symposium on Mixed and Augmented Reality (ISMAR) <10, 2011, Basel, Switzerland>

As large ships are never produced in masses, it often occurs that the construction process and production process overlap in time. Many shipbuilding companies have problems with discrepancies between the construction data and the real built ship. The assembly department often has to modify CAD data for a successful installation. We present an augmented reality system, where a user can visualize the construction data of pipes and modify these in the case of misalignment, collisions or any other conflicts. The modified pipe geometry can be stored and further used as input for CNC pipe bending machines. To guarantee an exactly orthogonal passage of the pipes through aligning bolt holes, we integrated an optical measurement tool into the pipe alignment process.

Show publication details

Webel, Sabine; Bockholt, Ulrich; Engelke, Timo; Peveri, Matteo; Olbrich, Manuel; Preusche, Carsten

Augmented Reality Training for Assembly and Maintenance Skills

2011

BIO Web of Conferences, Vol.1 (2011), 4 p.

The International Conference SKILLS <1, 2011, Montpellier, France>

Augmented Reality (AR) points out to be a good technology for training in the field of maintenance and assembly, as instructions or rather location-dependent information can be directly linked and/or attached to physical objects. Since objects to maintain usually contain a large number of similar components (e.g. screws, plugs, etc.) the provision of location-dependent information is vitally important. Another advantage is that AR-based training takes place with the real physical devices of the training scenario. Thus, the trainee also practices the real use of the tools whereby the corresponding sensorimotor skills are trained.

Show publication details

Kahn, Svenja; Malerczyk, Cornelius; Graf, Holger; Bockholt, Ulrich

Capturing Motion Skills with Silhouette-Based Numerical Pose Estimation

2011

BIO Web of Conferences, Vol.1 (2011), 4 p.

The International Conference SKILLS <1, 2011, Montpellier, France>

The capturing of human movements is an important step for the analysis of human skills, e.g. for sports analysis or for learning-by-demonstration tasks. In this paper we introduce a new markerless pose estimation method which estimates human poses from silhouettes. The presented numerical pose estimation algorithm adapts a non-deterministical annealing schedule for silhouette based motion capturing. The pose is estimated by numerically minimizing the differences between the silhouettes of synthesized views of a 3D avatar and the silhouettes of the real person in the camera images. The evaluation results of simulation experiments quantify the trade-off between the accuracy and the execution time of the presented algorithm.

Show publication details

Keil, Jens; Bockholt, Ulrich (Betreuer)

Conveying Information for Cultural Heritage with Mobile Augmented Reality: Concepts for Mixing Realities on Smartphones

2011

Darmstadt, Hochschule, Master Thesis, 2011

Augmented Reality (AR) technology is stepping into mass markets and everyday usage. While the technology evolves from lightweight to sophisticated technical approaches running on mobile devices, this thesis questions and explores how to perceive information with AR and how to interact with reality on mobile devices, mediated by it. The thesis analyzes recent smartphone-based approaches of video-see-through AR and proposes a categorization that shapes AR's nature. Thereby it shows how AR can sustainably add value for conveying information. It gives an overview of information mediation and interaction techniques that are suitable for mobile AR on smartphones and recent tablets in the context of cultural heritage. Examples and small experiments give an impression of the state-of-the-art and discuss what might still be missing. A practical exploration and demonstration is given in the context of Fraunhofer IGD's Olbrich project. Here, computer vision-based AR is employed outdoors in order to illustrate the history of House of Olbrich - one of Darmstadt's sights in the famous Artists' Colony. By the video see-through effect users can jump back in time visually with a smartphone: AR enriches a user's view on reality with a reconstruction based on historical blueprints. It reveals the building's original appearance in a sketch?like manner and communicates architectural characteristics in a reduced visual fashion. The presented AR techniques overcome current lightweight-AR and demonstrate how AR can add value to users through compelling visual depictions and how it extends user experience through enhanced interaction.

Show publication details

Webel, Sabine; Bockholt, Ulrich; Keil, Jens

Design Criteria for AR-Based Training of Maintenance and Assembly Tasks

2011

Shumaker, Randall (Ed.): Virtual and Mixed Reality - New Trends: Part I : International Conference, Virtual and Mixed Reality 2011. Berlin, Heidelberg, New York: Springer, 2011. (Lecture Notes in Computer Science (LNCS) 6773), pp. 123-132

International Conference on Virtual and Mixed Reality (VMR) <4, 2011, Orlando, FL, USA>

As the complexity of maintenance tasks can be enormous, the efficient training of technicians in performing those tasks becomes increasingly important. Maintenance training is a classical application field of Augmented Reality explored by different research groups. Mostly technical aspects (e.g tracking, 3D augmentations) have been in focus of this research field. In our paper we present results of interdisciplinary research based on the fusion of cognitive science, psychology and computer science. We focus on analyzing the improvement of AR-based training of maintenance skills by addressing also the necessary cognitive skills. Our aim is to find criteria for the design of AR-based maintenance training systems. A preliminary evaluation of the proposed design strategies has been conducted by expert trainers from industry.

Show publication details

Webel, Sabine; Bockholt, Ulrich; Engelke, Timo; Gavish, Nirit; Tecchia, Franco

Design Recommendations for Augmented Reality based Training of Maintenance Skills

2011

Alem, Leila (Ed.) et al.: Recent Trends of Mobile Collaborative Augmented Reality Systems. Berlin, Heidelberg, New York: Springer, 2011, pp. 69-82

International Workshop on Mobile Collaborative Augmented Reality (MCAR) <2010, Seoul, South Korea>

Training for service technicians inmaintenance tasks is a classical application field of Augmented Reality explored by different research groups. Mostly technical aspects (tracking, visualization etc.) have been in focus of this research field. In this chapter we present results of interdisciplinary research based on the fusion of cognitive science, psychology and computer science. We focus on analyzing the improvement of Augmented Reality based training of skills which are relevant for maintenance and assembly tasks. The skills considered in this work comprise sensorimotor skills as well as cognitive skills. Different experiments have been conducted in order to find recommendations for the design of Augmented Reality training systems which overcome problems of existing approaches. The suggestions concern the fields of content visualization and multimodal feedback.

Show publication details

Bockholt, Ulrich; Webel, Sabine; Staack, Ingo; Riedel, Michael; Rieß, Patrick; Olbrich, Manuel; Wuest, Harald

Kooperative Mixed Reality für Konstruktion und Fertigung im Schiffsbau

2011

Schenk, Michael (Ed.): 14. IFF-Wissenschaftstage 2011. Tagungsband : Digitales Engineering und virtuelle Techniken zum Planen, Testen und Betreiben technischer Systeme. Stuttgart: Fraunhofer Verlag, 2011, pp. 185-190

IFF-Wissenschaftstage <14, 2011, Magdeburg, Germany>

Verteilte Mixed Reality Systeme, d.h. VR/AR-Installationen, die sich an unterschiedlichen Standorten befinden und via Internet vernetzt sind, wurden bislang nur in akademischen Testszenarien eingesetzt. Sie konnten den Sprung in industrielle Nutzung jedoch nicht vollziehen. Im Szenario "VR-Meeting" des BMBF-Projektes AVILUS wurde auf Grundlage des Virtual und Augmented Reality Systems "instantreality" (www.instantreality.de) ein kollaboratives VR-System entwickelt, mit dem Entwicklungsteams an verteilten Standorten in einer virtuellen Konstruktionssitzung zusammenarbeiten. Dabei wurden zum einen Verfahren für eine sichere Datenverteilung realisiert, zum anderen wurden Methoden zur effizienten Protokollierung von verteilten immersiven Konstruktionssitzungen erforscht.

Show publication details

Hartmann, Philip; Kahn, Svenja; Bockholt, Ulrich; Kuijper, Arjan

Towards Symmetry Axis based Markerless Motion Capture

2011

Bender, Jan (Ed.) et al.: VRIPHYS 11: 8th Workshop in Virtual Reality Interactions and Physical Simulations. Goslar: Eurographics Association, 2011, pp. 73-82

International Workshop in Virtual Reality Interaction and Physical Simulations (VRIPhys) <8, 2011, Lyon, France>

A natural interaction with virtual environments is one of the key issues for the usability of Virtual Reality applications. Device-free, intuitive interactions with the virtual world can be achieved by capturing the movements of the user with markerless motion capture. In this work we present a markerless motion capture approach which can be used to estimate the human body pose in real-time with a single depth camera. The presented approach requires neither a 3D shape model of the tracked person nor a training phase in which body shapes are learned a priori. Instead, it analyzes the curvature of the human body to estimate the symmetry axes of the body joints. These symmetry axes are then used to calculate the pose of the tracked human in real-time. The presented approach was evaluated qualitatively with a time-of-flight and a Kinect depth camera. Furthermore, quantitative simulation results show that the proposed approach is promising for depth cameras which can reliably capture the surface curvature (and thus the normals) of a person and which have a resolution of at least 320x240 pixel.

Show publication details

Gavish, Nirit; Seco, Teresa Gutierrez; Webel, Sabine; Rodríguez, Jorge; Peveri, Matteo; Bockholt, Ulrich

Transfer of Skills Evaluation for Assembly and Maintenance Training

2011

BIO Web of Conferences, Vol.1 (2011), 4 p.

The International Conference SKILLS <1, 2011, Montpellier, France>

One of the research topics within the EU-project SKILLS1 was the training of Industrial Maintenance and Assembly (IMA) tasks. The IMA demonstrator developed comprehends two different training platforms, one based on technologies of Virtual Reality (VR) and the other one on Augmented Reality (AR). To qualify the efficiency of the developed training systems different studies have been conducted, followed by a final "Transfer of Skill" evaluation that has been performed by service technicians "SIDEL industrial training Entre" in Parma. This evaluation included qualitative methods (feedback collection in questionnaires) as well as quantitative methods (experiments with control groups). The results demonstrate that both platforms are useful and suitable training tools for IMA tasks, and that the AR training decreased the number of unsolved errors in the task.

Show publication details

Konietschke, Rainer; Tobergte, Andreas; Preusche, Carsten; Tripicchio, Paolo; Ruffaldi, Emanuele; Webel, Sabine; Bockholt, Ulrich

A Multimodal Training Platform for Minimally Invasive Robotic Surgery

2010

Avizzano, Carlo Alberto (Ed.) et al.: IEEE RO-MAN 2010 : 19th IEEE International Symposium on Robot and Human Interactive Communication. Proceedings [online]. New York: The Institute of Electrical and Electronics Engineers (IEEE), 2010, pp. 422-427

IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN) <19, 2010, Viareggio, Italy>

This paper gives an overview of a multimodal training platform developed for minimally invasive robotic surgery, based on the DLR MiroSurge system. It describes the technological components and integration of the hardware and software platform and presents the first integrated training tasks that enable surgeons to get familiar with the robotic system. The training platform shares the same surgical operator workstation as MiroSurge and simulates the behaviour of the telemanipulator arms and the surgical instruments. Like the real system the training platform provides haptic feedback and 3D-vision. However instead of the real telemanipulator itself, a virtual environment with abstracted tasks is connected to the operator workstation. This allows reduction in costs, to provide various levels of difficulty, and to focus on the skills to be taught. Thus a training platform is presented that aims at training a surgeon's skills in handling the robotic system MiroSurge rather than training surgery in general.

Show publication details

Schwenk, Karsten; Franke, Tobias; Drevensek, Timm; Kuijper, Arjan; Bockholt, Ulrich; Fellner, Dieter W.

Adapting Precomputed Radiance Transfer to Real-time Spectral Rendering

2010

Lensch, Hendrik P. A. (Ed.) et al.: Eurographics 2010. Short Papers, pp. 49-52

Eurographics <31, 2010, Norrköping, Sweden>

Spectral rendering takes the full visible spectrum into account when calculating light-surface interaction and can overcome the well-known deficiencies of rendering with tristimulus color models. We present a variant of the precomputed radiance transfer algorithm that is tailored towards real-time spectral rendering on modern graphics hardware. Our method renders diffuse, self-shadowing objects with spatially varying spectral reflectance properties under distant, dynamic, full-spectral illumination. To achieve real-time frame rates and practical memory requirements we split the light transfer function into an achromatic part that varies per vertex and a wavelength-dependent part that represents a spectral albedo texture map. As an additional optimization, we project reflectance and illuminant spectra into an orthonormal basis. One area of application for our research is virtual design applications that require relighting objects with high color fidelity at interactive frame rates.

Show publication details

Schwenk, Karsten; Kuijper, Arjan; Bockholt, Ulrich

Modeling Wavelength-Dependent BRDFs as Factored Tensors for Real-Time Spectral Rendering

2010

Institute for Systems and Technologies of Information, Control and Communication (INSTICC): VISIGRAPP 2010. Proceedings : International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications. INSTICC Press, 2010, pp. 165-172

International Conference on Computer Graphics Theory and Applications (GRAPP) <5, 2010, Angers, France>

Spectral rendering takes the full visible spectrum into account when calculating light-surface interaction and can overcome the well-known deficiencies of rendering with tristimulus color models. In this paper we show how to represent wavelength-dependent BRDFs as factored tensors. We use this representation for realtime spectral rendering on modern graphics hardware. Strong data compression and fast rendering times are achieved for mostly diffuse and moderately glossy isotropic surfaces. The method can handle high-resolution tabulated BRDFs, including non-reciprocal ones, which makes it well-suited for measured data. We analyze our approach numerically and visually. One area of application for our research is virtual design applications that require high color fidelity at interactive frame rates.

Show publication details

Engelke, Timo; Webel, Sabine; Bockholt, Ulrich; Wuest, Harald; Gavish, Nirit; Tecchia, Franco; Preusche, Carsten

Towards Automatic Generation of Multimodal AR-Training Applications and Workflow Descriptions

2010

Avizzano, Carlo Alberto (Ed.) et al.: IEEE RO-MAN 2010 : 19th IEEE International Symposium on Robot and Human Interactive Communication. Proceedings [online]. New York: The Institute of Electrical and Electronics Engineers (IEEE), 2010, pp. 434-439

IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN) <19, 2010, Viareggio, Italy>

Augmented Reality (AR) is a technology which has become very popular in the last years. In this context also the idea of using of AR for training applications has become very important. AR offers a large potential for training only if the training is well focused to the skills that have to be trained and if the training protocol is well designed. On the other hand, the generation of the training content to be transferred via AR is a comprehensive problem that is addressed in this paper. Thus, this paper tries to describe the whole chain of implementations and general aspects involved in the creation of AR training applications, including examples for used multimodal devices. This chain starts with the capturing of expert actions to be hold in "digital representation of skill". The digital representation of skill is transferred to the training protocol that specifies the storyboard of the AR training session. The paper includes two different implementations of AR training systems and describes the general idea of informational abstraction from low level data up to interaction and from design to application.

Show publication details

Riedel, Michael; Staack, Ingo; Rieß, Patrick; Wuest, Harald; Bockholt, Ulrich

Virtual und Augmented Reality Technologien zur Unterstützung von Konstruktion und Fertigung im U-Bootsbau

2010

Schenk, Michael (Ed.): 13. IFF-Wissenschaftstage 2010. Tagungsband : Digitales Engineering und virtuelle Techniken zum Planen, Testen und Betreiben technischer Systeme. Stuttgart: Fraunhofer Verlag, 2010, pp. 140-145

IFF-Wissenschaftstage <13, 2010, Magdeburg, Germany>

Um die globale Wettbewerbsfähigkeit nachhaltig zu sichern, die erhöhten technischen Anforderungen zu erfüllen und verkürzte Entwicklungszyklen bei gleichzeitig erhöhter Komplexität zu gewährleisten, ist der effiziente Einsatz von CAx- und virtuellen Technologien im Wertschöpfungsprozess für HDW unbedingte Voraussetzung. Dabei findet aufgrund von geringen Stückzahlen und hoher Individualität im U-Bootsbau eine enge Verzahnung von Fertigung und Entwicklung statt. Diese Verzahnung erfordert einen permanenten Abgleich zwischen digital konstruierten und real gefertigten Bauteilen. Um diese Anforderung zu unterstützen, werden im Rahmen des BMBF geförderten Projektes AVILUS Technologien entwickelt, die einerseits eine VR-gestützte Kooperation an verteilten Standorten unterstützten ("VRMeeting"), die andererseits aber mit Hilfe von Augmented Reality Technologien auch einen Abgleich von digitaler (Entwicklung) und realer Welt (Fertigung) unterstützen.

Show publication details

Kahn, Svenja; Bockholt, Ulrich

3D-Rekonstruktion mit einer Tiefenkamera für industrielle Augmented Reality Anwendungen

2009

Schenk, Michael (Ed.): 12. IFF-Wissenschaftstage 2009. Tagungsband : Digitales Engineering zum Planen, Testen und Betreiben technischer Systeme. Magdeburg: Fraunhofer IFF, 2009, pp. 105-112

IFF-Wissenschaftstage <12, 2009, Magdeburg, Germany>

Time-of-Flight Kameras sind im Gegensatz zu herkömmlichen Kameras in der Lage, nicht nur Farb- oder Graustufenwerte sondern auch Tiefeninformationen der aufgezeichneten Szene zu erfassen. Sie liefern für jedes erfasste Pixel neben einem Intensitätsbild auch Informationen über die Entfernung des jeweiligen Pixels und erfassen somit ein 2,5D Bild. Diese Tiefeninformationen können zur Rekonstruktion eines 3D-Modells einer Szene oder eines Objektes genutzt werden. Markerlose Tracking-Ansätze für Augmented Reality nutzen häufig Geometrieinformationen über die Szene, die mit virtuellen Zusatzinformationen überlagert werden soll. Mit Hilfe von Tiefenkameras wird es möglich, die vorliegende Szene zu rekonstruieren. Dafür wird die Szene aus verschiedenen Richtungen mit der Tiefenkamera erfasst und durch das Zusammenfügen der aus den einzelnen Bildern gewonnenen Tiefeninformationen automatisch rekonstruiert. Das so gewonnene 3D-Modell kann daraufhin bei markerlosen Augmented Reality Anwendungen zum Abgleich mit der jeweils aktuellen Sicht der Szene und somit zur Bestimmung der Kamerapose verwendet werden. Die über die getrackte Szene vorliegenden Geometrieinformationen sind dabei nicht auf den vor Beginn des Trackings erfassten Bereich beschränkt, sondern können auch während des Trackings dynamisch erweitert werden.

Show publication details

Preusche, Carsten; Konietschke, Rainer; Tobergte, Andreas; Ruffaldi, Emanuele; Webel, Sabine; Bockholt, Ulrich

Development of a Multimodal Skills Trainer for Minimally Invasive Telerobotic Surgery

2009

Gutiérrez, Teresa (Ed.) et al.: SKILLS 09 : Enaction on SKILLS. Donostia - San Sebastián: SKILLS Consortium, 2009, pp. 153-161

International Conference on Multimodal Interfaces for Skills Transfer (SKILLS) <1, 2009, Bilbao, Spain>

This paper gives an overview of the components and integration of the prototype for a multimodal skills training system developed for Minimally Invasive Telerobotic Surgery (MITS), which is based on DLR's Miro-Surge system. It describes the technological components and integration of the Hardware and Software platform. Furthermore, it presents the first integrated training environment and assessment scheme. The MITS skills trainer consists of a surgical workstation, providing force and 3D visual feedback, which is identical with the real workstation and a VR environment, in which the robotic systems and the training environment is simulated, and which can be controlled depending on the training situation and difficulty level.

Show publication details

Gavish, Nirit; Engelke, Timo; Webel, Sabine; Bockholt, Ulrich

Enhancing Trainee's Mental Model During AR Assembly Training

2009

Gutiérrez, Teresa (Ed.) et al.: SKILLS 09 : Enaction on SKILLS. Donostia - San Sebastián: SKILLS Consortium, 2009, pp. 189-190

International Conference on Multimodal Interfaces for Skills Transfer (SKILLS) <1, 2009, Bilbao, Spain>

When a learner of a new procedural skill is provided with elaborated knowledge: "how-it-works" knowledge or "context procedures", in addition to the "how-to-do-it" knowledge or "list procedures", his performance becomes more accurate, faster, and more flexible. It is hypnotized that a display of the device which is assembled during the assembly task has the potential to support the trainee's mental model building.

Show publication details

Fellner, Dieter W.; Behr, Johannes; Bockholt, Ulrich

instantreality - A Framework for Industrial Augmented and Virtual Reality Applications

2009

The 2nd Sino-German Workshop "Virtual Reality & Augmented Reality in Industry" : Invited Paper Proceedings. Participants Edition. Shanghai: Shanghai Jiao Tong University, 2009, pp. 78-83

Sino-German Workshop "Virtual Reality & Augmented Reality in Industry" <2, 2009, Shanghai, China>

Rapid development in processing power, graphic cards and mobile computers open up a wide domain for Mixed Reality applications. Thereby the Mixed Reality continuum covers the complete spectrum from Virtual Reality using immersive projection technology to Augmented Reality using mobile systems like Smartphones and UMPCs. At the Fraunhofer Institute for Computer Graphics (IGD) the Mixed Reality Framework instantreality (www.instantreality.org) has been developed as a single and consistent interface for AR/VR developers. This framework provides a comprehensive set of features to support classic Virtual Reality (VR) as well as mobile Augmented Reality (AR). The goal is to provide a very simple application interface which includes the latest research results in the fields of high-realistic rendering, 3D user interaction and total-immersive display technology. The system design is based on various industry standards to facilitate application development and deployment.

Show publication details

Wientapper, Folker; Ahrens, Katrin; Wuest, Harald; Bockholt, Ulrich

Linear-Projection-Based Classification of Human Postures in Time-of-Flight Data

2009

IEEE Systems, Man and Cybernetics Society: IEEE International Conference on Systems, Man and Cybernetics : SMC 2009. New York: IEEE Press, 2009, pp. 565-570

IEEE International Conference on Systems, Man and Cybernetics (SMC) <2009, San Antonio, TX, USA>

This paper presents a simple yet effective approach for classification of human postures by using a time-of-flight camera. We investigate and adopt linear projection techniques such as Locality Preserving Projections (LPP), Linear Discriminant Analysis (LDA) and Principal Component Analysis (PCA), which are more widespread in face recognition and other pattern recognition tasks.We analyze the relations between LPP and LDA and show experimentally that using LPP in a supervised manner effectively yields very similar results as LDA, implying that LPP may be regarded as a generalization of LDA. Features for offline training and online classification are created by adopting common image processing techniques such as background-subtraction and blob detection to the time-of-flight data.

Show publication details

Kramer, Michael; Honold, Marina; Hohl, Kathrin; Bockholt, Ulrich; Rettig, Alexander; Elbel, Martin; Dehner, Christoph

Reliability of a New Virtual Reality Test to Measure Cervicocephalic Kinaesthesia

2009

Journal of Electromyography and Kinesiology, Vol.19 (2009), 5, pp. e290-e300. First published as Article in Press 5 August 2008, DOI: 10.1016/j.jelekin.2008.05.005

The aim of this study was to investigate the cervicocephalic kinaesthesia of healthy subjects for gender and age effects and its reliability in a new virtual reality test procedure. 57 healthy subjects (30 male, 27 females; 18-64 years) were immersed into a virtual 3D scene via a headmounted display, which generated specific head movements. The joint repositioning error was determined in a static and dynamic test at the times T0, T1 (T0 + 10 minutes) and T2 (T0 + 24 hours). The intrasession reliability (T0-T1) and the intersession reliability (T0-T2) were analysed. In both tests no gender- or age-specific effects were found. In the overall group the means of the static test were 6.2 - 6.9 and of the dynamic test were 4.5 -4.9 . The intratest difference in the static test was -0.16 and the intertest difference was 0.47 . The intratest difference in the dynamic test was 0.42 and the intertest difference was 0.37 . The static and dynamic test was reproducible in healthy subjects, with minor deviations, irrespective of gender and age. The smaller interindividual differences in the dynamic test could be beneficial in the comparison of healthy individuals and individuals with cervical spine disorders.

Show publication details

Webel, Sabine; Staykova, Yana; Bockholt, Ulrich

Towards Workflow Acquisition of Assembly Skills using Hidden Markov Models

2009

IEEE Systems, Man and Cybernetics Society: IEEE International Conference on Systems, Man and Cybernetics : SMC 2009. New York: IEEE Press, 2009, pp. 857-862

IEEE International Conference on Systems, Man and Cybernetics (SMC) <2009, San Antonio, TX, USA>

In recent years, the demand for efficient systems which can capture and learn human skills has become increasingly important. In this paper an approach for acquiring and recognizing human assembly skills is presented. The underlying workflows of assembly skills are captured by using a simple multisensor data glove and camera tracking. To avoid the processing of redundant information, at first the relevant tasks of a workflow are identified by analyzing measuring information of the multisensor capturing system. Thus, only relevant data is comprised in the representation of a workflow. Unlike common approaches a workflow is modeled as entire unit using a continuous Hidden Markov Model (HMM). The recognition process of input patterns is based on an adaptive threshold model that can identify known workflow patterns and non-meaningful input patterns as well.

Show publication details

Bockholt, Ulrich; Webel, Sabine; Engelke, Timo; Graf, Holger; Behr, Johannes

Virtual and Augmented Reality Training Platforms for Transfer of Skills

2009

Gutiérrez, Teresa (Ed.) et al.: SKILLS 09 : Enaction on SKILLS. Donostia - San Sebastián: SKILLS Consortium, 2009, pp. 39-45

International Conference on Multimodal Interfaces for Skills Transfer (SKILLS) <1, 2009, Bilbao, Spain>

Within the European project SKILLS Virtual and Augmented Reality platforms have been developed that are used for the realization of different training simulators in the field of sports (training of rowing and juggling), surgery (training of maxillofacial and telerobotic surgery), rehabilitation (rehabilitation of upper limp), maintenance (training of assembly procedures) and robotics (training of robot programming). These platforms are integrating capturing and rendering technologies. The capturing technologies are used to register specific criteria of skills to be hold in a digital representation. Thus, parameters relevant for specific skills are qualified and used as evaluation factors within the training simulators. The training simulators not only can simulate the virtual environment of the training situation, but this environment can be enhanced via multimodal renderings, fostering the training process and accelerating the training effect. The integrative platforms from scalable and adaptable frameworks that can be used to realize different training protocols and that can be adapted to the trainee's degree of skill. Therefore the training protocol can be specified via standardized application scripts.

Show publication details

Jung, Yvonne; Recker, Ruth-Maria; Olbrich, Manuel; Bockholt, Ulrich

Using X3D for Medical Training Simulations

2008

Spencer, Stephen N. (Ed.): Proceedings WEB3D 2008 : 13th International Symposium on 3D Web Technology. New York: ACM Press, 2008, pp. 43-51

International Conference on 3D Web Technology (WEB3D) <13, 2008, Los Angeles, CA, USA>

Nowadays medical training simulators play an important role in education and further training of surgeons. With Virtual Reality based training systems it is possible to simulate a surgery under realistic conditions. Input data for the visualization of anatomic structures is tomographic image data. For the visualization of medical datasets direct volume rendering is the method of choice. In this paper we introduce a system based on the new X3D extension proposal of the Medical Working Group for a volume rendering component, with some extensions for controlling the speed vs. quality trade-off. For a convincing and instructive training simulation, our implementation delivers high performance combined with high quality visualization of medical datasets. Furthermore, it integrates haptic forcefeedback devices to assure realistic interactions.

Show publication details

Bockholt, Ulrich; Jung, Yvonne; Recker, Ruth-Maria; Olbrich, Manuel

Visuelles und haptisches Volumenrendering von medizinischen Bilddaten in Virtual Reality-Simulationen

2008

Schenk, Michael (Ed.): 11. IFF-Wissenschaftstage 2008. Tagungsband [CD-ROM]. Magdeburg: Fraunhofer IFF, 2008, pp. 31-37

IFF-Wissenschaftstage <11, 2008, Magdeburg, Germany>

Virtual-Reality Simulationen konnten sich in der chirurgischen Ausbildung etablieren, wobei die angebotenen Systeme von einfachen Trainingssimulatoren bis hin zu komplexen Chirurgiesimulatoren reichen. Diese Simulationen sind oft sehr überzeugend, häufig steckt allerdings ein großer Aufwand in der Aufbereitung von Trainingsfällen und der Generierung von Falldatenbanken. Die Herausforderung besteht hier also in der Vereinfachung der Generierung der Trainingsdatenbank. Eingangsdaten für die Trainingsdatenbank sind Tomographiedaten der Patienten, die als Volumendaten vorliegen. Mit dieser Motivation werden Volumenrendering-Methoden sowie Simulationsverfahren auf Volumendaten zur Verwendung in chirurgischen Trainingssimulatoren entwickelt und präsentiert. Die entwickelten Renderingverfahren betreffen sowohl visuelles als auch haptisches Rendering, da letzteres in endoskopischen Simulationen zur Vermittlung von chirurgischen Handfertigkeiten besonders wichtig ist. Die Trainingssimulatoren unterscheiden sich hierbei darin von anderen Trainingssystemen, dass die zu untersuchenden anatomischen Strukturen nicht exemplarisch modelliert werden sondern direkt aus Tomographiedaten rekonstruiert sind. Zur Evaluation der entwickelten Verfahren zum visuellen und haptischen Volumenrendering wurden Beispielanwendungen für die HNO-Medizin entwickelt.

Show publication details

Schumann, Marco; Bockholt, Ulrich

ViVERA-Virtuelles Kompetenznetzwerk zur virtuellen und erweiterten Realität: Schlussbericht des BMBF-Projektes.

2008

Magdeburg, 2008

Mit dem Vorhabem "Virtuelles Kompetenznetzwerk zur Virtuellen und Erweiterten Realität" (ViVERA) wurde das Ziel verfolgt, Kompetenzen von VR- und AR-Entwicklern zu vernetzen, Entwicklungsbedarf zu identifizieren, Erfahrungen von Entwicklern und Anwendern zu bündeln und auf andere Anwendungsbereiche zu übertragen. Im Fokus standen hier die Branchen Automobil-, Maschinen-, Anlagen- und Schiffbau sowie die Medizintechnik. Innerhalb des Projektes konzentrierten sich die Forschungsarbeiten zunächst auf anwendungsübergreifende Basistechnologien. Im Ergebnis des Projektes entstanden aus der Kombination von Basistechnologien und einer anwendungsspezifischen Weiterentwicklung von neun prototypischen Demonstrantoren, die neue Einsatzszenarien von VR/AR-Technologien in den Zielbranchen aufzeigen. Die Demonstratoren stehen in den durchführenden Instituten zur Verfügung und bilden dort die Grundlage für weitere Entwicklungen nach Anforderungen von Industriepartnern, um diese in einen praktischen Einsatz zu überführen.

Show publication details

Webel, Sabine; Bockholt, Ulrich

Immersive Annotation - Ein Annotationssystem für immersive Umgebungen

2007

Schenk, Michael (Ed.): 10. IFF-Wissenschaftstage 2007. Tagungsband [CD-ROM] : 15 Jahre Fraunhofer IFF. Magdeburg: Fraunhofer IFF, 2007, VR-Teil, pp. 193-200

IFF-Wissenschaftstage <10, 2007, Magdeburg, Germany>

Immer kürzer werdende Produktionszyklen und erhöhte Komplexitäten von Maschinen erfordern eine effektive und zeitsparende Entwicklungsumgebung, in der Simulation eine zunehmend wichtige Rolle spielt. In diesem Zusammenhang werden auch immer mehr Technologien aus dem Bereich der Virtuellen Realität (VR) eingesetzt, um Ergebnisse von CAE Prozessen in den verschiedenen Entwicklungsstufen zu visualisieren und um die nächsten Entwicklungsschritte zu diskutieren. Schwierigkeiten bereitet allerdings die Protokollierung der Absprachen und Diskussionsergebnisse. Dieses Paper beschreibt eine Annotationssystem, das die Erstellung, Integration und Protokollierung von Annotationen in Form von Texten, Skizzen, Bildern/Symbolen, Interaktionen, Audio- und Videoclips, sowie deren Darstellung auf zwei unterschiedliche Arten. Die Darstellungsformen der Annotationen basieren auf realen Metaphern, die in VR übertragen wurden.

Show publication details

Dold, Christian; Bockholt, Ulrich; Roth, Marcus; Heussel, Claus Peter; Gosepath, Jan; Sakas, Georgios

Segmentation and Navigation Support of Clinical Data Sets to Simulate the Bronchoscopy and Rhinoscopy

2007

Buzug, Thorsten M. (Ed.): Advances in Medical Engineering. Berlin, Heidelberg, New York: Springer, 2007. (Springer Proceedings in Physics 114), pp. 145-150

Workshop of Medical Robotics, Navigation and Visualization (MRNV) <2, 2007, Remagen, Germany>

A training and simulation system for therapy planning is developed based on patient specific imaging data. A real endoscope is used for navigation through the virtual patient. For this purpose sensors were built in the endoscope in order to track the translation, rotation and the angle of the distal end. Pre-processing (segmentation, tissue characterization) speeds-up the volume rendering up to real-time. Collision detection enables a realistic fly through the virtual patient.

Show publication details

Dold, Christian; Bockholt, Ulrich; Roth, Marcus; Heussel, Claus Peter; Gosepath, Jan; Sakas, Georgios

Simulations- und Trainingsystem für die Bronchoskopie / Rhinoskopie durch klinische Datensätze

2007

Deutsche Gesellschaft für Biomedizinische Technik im VDE (DGBMT): BMT 2007. Proceedings [CD-ROM]. 41. Jahrestagung der Deutschen Gesellschaft für Biomedizinische Technik (DGBMT) im VDE : Biomedizinische Technik - Biomedical Engineering, Vol. 52 (2007) Ergänzungsband. Berlin: de Gruyter, 2007, 2 pp.

Jahrestagung der Deutschen Gesellschaft für Biomedizinische Technik im VDE (BMT) <41, 2007, Aachen, Germany>

Auf Grundlage patientenspezifischer Bilddaten wurde ein Trainings- und Simulationssystem entwickelt, mit dem eine anstehende Therapie vorbereitet werden kann. Zur Navigation durch die Bilddaten wird ein echtes Endoskop benutzt. Hierzu wurden Sensoren im Endoskop eingebaut die die Translation, Rotation und den Winkel des distalen Endes abgreifen. Ein label file, generiert aus dem Segmentierungsergebnis, und ein Kollisisons Detektions Algorithmus unterstützen die Navigation indem die Position der relativen Sensoren bei Kollision angepasst wird.

Show publication details

Bockholt, Ulrich; Drevensek, Timm; Schäffer, Albert; Bisler, Alexander; Rettig, Alexander; Kramer, Michael

Simulation zur Stimulation von Kopfbewegungen zur VR gestützten Schleudertraumatherapie

2006

Schulze, Thomas (Ed.) et al.: Simulation und Visualisierung 2006. Proceedings. Erlangen: SCS-European Publishing House, 2006, pp. 49-62

Simulation und Visualisierung (SimVis) <2006, Magdeburg, Germany>

Funktionelle Störungen stellen die Ursache chronischer Schmerzen nach Schleudertrauma dar und verursachen immense Kosten im Gesundheitssystem. In diesem Verfahren wird durch den kombinierten Einsatz von Virtual Reality (VR) und Messungen der muskulären Aktivität die Diagnose chronischer Schmerzen nach Schleudertrauma verbessert. Die Diagnose resultiert in einer Kraftrückkopplung, die gezielt Kräfte auf den Kopf des Patienten übertragen soll. Hierzu wird der Patient mit Hilfe eines "Head Mounted Displays" (HMD) in eine virtuelle Welt versetzt. Der Patient verfolgt ein optisches Signal in der virtuellen Szene und wird dadurch animiert, spezifische Kopfbewegungen auszuführen. Durch das Einleiten fein dosierter Gegenkräfte soll die Nackenmuskulatur der Schmerzpatienten dabei, gesteuert durch VR Techniken, sensibel trainiert werden. Zeitgleich wird die bei den Bewegungen generierte elektrische Aktivität (EMG) des Musculus semispinalis capitis gemessen. Sie dient als Signal, um über einen Rückkopplungsmechanismus (Bio- Feedback) eine Überlastung des Patienten während der Therapie zu verhindern.

Show publication details

Sakas, Georgios; Bockholt, Ulrich

Simulators and Closed Interaction Loops: Guest Editors' Introduction

2006

IEEE Computer Graphics and Applications, Vol.26 (2006), 6, pp. 22-23

Show publication details

Dold, Christian; Bockholt, Ulrich; Sakas, Georgios; Roth, Marcus

Virtuelle Analyse und Simulation im Bereich der Bronchoskopie und Rhinoskopie mit echten Patientendaten

2006

Schenk, Michael (Ed.): 9. IFF-Wissenschaftstage 2006. Tagungsband : Virtual Reality and Augmented Reality zum Planen, Testen und Betreiben technischer Systeme. Magdeburg: Fraunhofer IFF, 2006, pp. 163-170

IFF-Wissenschaftstage <9, 2006, Magdeburg, Germany>

Auf Grundlage patientenspezifischer Bilddaten wird ein Trainings- / Simulationssystem entwickelt, mit dem eine anstehende Therapie vorbereitet werden kann. Die untersuchten Fälle können gleichzeitig zum Aufbau einer Trainingsdatenbank herangezogen werden.

Show publication details

Sakas, Georgios; Bockholt, Ulrich

Anwendungen der Virtuellen und Erweiterten Realität in der Medizin

2005

Schulze, Thomas (Ed.) et al.: Simulation und Visualisierung 2005. Proceedings. Erlangen: SCS-European Publishing House, 2005, pp. 383-392

Simulation und Visualisierung (SimVis) <16, 2005, Magdeburg>

Rechnergestützte Verfahren beginnen den medizinischen Alltag und die chirurgische Praxis zu revolutionieren. Insbesondere Techniken aus den Bereichen "Virtuelle Realität und Erweiterte Realität" bergen große Potentiale für medizinische Ausbildung, Diagnose, Operationsvorbereitung und intraoperative Unterstützung. Dabei versteht man unter "Virtueller Realität" eine 3D-Datenwelt, in die der Benutzer eintauchen und mit Hilfe neuer Interaktionsmechanismen intuitiv und effizient interagieren kann. Mit dem Begriff "Erweiterte Realität" wird die Technologie bezeichnet, in der die reale Umgebung mit computergenerierten Darstellungen bereichert und überlagert wird. Am Fraunhofer Institut für Graphische Datenverarbeitung werden medizinische Trainingssimulatoren entwickelt, mit denen angehende Chirurgen die Handhabung von Endoskop und chirurgischen Instrumenten üben. Darauf aufbauend wurden Projekte im Bereich der patientenspezifischen Therapieplanung durchgeführt: Dreidimensionale Modelle der anatomischen Strukturen können aus dem Tomographiedatensatz der Patienten rekonstruiert werden, mit Hilfe eines Planungssystems wird der bevorstehende Eingriff exakt spezifiziert. So wird sichergestellt, dass der optimale Ansatz für eine bevorstehende Operation vorbereitet wird. Die Ergebnisse der Operationsplanung werden mit Hilfe von Verfahren aus dem Bereich "Augmented Reality" intraoperativ visualisiert, um eine Navigationsunterstützung zu geben, die eine exaktes Umsetzen des Therapieplans garantiert.

Show publication details

Sakas, Georgios; Bockholt, Ulrich

Chirurgisches Training mit Technologien der Virtuellen Realität

2005

Endoskopie heute, Vol.18 (2005), 3, pp. 105-107

Rechnergestützte Verfahren beginnen den medizinischen Alltag und die chirurgische Praxis zu revolutionieren. Insbesondere Techniken aus den Bereichen "Virtuelle Realität" und "Erweiterte Realität" bergen große Potentiale für medizinische Ausbildung, Diagnose, Operationsvorbereitung und intraoperative Unterstützung. Dabei versteht man unter "Virtueller Realität" eine 3D-Datenwelt, in die der Benutzer eintauchen und mit Hilfe neuer Interaktionsmechanismen intuitiv und effizient interagieren kann. Mit dem Begriff "Erweiterte Realität" wird die Technologie bezeichnet, in der die reale Umgebung mit computergenerierten Darstellungen bereichert und überlagert wird. Am Fraunhofer Institut für Graphische Datenverarbeitung werden medizinische Trainingssimulatoren entwickelt, mit denen angehende Chirurgen die Handhabung von Endoskop und chirurgischen Instrumenten üben. Darauf aufbauend wurden Projekte im Bereich der patientenspezifischen Therapieplanung durchgeführt: Dreidimensionale Modelle der anatomischen Strukturen können aus dem Tomographiedatensatz der Patienten rekonstruiert werden, mit Hilfe eines Planungssystems wird der bevorstehende Eingriff exakt spezifiziert. So wird sichergestellt, dass der optimale Ansatz für eine bevorstehende Operation vorbereitet wird. Die Ergebnisse der Operationsplanung werden mit Hilfe von Verfahren aus dem Bereich "Augmented Reality" intraoperativ visualisiert, um eine Navigationsunterstützung zu geben, die eine exaktes Umsetzen des Therapieplans garantiert.

Show publication details

Webel, Sabine; Bockholt, Ulrich (Betreuer)

Interaktionsparadigmen zur Erstellung von Annotationen in immersiven Umgebungen

2005

Darmstadt, TU, Diplomarbeit, 2005

Die Arbeit beschäftigt sich mit der Konzeption eines Systems zur Erstellung von Annotationen in immersiven Umgebungen unter der Verwendung VR spezifischer Interaktionsmetaphern. In diesem Zusammenhang wurde ein Annotationssystem entwickelt, das unter Verwendung VR spezifischer Interaktionsmetaphern die Dokumentation von Ergebnisprotokollen innerhalb der VR Umgebung unterstützt. Das System beinhaltet das Erstellen von Annotationen unterschiedlicher Art (Text, Skizze, Symbol, Audio, etc.), verschiedene Formen zur Darstellung der Anmerkungen, sowie das Erzeugen und Einlesen eines Protokolls über alle vorhandenen Annotationen. Das Konzept wurde innerhalb des VR-Systems AVALON implementiert und getestet.

Show publication details

Friberg, Jessica; Behr, Johannes (Betreuer); Bockholt, Ulrich (Betreuer)

Real-time Rendering of Volumetric Tomography Data for the Integration into a Surgical Simulator

2005

Linköping, Univ., Master Thesis, 2005

Over the last fifteen years volume rendering have become an important part of examining, exploring and evaluating volume data. Huge medical datasets generated by Computed Tomography (CT) or Magnetic Resonance Imaging (MRI) have with volume rendering been made possible to explore in 3D. Even interaction with the data is today possible and it has been shown that this is of great value when diagnosing, practicing and/or learning. In order to make interactive volume rendering possible a range of sophisticated methods have been developed. Often high-end graphic hardware are applied when developing applications for 3D volume rendering. This is an obstacle when it comes to integrating the techniques into the normal hospital environment. It would be preferable if the applications are general enough to run on standard computers. In this thesis a volume renderer for standard computers with good visual result and a relatively high update rate, sufficient for interaction, is developed. The last years tremendous development of graphic hardware have made it possible to write programs, so called shaders, directly communicating with the graphic pipeline. The developed renderer employs the use of such shaders because of its superiority in speed to normal software rendering. One purpose for the renderer is that it will be used for exploring medical data as a part of a project between Fraunhofer Institut and the Hospital of Mainz.

Show publication details

Bockholt, Ulrich; Encarnação, José L. (Betreuer); Sakas, Georgios (Betreuer)

Rechnergestützte Therapie unter Berücksichtigung patientenspezifischer Biomechanik

2004

Darmstadt, TU, Diss., 2004

Computer-assisted methods have emerged in medical diagnosis and therapy planning. Thereby the planning often relies on a 3D model of the anatomical structures reconstructed from medical image data (CT, MRT, US). Within this dissertation the diagnostic possibilities are enhanced by combining the model generation of with a simulation of the patient specific biomechanics. Not only image data also biomechanical parameters are considered in the simulation. The pathologic function can be analyzed within the simulation, this analysis is combined with a description of the planned intervention and it results in a prediction of the therapy outcome and the change of functionality caused by the planned intervention. Using this approach different biomechanical phenomena have been examined in different medical application fields: Static analyses have been performed to specify an optimal prosthesis for the implantation of an artificial knee joint, a simulation of the patient specific jaw movement helps to detect occlusions and to verify the design of onlays and crowns in dental diagnostics. Stress-strain simulation is applied in spine surgery and flow simulation has been used in ENT medicine to analyze the patient-specific air flow in the nose. For the presented approaches not only the diagnostic method is presented also an evaluation strategy is outlined that can be applied to proof the significance of the simulation. The presented work results in an approach registering biomedical parameters in real-time and considering these parameters in a Virtual Reality based method for whiplash therapy.

Show publication details

Lutz, Bernd; Becker, Mario; Stricker, Didier; Bockholt, Ulrich

The Augmented Reality Ocular

2004

Encarnação, José L. (Honorary Chair) et al.: ACM SIGGRAPH International Conference on Virtual Reality Continuum and its Applications in Industry. Proceedings : ACM SIGGRAPH International Conference on Virtual Reality Continuum and its Applications in Industry. New York: ACM, 2004, pp. 352-354

ACM SIGGRAPH International Conference on Virtual Reality Continuum and its Applications in Industry (VRCAI) <1, 2004, Singapore>

In this paper we present the Augmented Reality Ocular, an extension of the traditional coin-operated binocular with augmented reality capabilities. The Augmented Reality Ocular enables a real implementation of the augmented reality paradigm and makes the technology available to a large public. After a review of the state of the art and a presentation of current related projects, we describe our approach of implementation. Finally different concept and purpose of applications are described in order to demonstrate the wide field of application of this new augmented reality device.

Show publication details

Bockholt, Ulrich; Bisler, Alexander; Becker, Mario; Müller-Wittig, Wolfgang K.; Voss, Gerrit

Augmented Reality for Enhancement of Endoscopic Interventions

2003

Chen, Jim (Ed.) et al.: IEEE Virtual Reality 2003 Conference. Proceedings. Los Alamitos, Calif.: IEEE Computer Society, 2003, pp. 97-101

IEEE Virtual Reality (VR) <10, 2003, Los Angeles, CA, USA>

Computer assisted operation planning systems win more and more recognition in the field of surgery. These systems offer new possibilities to prepare an intervention with the goal to shorten the expansive time in the operation room required for the intervention. The safest and most effective surgical approach should be selected. But often, it is difficult to transfer the output of the planning system to the intra-operative situation and so to consider the planning results in the real intervention. At the Fraunhofer Institute for Computer Graphics (IGD) in Darmstadt and the Centre for Advanced Media Technology (CAMTech) in Singapore, methods are developed to bridge the gap between the external planning session and the intra-operative case: Augmented Reality (AR) techniques are used to overlap preoperative scanned image data as well as results of the planning session to the operation field.

Show publication details

Turban, Georg; Müller-Wittig, Wolfgang K. (Betreuer); Bockholt, Ulrich (Betreuer)

Development of a Robust Registration Method for Computer Assisted Surgery

2003

Darmstadt, TU, Diplomarbeit, 2003

Current developments in surgery and computer science expose many new technologies and techniques. In the context of minimal invasive surgery, the surgeon is using very small instruments, which are inserted into patient's body through ports. The disadvantage of new surgical approaches like minimal invasive techniques is the more difficult intraoperative orientation for the surgeon, based on intraoperative sensors that deliver only a small part of the intraoperative situation. To assist the surgeon also preoperative data will be acquired. Those data can be used in combination with techniques of augmented reality to enhance the field of view of the surgeon. Registration aims to close the gap between the preoperative operating planning scenario and the intraoperative support by calculating the transformation that maps one data set onto the corresponding other. In the underlying medical scenario, the developed registration algorithm calculates the transformation that maps preoperative data onto intraoperative data. The main idea is not to calculate the transformation in a traditional way by building point correspondences on both data sets based on minimal distances, like the Iterative Closest Point-algorithm. The developed algorithm instead characterizes preoperative and intraoperative surface data to build the desired correspondences between both data sets. Those characteristics can be created for vertices for example and encodes the local neighborhood around such vertices. The integration of additional information like directed characteristics or histological information is elementary and was implemented to enhance the accuracy and performance of the algorithm.

Show publication details

Müller-Wittig, Wolfgang K.; Bockholt, Ulrich; Voss, Gerrit; Börner, Martin

A New Approach to Combine Visualization and Simulation for Preoperative Planning of Pedicle Screw Insertion

2002

The International Society for Computer Assisted Orthopaedic Surgery: Computer Assisted Orthopaedic Surgery. Proceedings 2002, pp. 53-55

Annual Meeting of the International Society for Computer Assisted Orthopaedic Surgery (CAOS) <2, 2002, Santa Fe, NM, USA>

Show publication details

Müller-Wittig, Wolfgang K.; Becker, Mario; Bockholt, Ulrich; Voss, Gerrit

Advanced Simulation Environment for Minimally Invasive Procedures in Gynecology

2002

Mills, John (Ed.): ITEC 2002. Europe's Training, Education and Simulation Conference Proceedings. CD-ROM. Lille, 2002, 4 p.

International Training and Education Conference & Exhibition (ITEC) <13, 2002, Lille, France>

Rapid development of medical field, expanding knowledge base and new technologies require continuing medical education to achieve life long learning and to keep the surgeons up to date. Consequently, specific training is necessary to guarantee qualification of the surgeons. To overcome the current drawbacks of traditional training systems (on-the-job training, plastic models etc.) for laparoscopy/hysteroscopy an intelligent adaptable training environment has been realized using Virtual Reality (VR), Multimedia (MM) technology, and Intelligent Tutoring Systems (ITS).

Show publication details

Müller-Wittig, Wolfgang K.; Becker, Mario; Elias, Thomas; Bockholt, Ulrich; Voss, Gerrit

Advanced Training Environment for Gynecologic Endoscopy

2002

Lemke, Heinz U. (Ed.) et al.: Computer Assisted Radiology and Surgery 2002. Proceedings. Berlin, Heidelberg, New York: Springer, 2002, pp. 107-112

Computer Assisted Radiology and Surgery (CARS) <16, 2002, Paris, France>

Rapid development of medical field, expanding knowledge base and new technologies require continuing medical education to achieve life long learning and to keep the surgeons up to date. Consequently, specific training is necessary to guarantee qualification of the surgeons. To overcome the current drawbacks of traditional training systems (on - the - job training, plastic models etc.) for labaroscopy / hysteroscopy an intelligent adaptable training environment has been realized using Virtual Reality (VR), Multimedia (MM) technology, and Intelligent Tutoring Systems (ITS).

Show publication details

Müller-Wittig, Wolfgang K.; Mlynski, Gunter; Weinhold, Ivo; Bockholt, Ulrich; Voss, Gerrit

Nasal Airflow Diagnosis - Comparison of Experimental Studies and Computer Simulations

2002

Westwood, James D. (Ed.): Medicine Meets Virtual Reality 2002. Proceedings : Digital Upgrades: Applying Moore's Law to Health. Amsterdam: IOS Press; Ohmsha, 2002. (Studies in Health Technology and Informatics 85), pp. 311-317

Medicine Meets Virtual Reality (MMVR) <10, 2002, Newport Beach, CA, USA>

The lack of situed diagnostic tools providing insight into patient specific flow characteristics of the nasal airflow is one of the main problems in functional diagnosis. Diagnostics methods currently used do not provide the necessary information for flow analysis. But the flow distribution is essential for a physiological repiration, in particular for cleaning moistening and tempering of the inhaled air as well as for the olfactory function of the nose. To overcome this current situation a cooperation project of the ENT surgeons and computer graphic engineers was established to develop the computer assisted planning system STAN (Simulation Tool for Airflow in the human Nose) combining Computer Fluid Dynamics (CFD) with advanced Computer Graphic Technology. The idea of the STAN system is to perform patient specific airflow simulations in the patient's nasal cavities. Therefore a geometrical model of the nasal airways is derived form the patient's tomography scans. A discretization of the surrounded flow volume is made by a computational grid. To establish the flow simulation Finite Element Methods are performed on the grid. A tailored visualization is offered to the surgeon that overlaps the flow pattern to the patient's tomography data shown in the coronal sagittal and transversal plane. The surgeon can not only analyze the patient's current respiration situation he has also the possibility to describe the planned surgical intervention. The goal is to simulate the flow distribution that can be expected after the surgical intervention and to offer a possibility to validate various surgical strategies. To verify the simulation results experimental investigation and measurements are made in nasal models. Silicon Models of patient's nose channels are made to analyze flow characteristics The CT or MR scans of the same patients are used as input data for the simulation. The experimental outcome is compared to the simulation results to validate this diagnostic approach.

Show publication details

Bockholt, Ulrich; Mlynski, Gunter; Weinhold, Ivo; Müller-Wittig, Wolfgang K.; Voss, Gerrit

Simulation of the Nasal Airflow - Generation and Evaluation of a Patient Specific Flow

2002

Hutten, Helmut (Ed.) et al.: Second European Medical and Biological Engineering Conference. Proceedings Vol. 2 : Advancement of Medicine and Health Care through Technology - the Challenge to Biomedical Engineering in Europe. Graz: Verlag der Technischen Universität, 2002, pp. 1170-1171

European Medical and Biological Engineering Conference (EMBEC) <2, 2002, Vienna, Austria>

Show publication details

Bisler, Alexander; Bockholt, Ulrich; Kordaß, Bernd; Suchan, M.; Voss, Gerrit

The Virtual Articulator

2002

International Journal of Computerized Dentistry, Vol.5 (2002), 2, pp. 101-106

The virtual articulator (VA) is intended for use as a tool for the analysis of the complex static and dynamic occlusal relations. Its chief application is in individual diagnosis and the simulation of the mechanical articulator. As entry data, the VA requires digital, 3-dimensional representations of the jaws and patient-specific data on jaw movement. It then simulates jaw movement and provides a dynamic visualization of the occlusal contacts. If no patient-specific data are available, the modus operandi of the mechanical articulator can be simulated. Virtual reality (VR) techniques applied in computer-aided diagnosis are revolutionizing medical routine in many disciplines. Particularly in the field of dental engineering, VR combined with CAD/CAM technology has great potential in implant planning and design in terms of greater precision and shorter treatment duration. The geometry of teeth can be digitized, the implant models can be designed with specialized CAD systems, and the final implants can be manufactured with a CNC machine. In this context the virtual articulator completes the digital manufacturing process.

Show publication details

Bisler, Alexander; Bockholt, Ulrich; Voss, Gerrit

The Virtual Articulator - Applying VR Technologies to Dentistry

2002

Institute of Electrical and Electronics Engineers (IEEE): Sixth IEEE International Conference on Information Visualisation 2002. Proceedings. Los Alamitos, Calif.: IEEE Computer Society, 2002, pp. 600-602

IEEE International Conference on Information Visualization (IV) <6, 2002, London, UK>

The use of Virtual Reality (VR) techniques for Computer-Aided Diagnosis is revolutionizing the medical routing in various medical disciplines. Especially in the field of dentistry VR in combination with CAD/CAM technologies offers a high potential for implant planning/design resulting in a higher accuracy and a shorter therapy time period. The geometry of the teeth is recognisable and digitisable using an intraoral camera, the implant models can be designed with specialized CAD systems and the final implants are generated using a CNC machine. In this context the "Virtual Articualtor" (VA) completes the digital manufacturing pipeline and offers the possibility to consider patient-specific jaw biomechanics in the implant generation process. The objective of this project is the development of a system for dental occlusion diagnosis. For this important diagnosis in today's dental routine the mechanical articulator is used. The VA simulates and augments the functionalities of the mechanical articulator. Therefore it takes a digital 3-D representation of the jaws as input data, generates a simulation of the jaw movements, and delivers a dynamical visualization of the occlusion points. The high technical demands to the system are given through the high accuracy necessary for dental applications and the large models that have to be handled in the system.

Show publication details

Kordaß, Bernd; Gärtner, Christian; Söhnel, Andreas; Bisler, Alexander; Voss, Gerrit; Bockholt, Ulrich; Seipel, Stefan

The Virtual Articulator in Dentistry: Concept and Development

2002

The Dental Clinics of North America, Vol.9 (2002), 3, pp.493-506

Show publication details

Bockholt, Ulrich; Müller-Wittig, Wolfgang K.; Los Arcos, José Luis; Oppelt, Peter; Stähler, Jan; Voss, Gerrit

Enhanced Training Environment for Minimally Invasive Surgery

2001

Institute of Electrical and Electronics Engineers (IEEE): Proceedings of the 10th IEEE International Workshop on Enabling Technologies: Infrastructure for Collaborative Enterprises. Los Alamitos, Calif.: IEEE Computer Society, 2001, pp. 269-272

IEEE International Workshop on Enabling Technologies: Infrastructure for Collaborative Enterprises (WET ICE) <10, 2001, Cambridge, MA, USA>

Rapid development of medical field, expanding knowledge base and new technologies require continuing medical education to achieve life long learning and to keep the surgeons up to date. Consequently, specific training is necessary to guarantee qualification of the surgeons. To overcome the current drawbacks of traditional training systems (on-the-job training, plastic models etc.) for laparoscopy/hysteroscopy an intelligent adaptable training environment has been realised using Virtual Reality (VR), Multimedia (MM) technology, ans Intelligent Tutoring Systems (ITS)

Show publication details

Voss, Gerrit; Bisler, Alexander; Bockholt, Ulrich; Müller-Wittig, Wolfgang K.; Schäffer, Albert

ICAPS - An Integrative Computer-Assisted Planning System for Pedicle Screw Insertion

2001

Westwood, James D. (Ed.): Medicine Meets Virtual Reality 2001. Proceedings : Outer Space, Inner Space, Virtual Space. Amsterdam: IOS Press; Ohmsha, 2001. (Studies in Health Technology and Informatics 81), pp. 561-563

Medicine Meets Virtual Reality (MMVR) <9, 2001, Newport Beach, CA, USA>

Robot Assisted Surgery (RAS) Systems win more and more recognition in the field of orthopaedics. Especially in Hip Surgery RAS has proved to be suited for application in medical routine. Often Robot Assisted Surgery Systems consist of a planning- and an interoperative component. According to specifications done with the planning software the tools are driven. Benefits of the robot assisted surgery should be higher precision and a better surgical outcome. In the co-operation project of several Fraunhofer Institutes "RoMed" (Robots and Manipulators for Medical Application) an exemplary application of robot aided spine surgery is developed. The planning software used in this context is proposed in the remaining article.

Show publication details

Müller-Wittig, Wolfgang K.; Bockholt, Ulrich; Schäffer, Albert; Voss, Gerrit; Haberland, N.; Börner, Martin

Integrative Preoperative Planning for Pedicle Screw Inseration

2001

The International Society for Computer Assisted Orthopaedic Surgery: Computer Assisted Orthopaedic Surgery. Proceedings 2001

Annual Meeting of the International Society for Computer Assisted Orthopaedic Surgery (CAOS) <2001, USA>

Show publication details

Müller-Wittig, Wolfgang K.; Bockholt, Ulrich; Los Arcos, José Luis; Oppelt, Peter; Stähler, Jan; Voss, Gerrit

LAHYSTOTRAIN - VR-based Intelligent Training Environment for Laparoscopy and Hysteroscopy

2001

Richir, Simon (Ed.) et al.: Virtual Reality International Conference (VRIC) 2001. Proceedings, pp. 225-233

Virtual Reality International Conference (VRIC) <2001, Laval, France>

Rapid developments in the medical field, as an expanding knowledge base and emerging new technologies require continuing medical education to achieve life long learning and to keep the surgeons up to date. Consequently, specific training is necessary to guarantee qualification of the surgeons. The goal of LAHYSTOTRAIN is to overcome the current drawbacks of traditional training methods for laparoscopic/ hysteroscopic procedures. A computer-assisted simulator for training and quality control in laparoscopy and hysteroscopy is developed using Virtual Reality (VR), Multimedia (MM) technology, and Intelligent Tutoring Systems (ITS).

Show publication details

Müller-Wittig, Wolfgang K.; Bisler, Alexander; Bockholt, Ulrich; Los Arcos, José Luis; Oppelt, Peter; Stähler, Jan; Voss, Gerrit

Lahystotrain: Development and Evaluation of a Complex Training System for Hysteroscopy

2001

Westwood, James D. (Ed.): Medicine Meets Virtual Reality 2001. Proceedings : Outer Space, Inner Space, Virtual Space. Amsterdam: IOS Press; Ohmsha, 2001. (Studies in Health Technology and Informatics 81), pp. 336-340

Medicine Meets Virtual Reality (MMVR) <9, 2001, Newport Beach, CA, USA>

Hysteroscopy has already become an irreplaceable method in gynaecoloic diagnosis and therapy. In the diagnostic case the hysteroscope with a 30° optic is insert transvaginally, in the therapeutic case the resctoscope with a 12° optic is used. The endoscoppic intervention requires special surgical skills for endoscope handling and remote instrument control. To acquire these skills currently hands-on training in clinical praxis has become standard, which is linked with higher danger for the women. To overcome current drawbacks of traditional training methods the European project LAHYSTOTRAIN was set up, that tries to combine Virtual Reality (VR), Multimedia (MM) technology, and Intelligent Tutoring Systems (ITS) to develop an alternative training system for hysteroscopic interventions. The first prototype of the LAHYSTOTRAIN demonstrator has been shown on several European conferences. An evaluation of the system was performed, with the idea, to collect feedback and impressions, that should be considered in further developments. This paper presents the LAHYSTOTRAIN prototype and the results of these evaluations.

Show publication details

Giere, Michael; Bockholt, Ulrich (Betreuer)

Virtuelle Endoskopie

2001

Darmstadt, FH, Diplomarbeit, 2001

In dieser Arbeit wird zunächst der aktuelle Stand der Technik im Bereich der Virtuellen Endoskopie vorgestellt, darauf aufbauend wird ein Konzept zur Generierung von virtuellen Endoskopien präsentiert und implementiert. Basierend auf der Technik des Keyframing wird die manuelle Planung von Kameraflügen und die anschließende Generierung von Animationen ermöglicht. Dazu können mittels eines Volumenrenderers Schlüsselpositionen für einen Kameraflug durch anatomischen Strukturen markiert und in die Keyframes übernommen werden. Mit Hilfe des Kochanek-Bartels-Splines wird eine geglättete Kurve, die nachträglich modifiziert werden kann, durch die Schlüsselpositionen gelegt und für die Berechnung der Kamerapositionen verwendet. Die Einführung einer flexiblen virtuellen Kamera ermöglicht in den Keyframes eine unabhängige Definition von Bewegungsrichtung, Geschwindigkeit und Orientierung und somit für den Anwender die Möglichkeit einer individuellen Steuerung. Für eine weiche Interpolation der Kamera-Orientierung wird ein Verfahren basierend auf einem Look-At-Spline vorgestellt. Die gezielte Festlegung der Kameraeigenschaften erlaubt eine wesentlich verbesserte Beobachtung relevanter Details und trägt so zu einer gesteigerten Qualität der Animationen bei. Um abrupte Änderungen der Kamera-Orientierung zu reduzieren wurde ein Verfahren entwickelt, das auf der Berechnung des Kamera-Koordinatensystems mit einem festen Aufwärtsvektor basiert. Jedem Keyframe kann außerdem ein Etikett zugeordnet werden, das sowohl die Verwendung der Liste der Keyframes erleichtert, als auch Informationen in einer Animation zur Verfügung stellt. Mit Hilfe der Keyframes können die Einstellungen der virtuellen Kamera für jedes Bild eines Kamerafluges berechnet werden (Inbetweening). Diese dienen als Grundlage für die Durchführung von Kontrollsequenzen und auch der Berechnung der fertigen Animation.

Show publication details

Müller, Wolfgang; Bockholt, Ulrich; Voss, Gerrit; Lahmer, Armin; Börner, Martin

Computer Assisted Preoperative Planning System for Total Knee Replacement

2000

Centers for Medical Robotics and Computer Assisted Surgery (MRCAS) at Carnegie Mellon University and UPMC Shadyside Hospital: Computer Assisted Orthopaedic Surgery. Proceedings. Pittsburgh, 2000, pp. 105-107

Annual North American Program on Computer Assisted Orthodpaedic Surgery <4, 2000, Pittsburgh, PA, USA>

Total knee replacement (TKR) is a common orthopaedic surgical intervention and includes the removal of bone sections from the end of the femur and the top of the tibia for replacement by prosthetic components. Pain relief and functional improvement are predictable clinical results. But the accuracy of the alignment affects the surgical outcome and the longevity of the prosthesis. Hence, current total knee implantation systems attempt to align the knee joint in the mechanical axis for placement of the total knee components. These approaches use templates and plain radiographs for preoperative planning and alignment devices for bone cuts. To overcome the inherent inaccuracy of the presently used systems a computer-assisted planning system has been developed delivering the necessary control data for the intraoperative surgical robot system.

Show publication details

Voss, Gerrit; Ecke, Ulrich; Müller, Wolfgang; Bockholt, Ulrich; Mann, Wolf

How to Become the "High Score Cyber Surgeon" - Endoscopic Training Using the Nasal Endoscopy Simulator (NES)

2000

Lemke, Heinz U. (Ed.): Computer Assisted Radiology and Surgery (CARS) 2000. Proceedings. Amsterdam; Lausanne: Elsevier, 2000. (International Congress Series 1214), pp. 290-293

Computer Assisted Radiology and Surgery (CARS) <14, 2000, San Francisco, CA, USA>

Rapid development in the medical field, an expanding knowledge base and upcoming technologies require a continuing medical education to achieve life long learning and to keep the surgeons up to date. Consequently, specific training is necessary to guarantee the qualification of the surgeon. In recent years several training systems have been developed using Virtual Reality (VR) techniques to overcome the current drawbacks of traditional training systems (on-the-job training, plastic models etc.). In a co-operation project of University Hospital in Mainz (Department of Otalaryngology Head and Neck Surgery) and the Technical University in Darmstadt the Nasal Endoscopy Simulator (NES) is developed. Similar to a scenario in a computer game several tasks grouped into levels have to be solved during the training process. These levels are starting with simple tasks in abstract virtual environments to train the handling of the endoscope and they are ending with scenarios using complex models of the endonasal sinus to train orientation as well as diagnostic and therapeutic procedures. During a simulated interventions errors of the trainee, for example collisions of instruments and high sensitive tissues, are detected and evaluated. The integrative adaptable training environment provides new opportunities in medical education and enhances the medical standard curriculum. This concept has a general applicability to other medical fields.

Show publication details

Bockholt, Ulrich; Voss, Gerrit; Müller, Wolfgang

Innovative Approaches in Computer-Assisted Surgery

2000

Nanyang Technological University: Sixth International Conference on Control, Automation, Robotics and Vision (ICARCV 2000). Conference Proceedings : Intelligent Automation In The New Millenium

International Conference on Control, Automation, Robotics and Vision (ICARCV) <6, 2000, Marina Mandarin, Singapore>

Total knee replacement (TKR) is a common orthopaedic surgical intervention and includes the removal of bone sections from the end of the femur and the top of the tibia for replacement by prosthetic components. Pain relief and functional improvement are predictable clinical results. But the accuracy of the alignment affects the surgical outcome and the longevity of the prosthesis. Hence, current total knee implantation systems attempt to align the knee joint in the mechanical axis for placement of the total knee components. These approaches use templates and plain radiographs for preoperative planning and alignment devices for bone cuts. To overcome the inherent inaccuracy of the presently used systems a computer-assisted planning system has been developed delivering the necessary control data for the intraoperative surgical robot system.

Show publication details

Voss, Gerrit; Bockholt, Ulrich; Los Arcos, José Luis; Müller, Wolfgang; Oppelt, Peter; Stähler, Jan

Lahystotrain: Intelligent training system for Laparoscopy and Hysteroscopy

2000

Westwood, James D. (Ed.): Medicine Meets Virtual Reality 2000. Proceedings : Envisioning Healing: Interactive Technology and the Patient-Practitioner Dialogue. Amsterdam: IOS Press; Ohmsha, 2000. (Studies in Health Technology and Informatics 70), pp. 359-364

Medicine Meets Virtual Reality (MMVR) <8, 2000, Newport Beach, CA, USA>

Rapid developments in the medical field, as an expanding knowledge base and emerging new technologies require continuing medical education to achieve life long learning and to keep the surgeons up to date. Consequently, specific training is necessary to guarantee qualification of the surgeons. The goal of LAHYSTOTRAIN is to overcome the current drawbacks of traditional training methods for laparoscopic/hysteroscopic procedures. A computer-assisted simulator for training and quality control in laparoscopy and hysteroscopy is developed using Virtual Reality (VR), Multimedia (MM) technology, and Intelligent Tutoring Systems (ITS).

Show publication details

Müller, Wolfgang; Bockholt, Ulrich; Voss, Gerrit; Lahmer, Armin; Börner, Martin

Planning system for computer assisted total knee replacement

2000

Westwood, James D. (Ed.): Medicine Meets Virtual Reality 2000. Proceedings : Envisioning Healing: Interactive Technology and the Patient-Practitioner Dialogue. Amsterdam: IOS Press; Ohmsha, 2000. (Studies in Health Technology and Informatics 70), pp. 214-219

Medicine Meets Virtual Reality (MMVR) <8, 2000, Newport Beach, CA, USA>

Total knee replacement (TKR) is a common orthopaedic surgical intervention and includes the removal of bone sections from the end of the femur and the top of the tibia for replacement by prosthetic components. Pain relief and functional improvement are redictable clinical results. But the accuracy of the alignment affects the surgical outcome and the longevity of the prosthesis. Hence, current total knee implantation systems attempt to align the knee joint in the mechanical axis for placement of the total knee components. These approaches use templates and plain radiographs for preoperative planning and alignment devices for bone cuts. To overcome the inherent inaccuracy of the presently used systems a computer-assisted planning system has been developed delivering the necessary control data for the intraoperative surgical robot system.

Show publication details

Bauer, Siegfried; Bockholt, Ulrich (Betreuer)

Simulation von Belastungsverteilungen der Wirbelsäule mit Hilfe von FEM

2000

Darmstadt, TU, Diplomarbeit, 2000

Die Finite Element Methode ist ein Verfahren, um das mechanische Verhalten von Materialien numerisch zu beschreiben. In den Ingenieurswissenschaften wird sie seit circa 20 Jahren eingesetzt und hat sich in der Forschung und der Praxis bewährt. Da in medizinischen Simulationen und Planungssystemen heute dazu übergegangen wird auch die Gewebeeigenschaften zu simulieren, bietet es sich an, mit der Finite Element Methode ein ausgereiftes Verfahren zu verwenden. Ziel dieser Arbeit ist die Entwicklung eines Verfahrens, mit dessen Hilfe die Belastung von Bandscheiben individuell, d.h. an patientenspezifischen Daten, simuliert werden kann. Dazu ist es nötig ein FEM-Gitter, beruhend auf patientenspezisischen Daten, zu erzeugen, ein Materialmodell zu entwickeln, das die Eigenschaften von Bandscheibengewebe nachbildet, und die Randbedingungen des gesamten Modells zu bestimmen. Anschließend soll die Integration in ein bestehendes Operationsplanungssystem möglich sein. Zur Erzeugung des FEM-Gitters wurden CT-Daten verwendet, aus denen zunächst ein dreidimensionales geometrisches Modell abgeleitet wurde. Aus diesem 3D-Modell wurde ein FEM-Gitter erzeugt, welches die Bedingungen erfüllt, die von den numerischen Algorithmen gestellt werden. Da die Anwendung der Finite Element Methode auf biologisches Gewebe bisher nicht weit verbreitet ist, mußten ein geeignetes Gewebemodell und die zugehörigen Parameter ausgewählt werden. Es wurden verschiedene Materialmodelle untersucht und schließlich das am besten geeignete, anisotrope Modell ausgewählt. Desweiteren wurden Randbedingungen festgelegt, die das Anbringen der Kräfte, die Lagerung des Gesamtmodells, bzw. die Kopplung der einzelnen Modellteile bestimmen. Zur Berechnung der auftretenden Kräfte wurde ein biomechanisches Modell eines Bewegungssegmentes der Wirbelsäule erstellt. Anhand dieses Modells wurden verschiedene Bewegungen bzw. Belastungen simuliert und die auftretenden Kräfte berechnet. Diese Belastungen dienten neben der Geometrie, der Randbedingungen und der Gewebeeigenschaften als Eingabegrößen für die verwendete FEM-Software (ANSYS). Mit Hilfe der Berechnungen konnten die Belastungen und deren Verteilung auf der Bandscheibenoberfläche berechnet werden. Die Ergebnisse können im Volumenrenderer visualisiert und mit den Tomographiedaten des Patienten überlagert werden. Das vorgestellte Verfahren erlaubt es, die Belastungen in den Bandscheiben zu quantifizieren und deren Verteilung darzustellen. Als Weiterentwicklung ist die Integration in ein Operationsplanungssystem denkbar, wodurch nicht nur der tatsächliche Zustand simuliert werden kann, sondern auch der potentielle Zustand nach einer Bandscheibenoperation unter Berücksichtigung der einsetztbaren Implantate. So könnte der Erfolg einer Operation im voraus besser abgeschätzt werden.

Show publication details

Bockholt, Ulrich; Mlynski, Gunter; Müller, Wolfgang; Voss, Gerrit

STAN - The simulation tool for airflow in the human nose

2000

Lemke, Heinz U. (Ed.): Computer Assisted Radiology and Surgery (CARS) 2000. Proceedings. Amsterdam; Lausanne: Elsevier, 2000. (International Congress Series 1214), pp. 285-289

Computer Assisted Radiology and Surgery (CARS) <14, 2000, San Francisco, CA, USA>

Computer assisted planning systems based on patient specific tomography data has been proofed to be useful in several medical fields. Patient's tomography data can be shown in several views and can be enhanced with additional information useful for the planing process. The STAN (Simulation Tool for Airflow in the human Nose) system overlaps patient data and CFD (computer fluid dynamics) data resulting from flow simulations of the airflow in the patient's nose channel. To verify the simulation results experimental investigations, and measurements are made in nasal models. Silicon Models of patient's nose channels are made to analyze flow characteristics. The CT or MR scans of the same patients are used as input data for the simulation. The experimental outcome is compared to the simulation results to validate this diagnostic approach.

Show publication details

Müller, Wolfgang; Bockholt, Ulrich; Lahmer, Armin; Voss, Gerrit; Börner, Martin

VRATS - Virtual-Reality-Arthroskopie-Trainingssimulator

2000

Der Radiologe, Vol.40 (2000), 3, pp. 290-294

Unzufriedenheit mit den traditionellen Ausbildungsmethoden in der endoskopischen Chirurgie führte zur Entwicklung eines Arthroskopietrainingssimulators, der Computergraphik und Techniken der Virtuellen Realität (VR) einsetzt. Basierend auf einem kernspintomographischen Bilddatensatz konnte mit Hilfe von Bildverarbeitungsmethoden und Visualisierungstechniken ein realistisches dreidimensionales Modell des Kniegelenks mit allen diagnostisch relevanten anatomischen Strukturen generiert und in den Simulator integriert werden. Die VR-Benutzer-Schnittstelle des Arthroskopietrainingssimulators besteht aus 2 Hauptkomponenten, der 3D-Interaktion, die das Führen der Instrumente realisiert, und der 2D-Benutzungsoberfläche, die das visuelle Feedback und die Kontrolle der Trainingsitzung ermöglicht. Auf diese Weise ist ein intuitives Handling der Instrumente garantiert. Der Chirurg ist in der Lage, eine arthroskopische Untersuchung an einen virtuellen Kniegelenk durchzuführen. Dazu sind auch erste Ansätze zur Simulation von Gewebeverformung implementiert, die aufgrund von Instrumenteinwirkungen auftreten.

Show publication details

Bockholt, Ulrich; Voss, Gerrit; Müller, Wolfgang

Anwendung der Virtuellen Realität in der Medizin

1999

Biomedical Journal, (1999), 53, pp.4-6

In der modernen Medizin spielt die Verarbeitung medizinischer Bilddaten eine wichtige Rolle. Bildgebende Verfahren wie Röntgen-, Ultraschall- oder Kernspintomographieaufnahmen liefern große Datenmengen, mit denen eine Diagnose erstellt und die Therapie geplant wird. Hierbei muß der Mediziner aus den zweidimensional präsentierten Bilddaten die dreidimensionale Anatomie erschließen. Um dabei Unterstützung zu geben, ist eine 3D-Rekonstruktion der anatomischen Strukturen von Nutzen. Unter einer 3D-Rekonstruktion versteht man den Vorgang, für eine anatomische Struktur ein dreidimensionales Modell aus den gegebenen Bilddaten abzuleiten. Diese dreidimensionalen Repräsentationen können genutzt werden, den Arzt bei der Diagnose und Therapieplanung zu unterstützen. Eine wichtige Voraussetzung dafür ist eine intuitive benutzerfreundliche Mensch-Maschine Schnittstelle. Um intuitiv durch computergenerierte dreidimensionale Szenen zu navigieren, werden Techniken aus dem Bereich der Virtuellen Realität (VR) eingesetzt. Seit 1993 werden in der Abteilung Visualisierung und Virtuelle Realität des IGD Anwendungen von Virtueller Realität (VR) entwickelt, die in unterschiedlichen Gebieten der Medizin eingesetzt werden können. Die Aktivitäten der Gruppe Medizin und VR liegen hauptsächlich in den folgenden Bereichen: - Anatomische Ausbildung - Chirurgisches Training - Operationsplanung In diesem Artikel werden einige aktuelle Projekte vorgestellt, die Möglichkeiten zeigen, VR in der Medizin einzusetzen.

Show publication details

Bockholt, Ulrich; Mlynski, Gunter; Müller, Wolfgang; Voss, Gerrit

Patient Specific Simulation of the Endonasal Airflow

1999

Kim, Myoung-Hee (Ed.) et al.: Advanced Medical Image Processing. Proceedings 1999. Darmstadt, 1999

Germany-Korea Joint Conference on Advanced Medical Image Processing <4, 1999, Darmstadt/Heidelberg>

Methods of Computational Fluid Dynamics (CFD) nowadays play an important role in the production process of the automotive industry. Progress made in recent years offers highly sophisticated airflow simulation models used in engineering for optimization and verification of aerodynamics. The key point of the Airflow Simulation Tool for Human Medical Application (ASTHMA) developed at the Darmstadt University of Technology in cooperation with the University Hospital in Greifswald is to use these techniques to support the rhinosurgeon in diagnosis and therapy planning. A system is developed that realizes a 3-D reconstruction of the endonasal cavaties based on computer tomography (CT) scans. This semiautomatic reconstruction method requires minimal manual intervention. The surface model is used to create an unstructured 3-D volume mesh suitable for Finite Volume simulations. In this way an individual simulation based on patient specific data can be realized. To verify the simulation result, experimental investigations, and measurements are made in nasal models at the University Hospital in Greifswald. The goal of this project is to investigate individual nasal complains and to detect respiratory disorders. Prior to performing a surgical procedure the surgeon should be able to simulate the disordered respiration and the optimized case expected after the intervention. In this way the effectiveness of surgical planing could be increased, resulting in a better surgical outcome.

Show publication details

Bockholt, Ulrich; Ecke, Ulrich; Müller, Wolfgang; Voß, Gerrit

Realtime Simulation of Tissue Deformation for the Nasal Endoscopy Simulator (NES)

1999

Westwood, James D. (Ed.) et al.: Medicine Meets Virtual Reality 1999. Proceedings : The Convergence of Physical and Informational Technologies: Options for a New Era in Healthcare. Amsterdam: IOS Press, 1999. (Studies in Health Technology and Informatics 62), pp. 74-75

Medicine Meets Virtual Reality (MMVR) <7, 1999, San Francisco, CA, USA>

Show publication details

Bockholt, Ulrich; Müller, Wolfgang; Voss, Gerrit; Ecke, Ulrich; Klimek, Ludger

Realtime Simulation of Tissue Deformation for the Nasal Endoscopy Simulator (NES)

1999

Computer Aided Surgery, Vol.4 (1999), 5, pp. 281-285

Endonasal sinus surgery requires a great amount of training before it can be adequately performed. The complicated anatomy involved, the proximity of relevant structures, and the variability of the anatomy due to inborn or iatrogenic variations make several complications possible. Today cadaver dissections are the "gold standard" for surgical training. To overcome the drawbacks of traditional training methods the Fraunhofer Institute for Computer Graphics is currently developing a highly interactive medical simulation system for nasal endoscopy and endonasal sinus surgery in cooperation with the Mainz University Hospital. For the simulation of a rhinoscopic procedure not only the realization of the 3-D interaction and the geometric representation of the anatomical structures is necessary, but also a real-time simulation of the deformation behavior constraint by the instrument collisions. The challenge is to close the gap between a maximal degree of realism and the needed real-time conditions.

Show publication details

Faißt, Andreas; Bockholt, Ulrich (Betreuer); Steinacker, Achim (Betreuer)

SteinEchtzeitsimulation eines chirurgischen Eingriffs mit dem Lasersystem zur Integration in einen Endoskopiesimulator

1999

Darmstadt, TU, Diplomarbeit, 1999

Show publication details

Bockholt, Ulrich; Mlynski, Gunter; Müller, Wolfgang; Voss, Gerrit

Using Simulation of the airflow in the human nose for rhinosurgical therapy planning

1999

Lemke, Heinz U. et al.: Computer Assisted Radiology and Surgery (CARS) '99. Proceedings. Amsterdam; Lausanne: Elsevier, 1999. (International Congress Series 1191), pp. 790-794

Computer Assisted Radiology and Surgery (CARS) <13, 1999, Paris, France>

Methods of Computational Fluid Dynamics (CFD) nowadays play an important role in the production process of the automotive industry. Progress made in recent years offers highly sophisticated airflow simulation models used in engineering for optimization and verification of aerodynamics. The key point of the Airflow Simulation Tool for Human Medical Application (ASTHMA) developed at the Darmstadt University of Technology in cooperation with the University Hospital in Greifswald is to use these techniques to support the rhinosurgeon in diagnosis and therapy planning [1]. A system is developed that realizes a 3-D reconstruction of the endonasal cavaties based on computer tomography (CT) scans. This semiautomatic reconstruction method requires minimal manual intervention. The surface model is used to create an unstructured 3-D volume mesh suitable for Finite Volume simulations. In this way an individual simulation based on patient specific data can be realized. To verify the simulation result, experimental investigations, and measurements are made in nasal models at the University Hospital in Greifswald. The goal of this project is to investigate individual nasal complains and to detect respiratory disorders. Prior to performing a surgical procedure the surgeon should be able to simulate the disordered respiration and the optimized case expected after the intervention. In this way the effectiveness of surgical planing could be increased, resulting in a better surgical outcome.

Show publication details

Müller, Wolfgang; Bockholt, Ulrich

The Virtual Reality Arthroscopy Training Simulator II

1998

Westwood, James D. et al.: Medicine Meets Virtual Reality 1998. Proceedings : Art, Science, Technology: Healthcare (R)Evolution TM. Amsterdam: IOS Press, 1998. (Studies in Health Technology and Informatics 50), pp. 13-19

Medicine Meets Virtual Reality (MMVR) <6, 1998, San Diego, CA, USA>

Arthroscopy has already become an irreplacable method in diagnostics. The arthroscope, with optics and light source, and the exploratory probe are inserted into the knee joint through two small incisions underneath the patella. Currently, the skills required for arthroscopy are taught through hands-on clinical experience. As arthroscopies became a more common procedure even in smaller hospitals, it became obvious that special training was necessary to guarantee qualification of the surgeons. On-the-job training proved to be insufficient. Therefore, research groups from the Berufsgenossenschaftliche Unfallklinik Frankfurt am Main approached the Fraunhofer Institute for Computer Graphics to develop a training system for arthroscopy based on virtual reality (VR) techniques. Two main issues are addressed: the three-dimensional (3-D) reconstruction process and the 3-D interaction. To provide the virtual environment, a realistic representation of the region of interest with all relevant anatomatical structures is required. Based on a magnetic resonance image sequence a realistic representation of the knee joint was obtained suitable for computer simulation. Two main components of the VR interface can be distinguished: the 3-D interaction to guide the surgical instruments and the 2-D graphical user interface for visual feedback and control of the session. Moreover, the 3-D interaction has to be realized by means of Virtual Reality techniques providing a simulation of an arthroscope and an intuitive handling of other surgical instruments. Currently, the main drawback of the developed simulator is the missing of haptic perception, especially of force feedback. In cooperation with the Department of Electro-Mechanical Construction at the Technical University Darmstadt a haptic display is designed and built for the VR arthroscopy training simulator. In parallel we developed a concept for the integration of the haptic display in a configurable way.

Show publication details

Müller, Wolfgang; Bockholt, Ulrich; Voß, Gerrit

Virtual Reality in Surgical Training

1998

Kim, Myoung-Hee et al.: Advanced Medical Image Processing. Proceedings 1998. Seoul, 1998, Session III, No. 5

Korea-Germany Joint Conference on Advanced Medical Image Processing <3, 1998, Seoul, Korea>

Arthroscopy has already become an irreplaceable method in diagnostics. The arthroscope, with optics and light source, and the exploratory probe are inserted into the knee joint through two small incisions underneath the patella. Currently, the skills required for arthroscopy are taught through hands-on clinical experience. As arthrocopies became a more common procedure even in smaller hospitals, it became obvious that special training was necessary to guarantee qualification of the surgeons. On-the-job training proved to be insufficient. Therefore, research groups from the Berufsgenossenschaftliche Unfallklinik Frankfurt am Main approached the Fraunhofer Institute for Computer Graphics to develop a training system for arthroscopy based on virtual reality (VR) techniques. Two main issues are addressed: the three-dimensional (3-D) recontruction process and the 3-D interaction. To provide the virtual environment, a realistic representation of the region of interest with all relevant anatomical structures is required. Based on a magnetic resonance image sequence a realistic representation of the knee joint was obtained suitable for computer simulation. Two main components of the VR interface can be distinguished: the 3-D interaction to guide the surgical instruments and the 2-D graphical user interface for visual feedback and control of the session. Moreover, the 3-D interaction has to be realized by means of VR techniques providing a simulation of an arthroscope and an intuitive handling of other surgical instruments. Currently, the main drawback of the developed simulator is the missing of haptic perception, especially of force feedback. In cooperation with the Department of Electro-Mechanical Construction at the Technical University Darmstadt a haptic display is designed and built for the VR arthroscopy training simulator. In parallel we developed a concept for the integration of the haptic display in a configurable way.