Schenk, Michael (Ed.): 14. IFF-Wissenschaftstage 2011. Tagungsband : Digitales Engineering und virtuelle Techniken zum Planen, Testen und Betreiben technischer Systeme. Stuttgart: Fraunhofer Verlag, 2011, pp. 191-196

Verteilte Mixed Reality Systeme, d.h. VR/AR-Installationen, die sich an unterschiedlichen Standorten befinden und via Internet vernetzt sind, wurden bislang nur in akademischen Testszenarien eingesetzt. Sie konnten den Sprung in industrielle Nutzung jedoch nicht vollziehen. Im Szenario "VR-Meeting" des BMBF-Projektes AVILUS wurde auf Grundlage des Virtual und Augmented Reality Systems "instantreality" (www.instantreality.de) ein kollaboratives VR-System entwickelt, mit dem Entwicklungsteams an verteilten Standorten in einer virtuellen Konstruktionssitzung zusammenarbeiten. Dabei wurden zum einen Verfahren für eine sichere Datenverteilung realisiert, zum anderen wurden Methoden zur effizienten Protokollierung von verteilten immersiven Konstruktionssitzungen erforscht.

BIO Web of Conferences, Vol.1 (2011), 4 p.

One of the research topics within the EU-project SKILLS1 was the training of Industrial Maintenance and Assembly (IMA) tasks. The IMA demonstrator developed comprehends two different training platforms, one based on technologies of Virtual Reality (VR) and the other one on Augmented Reality (AR). To qualify the efficiency of the developed training systems different studies have been conducted, followed by a final "Transfer of Skill" evaluation that has been performed by service technicians "SIDEL industrial training Entre" in Parma. This evaluation included qualitative methods (feedback collection in questionnaires) as well as quantitative methods (experiments with control groups). The results demonstrate that both platforms are useful and suitable training tools for IMA tasks, and that the AR training decreased the number of unsolved errors in the task.

Bender, Jan (Ed.) et al.: VRIPHYS 11: 8th Workshop in Virtual Reality Interactions and Physical Simulations. Goslar: Eurographics Association, 2011, pp. 73-82

A natural interaction with virtual environments is one of the key issues for the usability of Virtual Reality applications. Device-free, intuitive interactions with the virtual world can be achieved by capturing the movements of the user with markerless motion capture. In this work we present a markerless motion capture approach which can be used to estimate the human body pose in real-time with a single depth camera. The presented approach requires neither a 3D shape model of the tracked person nor a training phase in which body shapes are learned a priori. Instead, it analyzes the curvature of the human body to estimate the symmetry axes of the body joints. These symmetry axes are then used to calculate the pose of the tracked human in real-time. The presented approach was evaluated qualitatively with a time-of-flight and a Kinect depth camera. Furthermore, quantitative simulation results show that the proposed approach is promising for depth cameras which can reliably capture the surface curvature (and thus the normals) of a person and which have a resolution of at least 320x240 pixel.

BIO Web of Conferences, Vol.1 (2011), 4 p.

The capturing of human movements is an important step for the analysis of human skills, e.g. for sports analysis or for learning-by-demonstration tasks. In this paper we introduce a new markerless pose estimation method which estimates human poses from silhouettes. The presented numerical pose estimation algorithm adapts a non-deterministical annealing schedule for silhouette based motion capturing. The pose is estimated by numerically minimizing the differences between the silhouettes of synthesized views of a 3D avatar and the silhouettes of the real person in the camera images. The evaluation results of simulation experiments quantify the trade-off between the accuracy and the execution time of the presented algorithm.

Wiedmer, Martin (Ed.) et al.: 10th IEEE International Symposium on Mixed and Augmented Reality 2011. [USB Stick] : ISMAR. Science & Technology Proceedings, pp. 269-270

As large ships are never produced in masses, it often occurs that the construction process and production process overlap in time. Many shipbuilding companies have problems with discrepancies between the construction data and the real built ship. The assembly department often has to modify CAD data for a successful installation. We present an augmented reality system, where a user can visualize the construction data of pipes and modify these in the case of misalignment, collisions or any other conflicts. The modified pipe geometry can be stored and further used as input for CNC pipe bending machines. To guarantee an exactly orthogonal passage of the pipes through aligning bolt holes, we integrated an optical measurement tool into the pipe alignment process.

Alem, Leila (Ed.) et al.: Recent Trends of Mobile Collaborative Augmented Reality Systems. Berlin, Heidelberg, New York: Springer, 2011, pp. 69-82

Training for service technicians inmaintenance tasks is a classical application field of Augmented Reality explored by different research groups. Mostly technical aspects (tracking, visualization etc.) have been in focus of this research field. In this chapter we present results of interdisciplinary research based on the fusion of cognitive science, psychology and computer science. We focus on analyzing the improvement of Augmented Reality based training of skills which are relevant for maintenance and assembly tasks. The skills considered in this work comprise sensorimotor skills as well as cognitive skills. Different experiments have been conducted in order to find recommendations for the design of Augmented Reality training systems which overcome problems of existing approaches. The suggestions concern the fields of content visualization and multimodal feedback.

BIO Web of Conferences, Vol.1 (2011), 4 p.

Augmented Reality (AR) points out to be a good technology for training in the field of maintenance and assembly, as instructions or rather location-dependent information can be directly linked and/or attached to physical objects. Since objects to maintain usually contain a large number of similar components (e.g. screws, plugs, etc.) the provision of location-dependent information is vitally important. Another advantage is that AR-based training takes place with the real physical devices of the training scenario. Thus, the trainee also practices the real use of the tools whereby the corresponding sensorimotor skills are trained.

Shumaker, Randall (Ed.): Virtual and Mixed Reality - New Trends: Part I : International Conference, Virtual and Mixed Reality 2011. Berlin, Heidelberg, New York: Springer, 2011. (Lecture Notes in Computer Science (LNCS) 6773), pp. 123-132

As the complexity of maintenance tasks can be enormous, the efficient training of technicians in performing those tasks becomes increasingly important. Maintenance training is a classical application field of Augmented Reality explored by different research groups. Mostly technical aspects (e.g tracking, 3D augmentations) have been in focus of this research field. In our paper we present results of interdisciplinary research based on the fusion of cognitive science, psychology and computer science. We focus on analyzing the improvement of AR-based training of maintenance skills by addressing also the necessary cognitive skills. Our aim is to find criteria for the design of AR-based maintenance training systems. A preliminary evaluation of the proposed design strategies has been conducted by expert trainers from industry.

Avizzano, Carlo Alberto (Ed.) et al.: IEEE RO-MAN 2010 : 19th IEEE International Symposium on Robot and Human Interactive Communication. Proceedings [online]. New York: The Institute of Electrical and Electronics Engineers (IEEE), 2010, pp. 434-439

Augmented Reality (AR) is a technology which has become very popular in the last years. In this context also the idea of using of AR for training applications has become very important. AR offers a large potential for training only if the training is well focused to the skills that have to be trained and if the training protocol is well designed. On the other hand, the generation of the training content to be transferred via AR is a comprehensive problem that is addressed in this paper. Thus, this paper tries to describe the whole chain of implementations and general aspects involved in the creation of AR training applications, including examples for used multimodal devices. This chain starts with the capturing of expert actions to be hold in "digital representation of skill". The digital representation of skill is transferred to the training protocol that specifies the storyboard of the AR training session. The paper includes two different implementations of AR training systems and describes the general idea of informational abstraction from low level data up to interaction and from design to application.

Avizzano, Carlo Alberto (Ed.) et al.: IEEE RO-MAN 2010 : 19th IEEE International Symposium on Robot and Human Interactive Communication. Proceedings [online]. New York: The Institute of Electrical and Electronics Engineers (IEEE), 2010, pp. 422-427

This paper gives an overview of a multimodal training platform developed for minimally invasive robotic surgery, based on the DLR MiroSurge system. It describes the technological components and integration of the hardware and software platform and presents the first integrated training tasks that enable surgeons to get familiar with the robotic system. The training platform shares the same surgical operator workstation as MiroSurge and simulates the behaviour of the telemanipulator arms and the surgical instruments. Like the real system the training platform provides haptic feedback and 3D-vision. However instead of the real telemanipulator itself, a virtual environment with abstracted tasks is connected to the operator workstation. This allows reduction in costs, to provide various levels of difficulty, and to focus on the skills to be taught. Thus a training platform is presented that aims at training a surgeon's skills in handling the robotic system MiroSurge rather than training surgery in general.

Schenk, Michael (Ed.): 13. IFF-Wissenschaftstage 2010. Tagungsband : Digitales Engineering und virtuelle Techniken zum Planen, Testen und Betreiben technischer Systeme. Stuttgart: Fraunhofer Verlag, 2010, pp. 140-145

Um die globale Wettbewerbsfähigkeit nachhaltig zu sichern, die erhöhten technischen Anforderungen zu erfüllen und verkürzte Entwicklungszyklen bei gleichzeitig erhöhter Komplexität zu gewährleisten, ist der effiziente Einsatz von CAx- und virtuellen Technologien im Wertschöpfungsprozess für HDW unbedingte Voraussetzung. Dabei findet aufgrund von geringen Stückzahlen und hoher Individualität im U-Bootsbau eine enge Verzahnung von Fertigung und Entwicklung statt. Diese Verzahnung erfordert einen permanenten Abgleich zwischen digital konstruierten und real gefertigten Bauteilen. Um diese Anforderung zu unterstützen, werden im Rahmen des BMBF geförderten Projektes AVILUS Technologien entwickelt, die einerseits eine VR-gestützte Kooperation an verteilten Standorten unterstützten ("VRMeeting"), die andererseits aber mit Hilfe von Augmented Reality Technologien auch einen Abgleich von digitaler (Entwicklung) und realer Welt (Fertigung) unterstützen.

Lensch, Hendrik P. A. (Ed.) et al.: Eurographics 2010. Short Papers, pp. 49-52

Spectral rendering takes the full visible spectrum into account when calculating light-surface interaction and can overcome the well-known deficiencies of rendering with tristimulus color models. We present a variant of the precomputed radiance transfer algorithm that is tailored towards real-time spectral rendering on modern graphics hardware. Our method renders diffuse, self-shadowing objects with spatially varying spectral reflectance properties under distant, dynamic, full-spectral illumination. To achieve real-time frame rates and practical memory requirements we split the light transfer function into an achromatic part that varies per vertex and a wavelength-dependent part that represents a spectral albedo texture map. As an additional optimization, we project reflectance and illuminant spectra into an orthonormal basis. One area of application for our research is virtual design applications that require relighting objects with high color fidelity at interactive frame rates.

Institute for Systems and Technologies of Information, Control and Communication (INSTICC): VISIGRAPP 2010. Proceedings : International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications. INSTICC Press, 2010, pp. 165-172

Spectral rendering takes the full visible spectrum into account when calculating light-surface interaction and can overcome the well-known deficiencies of rendering with tristimulus color models. In this paper we show how to represent wavelength-dependent BRDFs as factored tensors. We use this representation for realtime spectral rendering on modern graphics hardware. Strong data compression and fast rendering times are achieved for mostly diffuse and moderately glossy isotropic surfaces. The method can handle high-resolution tabulated BRDFs, including non-reciprocal ones, which makes it well-suited for measured data. We analyze our approach numerically and visually. One area of application for our research is virtual design applications that require high color fidelity at interactive frame rates.

Gutiérrez, Teresa (Ed.) et al.: SKILLS 09 : Enaction on SKILLS. Donostia - San Sebastián: SKILLS Consortium, 2009, pp. 39-45

Within the European project SKILLS Virtual and Augmented Reality platforms have been developed that are used for the realization of different training simulators in the field of sports (training of rowing and juggling), surgery (training of maxillofacial and telerobotic surgery), rehabilitation (rehabilitation of upper limp), maintenance (training of assembly procedures) and robotics (training of robot programming). These platforms are integrating capturing and rendering technologies. The capturing technologies are used to register specific criteria of skills to be hold in a digital representation. Thus, parameters relevant for specific skills are qualified and used as evaluation factors within the training simulators. The training simulators not only can simulate the virtual environment of the training situation, but this environment can be enhanced via multimodal renderings, fostering the training process and accelerating the training effect. The integrative platforms from scalable and adaptable frameworks that can be used to realize different training protocols and that can be adapted to the trainee's degree of skill. Therefore the training protocol can be specified via standardized application scripts.

The 2nd Sino-German Workshop "Virtual Reality & Augmented Reality in Industry" : Invited Paper Proceedings. Participants Edition. Shanghai: Shanghai Jiao Tong University, 2009, pp. 78-83

Rapid development in processing power, graphic cards and mobile computers open up a wide domain for Mixed Reality applications. Thereby the Mixed Reality continuum covers the complete spectrum from Virtual Reality using immersive projection technology to Augmented Reality using mobile systems like Smartphones and UMPCs. At the Fraunhofer Institute for Computer Graphics (IGD) the Mixed Reality Framework instantreality (www.instantreality.org) has been developed as a single and consistent interface for AR/VR developers. This framework provides a comprehensive set of features to support classic Virtual Reality (VR) as well as mobile Augmented Reality (AR). The goal is to provide a very simple application interface which includes the latest research results in the fields of high-realistic rendering, 3D user interaction and total-immersive display technology. The system design is based on various industry standards to facilitate application development and deployment.

Gutiérrez, Teresa (Ed.) et al.: SKILLS 09 : Enaction on SKILLS. Donostia - San Sebastián: SKILLS Consortium, 2009, pp. 189-190

When a learner of a new procedural skill is provided with elaborated knowledge: "how-it-works" knowledge or "context procedures", in addition to the "how-to-do-it" knowledge or "list procedures", his performance becomes more accurate, faster, and more flexible. It is hypnotized that a display of the device which is assembled during the assembly task has the potential to support the trainee's mental model building.

Schenk, Michael (Ed.): 12. IFF-Wissenschaftstage 2009. Tagungsband : Digitales Engineering zum Planen, Testen und Betreiben technischer Systeme. Magdeburg: Fraunhofer IFF, 2009, pp. 105-112

Time-of-Flight Kameras sind im Gegensatz zu herkömmlichen Kameras in der Lage, nicht nur Farb- oder Graustufenwerte sondern auch Tiefeninformationen der aufgezeichneten Szene zu erfassen. Sie liefern für jedes erfasste Pixel neben einem Intensitätsbild auch Informationen über die Entfernung des jeweiligen Pixels und erfassen somit ein 2,5D Bild. Diese Tiefeninformationen können zur Rekonstruktion eines 3D-Modells einer Szene oder eines Objektes genutzt werden. Markerlose Tracking-Ansätze für Augmented Reality nutzen häufig Geometrieinformationen über die Szene, die mit virtuellen Zusatzinformationen überlagert werden soll. Mit Hilfe von Tiefenkameras wird es möglich, die vorliegende Szene zu rekonstruieren. Dafür wird die Szene aus verschiedenen Richtungen mit der Tiefenkamera erfasst und durch das Zusammenfügen der aus den einzelnen Bildern gewonnenen Tiefeninformationen automatisch rekonstruiert. Das so gewonnene 3D-Modell kann daraufhin bei markerlosen Augmented Reality Anwendungen zum Abgleich mit der jeweils aktuellen Sicht der Szene und somit zur Bestimmung der Kamerapose verwendet werden. Die über die getrackte Szene vorliegenden Geometrieinformationen sind dabei nicht auf den vor Beginn des Trackings erfassten Bereich beschränkt, sondern können auch während des Trackings dynamisch erweitert werden.

Journal of Electromyography and Kinesiology, Vol.19 (2009), 5, pp. e290-e300. First published as Article in Press 5 August 2008, DOI: 10.1016/j.jelekin.2008.05.005

The aim of this study was to investigate the cervicocephalic kinaesthesia of healthy subjects for gender and age effects and its reliability in a new virtual reality test procedure. 57 healthy subjects (30 male, 27 females; 18-64 years) were immersed into a virtual 3D scene via a headmounted display, which generated specific head movements. The joint repositioning error was determined in a static and dynamic test at the times T0, T1 (T0 + 10 minutes) and T2 (T0 + 24 hours). The intrasession reliability (T0-T1) and the intersession reliability (T0-T2) were analysed. In both tests no gender- or age-specific effects were found. In the overall group the means of the static test were 6.2 - 6.9 and of the dynamic test were 4.5 -4.9 . The intratest difference in the static test was -0.16 and the intertest difference was 0.47 . The intratest difference in the dynamic test was 0.42 and the intertest difference was 0.37 . The static and dynamic test was reproducible in healthy subjects, with minor deviations, irrespective of gender and age. The smaller interindividual differences in the dynamic test could be beneficial in the comparison of healthy individuals and individuals with cervical spine disorders.

Gutiérrez, Teresa (Ed.) et al.: SKILLS 09 : Enaction on SKILLS. Donostia - San Sebastián: SKILLS Consortium, 2009, pp. 153-161

This paper gives an overview of the components and integration of the prototype for a multimodal skills training system developed for Minimally Invasive Telerobotic Surgery (MITS), which is based on DLR's Miro-Surge system. It describes the technological components and integration of the Hardware and Software platform. Furthermore, it presents the first integrated training environment and assessment scheme. The MITS skills trainer consists of a surgical workstation, providing force and 3D visual feedback, which is identical with the real workstation and a VR environment, in which the robotic systems and the training environment is simulated, and which can be controlled depending on the training situation and difficulty level.

IEEE Systems, Man and Cybernetics Society: IEEE International Conference on Systems, Man and Cybernetics : SMC 2009. New York: IEEE Press, 2009, pp. 857-862

In recent years, the demand for efficient systems which can capture and learn human skills has become increasingly important. In this paper an approach for acquiring and recognizing human assembly skills is presented. The underlying workflows of assembly skills are captured by using a simple multisensor data glove and camera tracking. To avoid the processing of redundant information, at first the relevant tasks of a workflow are identified by analyzing measuring information of the multisensor capturing system. Thus, only relevant data is comprised in the representation of a workflow. Unlike common approaches a workflow is modeled as entire unit using a continuous Hidden Markov Model (HMM). The recognition process of input patterns is based on an adaptive threshold model that can identify known workflow patterns and non-meaningful input patterns as well.

IEEE Systems, Man and Cybernetics Society: IEEE International Conference on Systems, Man and Cybernetics : SMC 2009. New York: IEEE Press, 2009, pp. 565-570

This paper presents a simple yet effective approach for classification of human postures by using a time-of-flight camera. We investigate and adopt linear projection techniques such as Locality Preserving Projections (LPP), Linear Discriminant Analysis (LDA) and Principal Component Analysis (PCA), which are more widespread in face recognition and other pattern recognition tasks.We analyze the relations between LPP and LDA and show experimentally that using LPP in a supervised manner effectively yields very similar results as LDA, implying that LPP may be regarded as a generalization of LDA. Features for offline training and online classification are created by adopting common image processing techniques such as background-subtraction and blob detection to the time-of-flight data.

Schenk, Michael (Ed.): 11. IFF-Wissenschaftstage 2008. Tagungsband [CD-ROM]. Magdeburg: Fraunhofer IFF, 2008, pp. 31-37

Virtual-Reality Simulationen konnten sich in der chirurgischen Ausbildung etablieren, wobei die angebotenen Systeme von einfachen Trainingssimulatoren bis hin zu komplexen Chirurgiesimulatoren reichen. Diese Simulationen sind oft sehr überzeugend, häufig steckt allerdings ein großer Aufwand in der Aufbereitung von Trainingsfällen und der Generierung von Falldatenbanken. Die Herausforderung besteht hier also in der Vereinfachung der Generierung der Trainingsdatenbank. Eingangsdaten für die Trainingsdatenbank sind Tomographiedaten der Patienten, die als Volumendaten vorliegen. Mit dieser Motivation werden Volumenrendering-Methoden sowie Simulationsverfahren auf Volumendaten zur Verwendung in chirurgischen Trainingssimulatoren entwickelt und präsentiert. Die entwickelten Renderingverfahren betreffen sowohl visuelles als auch haptisches Rendering, da letzteres in endoskopischen Simulationen zur Vermittlung von chirurgischen Handfertigkeiten besonders wichtig ist. Die Trainingssimulatoren unterscheiden sich hierbei darin von anderen Trainingssystemen, dass die zu untersuchenden anatomischen Strukturen nicht exemplarisch modelliert werden sondern direkt aus Tomographiedaten rekonstruiert sind. Zur Evaluation der entwickelten Verfahren zum visuellen und haptischen Volumenrendering wurden Beispielanwendungen für die HNO-Medizin entwickelt.

Spencer, Stephen N. (Ed.): Proceedings WEB3D 2008 : 13th International Symposium on 3D Web Technology. New York: ACM Press, 2008, pp. 43-51

Nowadays medical training simulators play an important role in education and further training of surgeons. With Virtual Reality based training systems it is possible to simulate a surgery under realistic conditions. Input data for the visualization of anatomic structures is tomographic image data. For the visualization of medical datasets direct volume rendering is the method of choice. In this paper we introduce a system based on the new X3D extension proposal of the Medical Working Group for a volume rendering component, with some extensions for controlling the speed vs. quality trade-off. For a convincing and instructive training simulation, our implementation delivers high performance combined with high quality visualization of medical datasets. Furthermore, it integrates haptic forcefeedback devices to assure realistic interactions.

Mit dem Vorhabem "Virtuelles Kompetenznetzwerk zur Virtuellen und Erweiterten Realität" (ViVERA) wurde das Ziel verfolgt, Kompetenzen von VR- und AR-Entwicklern zu vernetzen, Entwicklungsbedarf zu identifizieren, Erfahrungen von Entwicklern und Anwendern zu bündeln und auf andere Anwendungsbereiche zu übertragen. Im Fokus standen hier die Branchen Automobil-, Maschinen-, Anlagen- und Schiffbau sowie die Medizintechnik. Innerhalb des Projektes konzentrierten sich die Forschungsarbeiten zunächst auf anwendungsübergreifende Basistechnologien. Im Ergebnis des Projektes entstanden aus der Kombination von Basistechnologien und einer anwendungsspezifischen Weiterentwicklung von neun prototypischen Demonstrantoren, die neue Einsatzszenarien von VR/AR-Technologien in den Zielbranchen aufzeigen. Die Demonstratoren stehen in den durchführenden Instituten zur Verfügung und bilden dort die Grundlage für weitere Entwicklungen nach Anforderungen von Industriepartnern, um diese in einen praktischen Einsatz zu überführen.

Buzug, Thorsten M. (Ed.): Advances in Medical Engineering. Berlin, Heidelberg, New York: Springer, 2007. (Springer Proceedings in Physics 114), pp. 145-150

A training and simulation system for therapy planning is developed based on patient specific imaging data. A real endoscope is used for navigation through the virtual patient. For this purpose sensors were built in the endoscope in order to track the translation, rotation and the angle of the distal end. Pre-processing (segmentation, tissue characterization) speeds-up the volume rendering up to real-time. Collision detection enables a realistic fly through the virtual patient.

Deutsche Gesellschaft für Biomedizinische Technik im VDE (DGBMT): BMT 2007. Proceedings [CD-ROM]. 41. Jahrestagung der Deutschen Gesellschaft für Biomedizinische Technik (DGBMT) im VDE : Biomedizinische Technik - Biomedical Engineering, Vol. 52 (2007) Ergänzungsband. Berlin: de Gruyter, 2007, 2 pp.

Auf Grundlage patientenspezifischer Bilddaten wurde ein Trainings- und Simulationssystem entwickelt, mit dem eine anstehende Therapie vorbereitet werden kann. Zur Navigation durch die Bilddaten wird ein echtes Endoskop benutzt. Hierzu wurden Sensoren im Endoskop eingebaut die die Translation, Rotation und den Winkel des distalen Endes abgreifen. Ein label file, generiert aus dem Segmentierungsergebnis, und ein Kollisisons Detektions Algorithmus unterstützen die Navigation indem die Position der relativen Sensoren bei Kollision angepasst wird.

Schenk, Michael (Ed.): 10. IFF-Wissenschaftstage 2007. Tagungsband [CD-ROM] : 15 Jahre Fraunhofer IFF. Magdeburg: Fraunhofer IFF, 2007, VR-Teil, pp. 193-200

Immer kürzer werdende Produktionszyklen und erhöhte Komplexitäten von Maschinen erfordern eine effektive und zeitsparende Entwicklungsumgebung, in der Simulation eine zunehmend wichtige Rolle spielt. In diesem Zusammenhang werden auch immer mehr Technologien aus dem Bereich der Virtuellen Realität (VR) eingesetzt, um Ergebnisse von CAE Prozessen in den verschiedenen Entwicklungsstufen zu visualisieren und um die nächsten Entwicklungsschritte zu diskutieren. Schwierigkeiten bereitet allerdings die Protokollierung der Absprachen und Diskussionsergebnisse. Dieses Paper beschreibt eine Annotationssystem, das die Erstellung, Integration und Protokollierung von Annotationen in Form von Texten, Skizzen, Bildern/Symbolen, Interaktionen, Audio- und Videoclips, sowie deren Darstellung auf zwei unterschiedliche Arten. Die Darstellungsformen der Annotationen basieren auf realen Metaphern, die in VR übertragen wurden.

Schulze, Thomas (Ed.) et al.: Simulation und Visualisierung 2006. Proceedings. Erlangen: SCS-European Publishing House, 2006, pp. 49-62

Funktionelle Störungen stellen die Ursache chronischer Schmerzen nach Schleudertrauma dar und verursachen immense Kosten im Gesundheitssystem. In diesem Verfahren wird durch den kombinierten Einsatz von Virtual Reality (VR) und Messungen der muskulären Aktivität die Diagnose chronischer Schmerzen nach Schleudertrauma verbessert. Die Diagnose resultiert in einer Kraftrückkopplung, die gezielt Kräfte auf den Kopf des Patienten übertragen soll. Hierzu wird der Patient mit Hilfe eines "Head Mounted Displays" (HMD) in eine virtuelle Welt versetzt. Der Patient verfolgt ein optisches Signal in der virtuellen Szene und wird dadurch animiert, spezifische Kopfbewegungen auszuführen. Durch das Einleiten fein dosierter Gegenkräfte soll die Nackenmuskulatur der Schmerzpatienten dabei, gesteuert durch VR Techniken, sensibel trainiert werden. Zeitgleich wird die bei den Bewegungen generierte elektrische Aktivität (EMG) des Musculus semispinalis capitis gemessen. Sie dient als Signal, um über einen Rückkopplungsmechanismus (Bio- Feedback) eine Überlastung des Patienten während der Therapie zu verhindern.

Schenk, Michael (Ed.): 9. IFF-Wissenschaftstage 2006. Tagungsband : Virtual Reality and Augmented Reality zum Planen, Testen und Betreiben technischer Systeme. Magdeburg: Fraunhofer IFF, 2006, pp. 163-170

Auf Grundlage patientenspezifischer Bilddaten wird ein Trainings- / Simulationssystem entwickelt, mit dem eine anstehende Therapie vorbereitet werden kann. Die untersuchten Fälle können gleichzeitig zum Aufbau einer Trainingsdatenbank herangezogen werden.

IEEE Computer Graphics and Applications, Vol.26 (2006), 6, pp. 22-23

Over the last fifteen years volume rendering have become an important part of examining, exploring and evaluating volume data. Huge medical datasets generated by Computed Tomography (CT) or Magnetic Resonance Imaging (MRI) have with volume rendering been made possible to explore in 3D. Even interaction with the data is today possible and it has been shown that this is of great value when diagnosing, practicing and/or learning. In order to make interactive volume rendering possible a range of sophisticated methods have been developed. Often high-end graphic hardware are applied when developing applications for 3D volume rendering. This is an obstacle when it comes to integrating the techniques into the normal hospital environment. It would be preferable if the applications are general enough to run on standard computers. In this thesis a volume renderer for standard computers with good visual result and a relatively high update rate, sufficient for interaction, is developed. The last years tremendous development of graphic hardware have made it possible to write programs, so called shaders, directly communicating with the graphic pipeline. The developed renderer employs the use of such shaders because of its superiority in speed to normal software rendering. One purpose for the renderer is that it will be used for exploring medical data as a part of a project between Fraunhofer Institut and the Hospital of Mainz.

Endoskopie heute, Vol.18 (2005), 3, pp. 105-107

Rechnergestützte Verfahren beginnen den medizinischen Alltag und die chirurgische Praxis zu revolutionieren. Insbesondere Techniken aus den Bereichen "Virtuelle Realität" und "Erweiterte Realität" bergen große Potentiale für medizinische Ausbildung, Diagnose, Operationsvorbereitung und intraoperative Unterstützung. Dabei versteht man unter "Virtueller Realität" eine 3D-Datenwelt, in die der Benutzer eintauchen und mit Hilfe neuer Interaktionsmechanismen intuitiv und effizient interagieren kann. Mit dem Begriff "Erweiterte Realität" wird die Technologie bezeichnet, in der die reale Umgebung mit computergenerierten Darstellungen bereichert und überlagert wird. Am Fraunhofer Institut für Graphische Datenverarbeitung werden medizinische Trainingssimulatoren entwickelt, mit denen angehende Chirurgen die Handhabung von Endoskop und chirurgischen Instrumenten üben. Darauf aufbauend wurden Projekte im Bereich der patientenspezifischen Therapieplanung durchgeführt: Dreidimensionale Modelle der anatomischen Strukturen können aus dem Tomographiedatensatz der Patienten rekonstruiert werden, mit Hilfe eines Planungssystems wird der bevorstehende Eingriff exakt spezifiziert. So wird sichergestellt, dass der optimale Ansatz für eine bevorstehende Operation vorbereitet wird. Die Ergebnisse der Operationsplanung werden mit Hilfe von Verfahren aus dem Bereich "Augmented Reality" intraoperativ visualisiert, um eine Navigationsunterstützung zu geben, die eine exaktes Umsetzen des Therapieplans garantiert.

Schulze, Thomas (Ed.) et al.: Simulation und Visualisierung 2005. Proceedings. Erlangen: SCS-European Publishing House, 2005, pp. 383-392

Rechnergestützte Verfahren beginnen den medizinischen Alltag und die chirurgische Praxis zu revolutionieren. Insbesondere Techniken aus den Bereichen "Virtuelle Realität und Erweiterte Realität" bergen große Potentiale für medizinische Ausbildung, Diagnose, Operationsvorbereitung und intraoperative Unterstützung. Dabei versteht man unter "Virtueller Realität" eine 3D-Datenwelt, in die der Benutzer eintauchen und mit Hilfe neuer Interaktionsmechanismen intuitiv und effizient interagieren kann. Mit dem Begriff "Erweiterte Realität" wird die Technologie bezeichnet, in der die reale Umgebung mit computergenerierten Darstellungen bereichert und überlagert wird. Am Fraunhofer Institut für Graphische Datenverarbeitung werden medizinische Trainingssimulatoren entwickelt, mit denen angehende Chirurgen die Handhabung von Endoskop und chirurgischen Instrumenten üben. Darauf aufbauend wurden Projekte im Bereich der patientenspezifischen Therapieplanung durchgeführt: Dreidimensionale Modelle der anatomischen Strukturen können aus dem Tomographiedatensatz der Patienten rekonstruiert werden, mit Hilfe eines Planungssystems wird der bevorstehende Eingriff exakt spezifiziert. So wird sichergestellt, dass der optimale Ansatz für eine bevorstehende Operation vorbereitet wird. Die Ergebnisse der Operationsplanung werden mit Hilfe von Verfahren aus dem Bereich "Augmented Reality" intraoperativ visualisiert, um eine Navigationsunterstützung zu geben, die eine exaktes Umsetzen des Therapieplans garantiert.

Schulze, Thomas (Ed.) et al.: Simulation und Visualisierung 2005. Proceedings. Erlangen: SCS-European Publishing House, 2005, pp. 383-392

Rechnergestützte Verfahren beginnen den medizinischen Alltag und die chirurgische Praxis zu revolutionieren. Insbesondere Techniken aus den Bereichen "Virtuelle Realität und Erweiterte Realität" bergen große Potentiale für medizinische Ausbildung, Diagnose, Operationsvorbereitung und intraoperative Unterstützung. Dabei versteht man unter "Virtueller Realität" eine 3D-Datenwelt, in die der Benutzer eintauchen und mit Hilfe neuer Interaktionsmechanismen intuitiv und effizient interagieren kann. Mit dem Begriff "Erweiterte Realität" wird die Technologie bezeichnet, in der die reale Umgebung mit computergenerierten Darstellungen bereichert und überlagert wird. Am Fraunhofer Institut für Graphische Datenverarbeitung werden medizinische Trainingssimulatoren entwickelt, mit denen angehende Chirurgen die Handhabung von Endoskop und chirurgischen Instrumenten üben. Darauf aufbauend wurden Projekte im Bereich der patientenspezifischen Therapieplanung durchgeführt: Dreidimensionale Modelle der anatomischen Strukturen können aus dem Tomographiedatensatz der Patienten rekonstruiert werden, mit Hilfe eines Planungssystems wird der bevorstehende Eingriff exakt spezifiziert. So wird sichergestellt, dass der optimale Ansatz für eine bevorstehende Operation vorbereitet wird. Die Ergebnisse der Operationsplanung werden mit Hilfe von Verfahren aus dem Bereich "Augmented Reality" intraoperativ visualisiert, um eine Navigationsunterstützung zu geben, die eine exaktes Umsetzen des Therapieplans garantiert.

Die Arbeit beschäftigt sich mit der Konzeption eines Systems zur Erstellung von Annotationen in immersiven Umgebungen unter der Verwendung VR spezifischer Interaktionsmetaphern. In diesem Zusammenhang wurde ein Annotationssystem entwickelt, das unter Verwendung VR spezifischer Interaktionsmetaphern die Dokumentation von Ergebnisprotokollen innerhalb der VR Umgebung unterstützt. Das System beinhaltet das Erstellen von Annotationen unterschiedlicher Art (Text, Skizze, Symbol, Audio, etc.), verschiedene Formen zur Darstellung der Anmerkungen, sowie das Erzeugen und Einlesen eines Protokolls über alle vorhandenen Annotationen. Das Konzept wurde innerhalb des VR-Systems AVALON implementiert und getestet.

Computer-assisted methods have emerged in medical diagnosis and therapy planning. Thereby the planning often relies on a 3D model of the anatomical structures reconstructed from medical image data (CT, MRT, US). Within this dissertation the diagnostic possibilities are enhanced by combining the model generation of with a simulation of the patient specific biomechanics. Not only image data also biomechanical parameters are considered in the simulation. The pathologic function can be analyzed within the simulation, this analysis is combined with a description of the planned intervention and it results in a prediction of the therapy outcome and the change of functionality caused by the planned intervention. Using this approach different biomechanical phenomena have been examined in different medical application fields: Static analyses have been performed to specify an optimal prosthesis for the implantation of an artificial knee joint, a simulation of the patient specific jaw movement helps to detect occlusions and to verify the design of onlays and crowns in dental diagnostics. Stress-strain simulation is applied in spine surgery and flow simulation has been used in ENT medicine to analyze the patient-specific air flow in the nose. For the presented approaches not only the diagnostic method is presented also an evaluation strategy is outlined that can be applied to proof the significance of the simulation. The presented work results in an approach registering biomedical parameters in real-time and considering these parameters in a Virtual Reality based method for whiplash therapy.

Encarnação, José L. (Honorary Chair) et al.: ACM SIGGRAPH International Conference on Virtual Reality Continuum and its Applications in Industry. Proceedings : ACM SIGGRAPH International Conference on Virtual Reality Continuum and its Applications in Industry. New York: ACM, 2004, pp. 352-354

In this paper we present the Augmented Reality Ocular, an extension of the traditional coin-operated binocular with augmented reality capabilities. The Augmented Reality Ocular enables a real implementation of the augmented reality paradigm and makes the technology available to a large public. After a review of the state of the art and a presentation of current related projects, we describe our approach of implementation. Finally different concept and purpose of applications are described in order to demonstrate the wide field of application of this new augmented reality device.

Chen, Jim (Ed.) et al.: IEEE Virtual Reality 2003 Conference. Proceedings. Los Alamitos, Calif.: IEEE Computer Society, 2003, pp. 97-101

Computer assisted operation planning systems win more and more recognition in the field of surgery. These systems offer new possibilities to prepare an intervention with the goal to shorten the expansive time in the operation room required for the intervention. The safest and most effective surgical approach should be selected. But often, it is difficult to transfer the output of the planning system to the intra-operative situation and so to consider the planning results in the real intervention. At the Fraunhofer Institute for Computer Graphics (IGD) in Darmstadt and the Centre for Advanced Media Technology (CAMTech) in Singapore, methods are developed to bridge the gap between the external planning session and the intra-operative case: Augmented Reality (AR) techniques are used to overlap preoperative scanned image data as well as results of the planning session to the operation field.

Current developments in surgery and computer science expose many new technologies and techniques. In the context of minimal invasive surgery, the surgeon is using very small instruments, which are inserted into patient's body through ports. The disadvantage of new surgical approaches like minimal invasive techniques is the more difficult intraoperative orientation for the surgeon, based on intraoperative sensors that deliver only a small part of the intraoperative situation. To assist the surgeon also preoperative data will be acquired. Those data can be used in combination with techniques of augmented reality to enhance the field of view of the surgeon. Registration aims to close the gap between the preoperative operating planning scenario and the intraoperative support by calculating the transformation that maps one data set onto the corresponding other. In the underlying medical scenario, the developed registration algorithm calculates the transformation that maps preoperative data onto intraoperative data. The main idea is not to calculate the transformation in a traditional way by building point correspondences on both data sets based on minimal distances, like the Iterative Closest Point-algorithm. The developed algorithm instead characterizes preoperative and intraoperative surface data to build the desired correspondences between both data sets. Those characteristics can be created for vertices for example and encodes the local neighborhood around such vertices. The integration of additional information like directed characteristics or histological information is elementary and was implemented to enhance the accuracy and performance of the algorithm.

International Journal of Computerized Dentistry, Vol.5 (2002), 2, pp. 101-106

The virtual articulator (VA) is intended for use as a tool for the analysis of the complex static and dynamic occlusal relations. Its chief application is in individual diagnosis and the simulation of the mechanical articulator. As entry data, the VA requires digital, 3-dimensional representations of the jaws and patient-specific data on jaw movement. It then simulates jaw movement and provides a dynamic visualization of the occlusal contacts. If no patient-specific data are available, the modus operandi of the mechanical articulator can be simulated. Virtual reality (VR) techniques applied in computer-aided diagnosis are revolutionizing medical routine in many disciplines. Particularly in the field of dental engineering, VR combined with CAD/CAM technology has great potential in implant planning and design in terms of greater precision and shorter treatment duration. The geometry of teeth can be digitized, the implant models can be designed with specialized CAD systems, and the final implants can be manufactured with a CNC machine. In this context the virtual articulator completes the digital manufacturing process.

Institute of Electrical and Electronics Engineers (IEEE): Sixth IEEE International Conference on Information Visualisation 2002. Proceedings. Los Alamitos, Calif.: IEEE Computer Society, 2002, pp. 600-602

The use of Virtual Reality (VR) techniques for Computer-Aided Diagnosis is revolutionizing the medical routing in various medical disciplines. Especially in the field of dentistry VR in combination with CAD/CAM technologies offers a high potential for implant planning/design resulting in a higher accuracy and a shorter therapy time period. The geometry of the teeth is recognisable and digitisable using an intraoral camera, the implant models can be designed with specialized CAD systems and the final implants are generated using a CNC machine. In this context the "Virtual Articualtor" (VA) completes the digital manufacturing pipeline and offers the possibility to consider patient-specific jaw biomechanics in the implant generation process. The objective of this project is the development of a system for dental occlusion diagnosis. For this important diagnosis in today's dental routine the mechanical articulator is used. The VA simulates and augments the functionalities of the mechanical articulator. Therefore it takes a digital 3-D representation of the jaws as input data, generates a simulation of the jaw movements, and delivers a dynamical visualization of the occlusion points. The high technical demands to the system are given through the high accuracy necessary for dental applications and the large models that have to be handled in the system.

Hutten, Helmut (Ed.) et al.: Second European Medical and Biological Engineering Conference. Proceedings Vol. 2 : Advancement of Medicine and Health Care through Technology - the Challenge to Biomedical Engineering in Europe. Graz: Verlag der Technischen Universität, 2002, pp. 1170-1171

The Dental Clinics of North America, Vol.9 (2002), 3, pp.493-506

Mills, John (Ed.): ITEC 2002. Europe's Training, Education and Simulation Conference Proceedings. CD-ROM. Lille, 2002, 4 p.

Rapid development of medical field, expanding knowledge base and new technologies require continuing medical education to achieve life long learning and to keep the surgeons up to date. Consequently, specific training is necessary to guarantee qualification of the surgeons. To overcome the current drawbacks of traditional training systems (on-the-job training, plastic models etc.) for laparoscopy/hysteroscopy an intelligent adaptable training environment has been realized using Virtual Reality (VR), Multimedia (MM) technology, and Intelligent Tutoring Systems (ITS).

Lemke, Heinz U. (Ed.) et al.: Computer Assisted Radiology and Surgery 2002. Proceedings. Berlin, Heidelberg, New York: Springer, 2002, pp. 107-112

Rapid development of medical field, expanding knowledge base and new technologies require continuing medical education to achieve life long learning and to keep the surgeons up to date. Consequently, specific training is necessary to guarantee qualification of the surgeons. To overcome the current drawbacks of traditional training systems (on - the - job training, plastic models etc.) for labaroscopy / hysteroscopy an intelligent adaptable training environment has been realized using Virtual Reality (VR), Multimedia (MM) technology, and Intelligent Tutoring Systems (ITS).

The International Society for Computer Assisted Orthopaedic Surgery: Computer Assisted Orthopaedic Surgery. Proceedings 2002, pp. 53-55

Westwood, James D. (Ed.): Medicine Meets Virtual Reality 2002. Proceedings : Digital Upgrades: Applying Moore's Law to Health. Amsterdam: IOS Press; Ohmsha, 2002. (Studies in Health Technology and Informatics 85), pp. 311-317

The lack of situed diagnostic tools providing insight into patient specific flow characteristics of the nasal airflow is one of the main problems in functional diagnosis. Diagnostics methods currently used do not provide the necessary information for flow analysis. But the flow distribution is essential for a physiological repiration, in particular for cleaning moistening and tempering of the inhaled air as well as for the olfactory function of the nose. To overcome this current situation a cooperation project of the ENT surgeons and computer graphic engineers was established to develop the computer assisted planning system STAN (Simulation Tool for Airflow in the human Nose) combining Computer Fluid Dynamics (CFD) with advanced Computer Graphic Technology. The idea of the STAN system is to perform patient specific airflow simulations in the patient's nasal cavities. Therefore a geometrical model of the nasal airways is derived form the patient's tomography scans. A discretization of the surrounded flow volume is made by a computational grid. To establish the flow simulation Finite Element Methods are performed on the grid. A tailored visualization is offered to the surgeon that overlaps the flow pattern to the patient's tomography data shown in the coronal sagittal and transversal plane. The surgeon can not only analyze the patient's current respiration situation he has also the possibility to describe the planned surgical intervention. The goal is to simulate the flow distribution that can be expected after the surgical intervention and to offer a possibility to validate various surgical strategies. To verify the simulation results experimental investigation and measurements are made in nasal models. Silicon Models of patient's nose channels are made to analyze flow characteristics The CT or MR scans of the same patients are used as input data for the simulation. The experimental outcome is compared to the simulation results to validate this diagnostic approach.

Institute of Electrical and Electronics Engineers (IEEE): Proceedings of the 10th IEEE International Workshop on Enabling Technologies: Infrastructure for Collaborative Enterprises. Los Alamitos, Calif.: IEEE Computer Society, 2001, pp. 269-272

Rapid development of medical field, expanding knowledge base and new technologies require continuing medical education to achieve life long learning and to keep the surgeons up to date. Consequently, specific training is necessary to guarantee qualification of the surgeons. To overcome the current drawbacks of traditional training systems (on-the-job training, plastic models etc.) for laparoscopy/hysteroscopy an intelligent adaptable training environment has been realised using Virtual Reality (VR), Multimedia (MM) technology, ans Intelligent Tutoring Systems (ITS)

In dieser Arbeit wird zunächst der aktuelle Stand der Technik im Bereich der Virtuellen Endoskopie vorgestellt, darauf aufbauend wird ein Konzept zur Generierung von virtuellen Endoskopien präsentiert und implementiert. Basierend auf der Technik des Keyframing wird die manuelle Planung von Kameraflügen und die anschließende Generierung von Animationen ermöglicht. Dazu können mittels eines Volumenrenderers Schlüsselpositionen für einen Kameraflug durch anatomischen Strukturen markiert und in die Keyframes übernommen werden. Mit Hilfe des Kochanek-Bartels-Splines wird eine geglättete Kurve, die nachträglich modifiziert werden kann, durch die Schlüsselpositionen gelegt und für die Berechnung der Kamerapositionen verwendet. Die Einführung einer flexiblen virtuellen Kamera ermöglicht in den Keyframes eine unabhängige Definition von Bewegungsrichtung, Geschwindigkeit und Orientierung und somit für den Anwender die Möglichkeit einer individuellen Steuerung. Für eine weiche Interpolation der Kamera-Orientierung wird ein Verfahren basierend auf einem Look-At-Spline vorgestellt. Die gezielte Festlegung der Kameraeigenschaften erlaubt eine wesentlich verbesserte Beobachtung relevanter Details und trägt so zu einer gesteigerten Qualität der Animationen bei. Um abrupte Änderungen der Kamera-Orientierung zu reduzieren wurde ein Verfahren entwickelt, das auf der Berechnung des Kamera-Koordinatensystems mit einem festen Aufwärtsvektor basiert. Jedem Keyframe kann außerdem ein Etikett zugeordnet werden, das sowohl die Verwendung der Liste der Keyframes erleichtert, als auch Informationen in einer Animation zur Verfügung stellt. Mit Hilfe der Keyframes können die Einstellungen der virtuellen Kamera für jedes Bild eines Kamerafluges berechnet werden (Inbetweening). Diese dienen als Grundlage für die Durchführung von Kontrollsequenzen und auch der Berechnung der fertigen Animation.

Westwood, James D. (Ed.): Medicine Meets Virtual Reality 2001. Proceedings : Outer Space, Inner Space, Virtual Space. Amsterdam: IOS Press; Ohmsha, 2001. (Studies in Health Technology and Informatics 81), pp. 336-340

Hysteroscopy has already become an irreplaceable method in gynaecoloic diagnosis and therapy. In the diagnostic case the hysteroscope with a 30° optic is insert transvaginally, in the therapeutic case the resctoscope with a 12° optic is used. The endoscoppic intervention requires special surgical skills for endoscope handling and remote instrument control. To acquire these skills currently hands-on training in clinical praxis has become standard, which is linked with higher danger for the women. To overcome current drawbacks of traditional training methods the European project LAHYSTOTRAIN was set up, that tries to combine Virtual Reality (VR), Multimedia (MM) technology, and Intelligent Tutoring Systems (ITS) to develop an alternative training system for hysteroscopic interventions. The first prototype of the LAHYSTOTRAIN demonstrator has been shown on several European conferences. An evaluation of the system was performed, with the idea, to collect feedback and impressions, that should be considered in further developments. This paper presents the LAHYSTOTRAIN prototype and the results of these evaluations.

Westwood, James D. (Ed.): Medicine Meets Virtual Reality 2001. Proceedings : Outer Space, Inner Space, Virtual Space. Amsterdam: IOS Press; Ohmsha, 2001. (Studies in Health Technology and Informatics 81), pp. 336-340

Hysteroscopy has already become an irreplaceable method in gynaecoloic diagnosis and therapy. In the diagnostic case the hysteroscope with a 30° optic is insert transvaginally, in the therapeutic case the resctoscope with a 12° optic is used. The endoscoppic intervention requires special surgical skills for endoscope handling and remote instrument control. To acquire these skills currently hands-on training in clinical praxis has become standard, which is linked with higher danger for the women. To overcome current drawbacks of traditional training methods the European project LAHYSTOTRAIN was set up, that tries to combine Virtual Reality (VR), Multimedia (MM) technology, and Intelligent Tutoring Systems (ITS) to develop an alternative training system for hysteroscopic interventions. The first prototype of the LAHYSTOTRAIN demonstrator has been shown on several European conferences. An evaluation of the system was performed, with the idea, to collect feedback and impressions, that should be considered in further developments. This paper presents the LAHYSTOTRAIN prototype and the results of these evaluations.

Westwood, James D. (Ed.): Medicine Meets Virtual Reality 2001. Proceedings : Outer Space, Inner Space, Virtual Space. Amsterdam: IOS Press; Ohmsha, 2001. (Studies in Health Technology and Informatics 81), pp. 336-340

Hysteroscopy has already become an irreplaceable method in gynaecoloic diagnosis and therapy. In the diagnostic case the hysteroscope with a 30° optic is insert transvaginally, in the therapeutic case the resctoscope with a 12° optic is used. The endoscoppic intervention requires special surgical skills for endoscope handling and remote instrument control. To acquire these skills currently hands-on training in clinical praxis has become standard, which is linked with higher danger for the women. To overcome current drawbacks of traditional training methods the European project LAHYSTOTRAIN was set up, that tries to combine Virtual Reality (VR), Multimedia (MM) technology, and Intelligent Tutoring Systems (ITS) to develop an alternative training system for hysteroscopic interventions. The first prototype of the LAHYSTOTRAIN demonstrator has been shown on several European conferences. An evaluation of the system was performed, with the idea, to collect feedback and impressions, that should be considered in further developments. This paper presents the LAHYSTOTRAIN prototype and the results of these evaluations.

Richir, Simon (Ed.) et al.: Virtual Reality International Conference (VRIC) 2001. Proceedings, pp. 225-233

Rapid developments in the medical field, as an expanding knowledge base and emerging new technologies require continuing medical education to achieve life long learning and to keep the surgeons up to date. Consequently, specific training is necessary to guarantee qualification of the surgeons. The goal of LAHYSTOTRAIN is to overcome the current drawbacks of traditional training methods for laparoscopic/ hysteroscopic procedures. A computer-assisted simulator for training and quality control in laparoscopy and hysteroscopy is developed using Virtual Reality (VR), Multimedia (MM) technology, and Intelligent Tutoring Systems (ITS).

The International Society for Computer Assisted Orthopaedic Surgery: Computer Assisted Orthopaedic Surgery. Proceedings 2001

Westwood, James D. (Ed.): Medicine Meets Virtual Reality 2001. Proceedings : Outer Space, Inner Space, Virtual Space. Amsterdam: IOS Press; Ohmsha, 2001. (Studies in Health Technology and Informatics 81), pp. 561-563

Robot Assisted Surgery (RAS) Systems win more and more recognition in the field of orthopaedics. Especially in Hip Surgery RAS has proved to be suited for application in medical routine. Often Robot Assisted Surgery Systems consist of a planning- and an interoperative component. According to specifications done with the planning software the tools are driven. Benefits of the robot assisted surgery should be higher precision and a better surgical outcome. In the co-operation project of several Fraunhofer Institutes "RoMed" (Robots and Manipulators for Medical Application) an exemplary application of robot aided spine surgery is developed. The planning software used in this context is proposed in the remaining article.

Die Finite Element Methode ist ein Verfahren, um das mechanische Verhalten von Materialien numerisch zu beschreiben. In den Ingenieurswissenschaften wird sie seit circa 20 Jahren eingesetzt und hat sich in der Forschung und der Praxis bewährt. Da in medizinischen Simulationen und Planungssystemen heute dazu übergegangen wird auch die Gewebeeigenschaften zu simulieren, bietet es sich an, mit der Finite Element Methode ein ausgereiftes Verfahren zu verwenden. Ziel dieser Arbeit ist die Entwicklung eines Verfahrens, mit dessen Hilfe die Belastung von Bandscheiben individuell, d.h. an patientenspezifischen Daten, simuliert werden kann. Dazu ist es nötig ein FEM-Gitter, beruhend auf patientenspezisischen Daten, zu erzeugen, ein Materialmodell zu entwickeln, das die Eigenschaften von Bandscheibengewebe nachbildet, und die Randbedingungen des gesamten Modells zu bestimmen. Anschließend soll die Integration in ein bestehendes Operationsplanungssystem möglich sein. Zur Erzeugung des FEM-Gitters wurden CT-Daten verwendet, aus denen zunächst ein dreidimensionales geometrisches Modell abgeleitet wurde. Aus diesem 3D-Modell wurde ein FEM-Gitter erzeugt, welches die Bedingungen erfüllt, die von den numerischen Algorithmen gestellt werden. Da die Anwendung der Finite Element Methode auf biologisches Gewebe bisher nicht weit verbreitet ist, mußten ein geeignetes Gewebemodell und die zugehörigen Parameter ausgewählt werden. Es wurden verschiedene Materialmodelle untersucht und schließlich das am besten geeignete, anisotrope Modell ausgewählt. Desweiteren wurden Randbedingungen festgelegt, die das Anbringen der Kräfte, die Lagerung des Gesamtmodells, bzw. die Kopplung der einzelnen Modellteile bestimmen. Zur Berechnung der auftretenden Kräfte wurde ein biomechanisches Modell eines Bewegungssegmentes der Wirbelsäule erstellt. Anhand dieses Modells wurden verschiedene Bewegungen bzw. Belastungen simuliert und die auftretenden Kräfte berechnet. Diese Belastungen dienten neben der Geometrie, der Randbedingungen und der Gewebeeigenschaften als Eingabegrößen für die verwendete FEM-Software (ANSYS). Mit Hilfe der Berechnungen konnten die Belastungen und deren Verteilung auf der Bandscheibenoberfläche berechnet werden. Die Ergebnisse können im Volumenrenderer visualisiert und mit den Tomographiedaten des Patienten überlagert werden. Das vorgestellte Verfahren erlaubt es, die Belastungen in den Bandscheiben zu quantifizieren und deren Verteilung darzustellen. Als Weiterentwicklung ist die Integration in ein Operationsplanungssystem denkbar, wodurch nicht nur der tatsächliche Zustand simuliert werden kann, sondern auch der potentielle Zustand nach einer Bandscheibenoperation unter Berücksichtigung der einsetztbaren Implantate. So könnte der Erfolg einer Operation im voraus besser abgeschätzt werden.

Lemke, Heinz U. (Ed.): Computer Assisted Radiology and Surgery (CARS) 2000. Proceedings. Amsterdam; Lausanne: Elsevier, 2000. (International Congress Series 1214), pp. 285-289

Computer assisted planning systems based on patient specific tomography data has been proofed to be useful in several medical fields. Patient's tomography data can be shown in several views and can be enhanced with additional information useful for the planing process. The STAN (Simulation Tool for Airflow in the human Nose) system overlaps patient data and CFD (computer fluid dynamics) data resulting from flow simulations of the airflow in the patient's nose channel. To verify the simulation results experimental investigations, and measurements are made in nasal models. Silicon Models of patient's nose channels are made to analyze flow characteristics. The CT or MR scans of the same patients are used as input data for the simulation. The experimental outcome is compared to the simulation results to validate this diagnostic approach.

Nanyang Technological University: Sixth International Conference on Control, Automation, Robotics and Vision (ICARCV 2000). Conference Proceedings : Intelligent Automation In The New Millenium

Total knee replacement (TKR) is a common orthopaedic surgical intervention and includes the removal of bone sections from the end of the femur and the top of the tibia for replacement by prosthetic components. Pain relief and functional improvement are predictable clinical results. But the accuracy of the alignment affects the surgical outcome and the longevity of the prosthesis. Hence, current total knee implantation systems attempt to align the knee joint in the mechanical axis for placement of the total knee components. These approaches use templates and plain radiographs for preoperative planning and alignment devices for bone cuts. To overcome the inherent inaccuracy of the presently used systems a computer-assisted planning system has been developed delivering the necessary control data for the intraoperative surgical robot system.

Der Radiologe, Vol.40 (2000), 3, pp. 290-294

Unzufriedenheit mit den traditionellen Ausbildungsmethoden in der endoskopischen Chirurgie führte zur Entwicklung eines Arthroskopietrainingssimulators, der Computergraphik und Techniken der Virtuellen Realität (VR) einsetzt. Basierend auf einem kernspintomographischen Bilddatensatz konnte mit Hilfe von Bildverarbeitungsmethoden und Visualisierungstechniken ein realistisches dreidimensionales Modell des Kniegelenks mit allen diagnostisch relevanten anatomischen Strukturen generiert und in den Simulator integriert werden. Die VR-Benutzer-Schnittstelle des Arthroskopietrainingssimulators besteht aus 2 Hauptkomponenten, der 3D-Interaktion, die das Führen der Instrumente realisiert, und der 2D-Benutzungsoberfläche, die das visuelle Feedback und die Kontrolle der Trainingsitzung ermöglicht. Auf diese Weise ist ein intuitives Handling der Instrumente garantiert. Der Chirurg ist in der Lage, eine arthroskopische Untersuchung an einen virtuellen Kniegelenk durchzuführen. Dazu sind auch erste Ansätze zur Simulation von Gewebeverformung implementiert, die aufgrund von Instrumenteinwirkungen auftreten.

Westwood, James D. (Ed.): Medicine Meets Virtual Reality 2000. Proceedings : Envisioning Healing: Interactive Technology and the Patient-Practitioner Dialogue. Amsterdam: IOS Press; Ohmsha, 2000. (Studies in Health Technology and Informatics 70), pp. 214-219

Total knee replacement (TKR) is a common orthopaedic surgical intervention and includes the removal of bone sections from the end of the femur and the top of the tibia for replacement by prosthetic components. Pain relief and functional improvement are redictable clinical results. But the accuracy of the alignment affects the surgical outcome and the longevity of the prosthesis. Hence, current total knee implantation systems attempt to align the knee joint in the mechanical axis for placement of the total knee components. These approaches use templates and plain radiographs for preoperative planning and alignment devices for bone cuts. To overcome the inherent inaccuracy of the presently used systems a computer-assisted planning system has been developed delivering the necessary control data for the intraoperative surgical robot system.

Centers for Medical Robotics and Computer Assisted Surgery (MRCAS) at Carnegie Mellon University and UPMC Shadyside Hospital: Computer Assisted Orthopaedic Surgery. Proceedings. Pittsburgh, 2000, pp. 105-107

Total knee replacement (TKR) is a common orthopaedic surgical intervention and includes the removal of bone sections from the end of the femur and the top of the tibia for replacement by prosthetic components. Pain relief and functional improvement are predictable clinical results. But the accuracy of the alignment affects the surgical outcome and the longevity of the prosthesis. Hence, current total knee implantation systems attempt to align the knee joint in the mechanical axis for placement of the total knee components. These approaches use templates and plain radiographs for preoperative planning and alignment devices for bone cuts. To overcome the inherent inaccuracy of the presently used systems a computer-assisted planning system has been developed delivering the necessary control data for the intraoperative surgical robot system.

Westwood, James D. (Ed.): Medicine Meets Virtual Reality 2000. Proceedings : Envisioning Healing: Interactive Technology and the Patient-Practitioner Dialogue. Amsterdam: IOS Press; Ohmsha, 2000. (Studies in Health Technology and Informatics 70), pp. 359-364

Rapid developments in the medical field, as an expanding knowledge base and emerging new technologies require continuing medical education to achieve life long learning and to keep the surgeons up to date. Consequently, specific training is necessary to guarantee qualification of the surgeons. The goal of LAHYSTOTRAIN is to overcome the current drawbacks of traditional training methods for laparoscopic/hysteroscopic procedures. A computer-assisted simulator for training and quality control in laparoscopy and hysteroscopy is developed using Virtual Reality (VR), Multimedia (MM) technology, and Intelligent Tutoring Systems (ITS).

Lemke, Heinz U. (Ed.): Computer Assisted Radiology and Surgery (CARS) 2000. Proceedings. Amsterdam; Lausanne: Elsevier, 2000. (International Congress Series 1214), pp. 290-293

Rapid development in the medical field, an expanding knowledge base and upcoming technologies require a continuing medical education to achieve life long learning and to keep the surgeons up to date. Consequently, specific training is necessary to guarantee the qualification of the surgeon. In recent years several training systems have been developed using Virtual Reality (VR) techniques to overcome the current drawbacks of traditional training systems (on-the-job training, plastic models etc.). In a co-operation project of University Hospital in Mainz (Department of Otalaryngology Head and Neck Surgery) and the Technical University in Darmstadt the Nasal Endoscopy Simulator (NES) is developed. Similar to a scenario in a computer game several tasks grouped into levels have to be solved during the training process. These levels are starting with simple tasks in abstract virtual environments to train the handling of the endoscope and they are ending with scenarios using complex models of the endonasal sinus to train orientation as well as diagnostic and therapeutic procedures. During a simulated interventions errors of the trainee, for example collisions of instruments and high sensitive tissues, are detected and evaluated. The integrative adaptable training environment provides new opportunities in medical education and enhances the medical standard curriculum. This concept has a general applicability to other medical fields.

Westwood, James D. (Ed.) et al.: Medicine Meets Virtual Reality 1999. Proceedings : The Convergence of Physical and Informational Technologies: Options for a New Era in Healthcare. Amsterdam: IOS Press, 1999. (Studies in Health Technology and Informatics 62), pp. 74-75

Kim, Myoung-Hee (Ed.) et al.: Advanced Medical Image Processing. Proceedings 1999. Darmstadt, 1999

Methods of Computational Fluid Dynamics (CFD) nowadays play an important role in the production process of the automotive industry. Progress made in recent years offers highly sophisticated airflow simulation models used in engineering for optimization and verification of aerodynamics. The key point of the Airflow Simulation Tool for Human Medical Application (ASTHMA) developed at the Darmstadt University of Technology in cooperation with the University Hospital in Greifswald is to use these techniques to support the rhinosurgeon in diagnosis and therapy planning. A system is developed that realizes a 3-D reconstruction of the endonasal cavaties based on computer tomography (CT) scans. This semiautomatic reconstruction method requires minimal manual intervention. The surface model is used to create an unstructured 3-D volume mesh suitable for Finite Volume simulations. In this way an individual simulation based on patient specific data can be realized. To verify the simulation result, experimental investigations, and measurements are made in nasal models at the University Hospital in Greifswald. The goal of this project is to investigate individual nasal complains and to detect respiratory disorders. Prior to performing a surgical procedure the surgeon should be able to simulate the disordered respiration and the optimized case expected after the intervention. In this way the effectiveness of surgical planing could be increased, resulting in a better surgical outcome.

Lemke, Heinz U. et al.: Computer Assisted Radiology and Surgery (CARS) '99. Proceedings. Amsterdam; Lausanne: Elsevier, 1999. (International Congress Series 1191), pp. 790-794

Methods of Computational Fluid Dynamics (CFD) nowadays play an important role in the production process of the automotive industry. Progress made in recent years offers highly sophisticated airflow simulation models used in engineering for optimization and verification of aerodynamics. The key point of the Airflow Simulation Tool for Human Medical Application (ASTHMA) developed at the Darmstadt University of Technology in cooperation with the University Hospital in Greifswald is to use these techniques to support the rhinosurgeon in diagnosis and therapy planning [1]. A system is developed that realizes a 3-D reconstruction of the endonasal cavaties based on computer tomography (CT) scans. This semiautomatic reconstruction method requires minimal manual intervention. The surface model is used to create an unstructured 3-D volume mesh suitable for Finite Volume simulations. In this way an individual simulation based on patient specific data can be realized. To verify the simulation result, experimental investigations, and measurements are made in nasal models at the University Hospital in Greifswald. The goal of this project is to investigate individual nasal complains and to detect respiratory disorders. Prior to performing a surgical procedure the surgeon should be able to simulate the disordered respiration and the optimized case expected after the intervention. In this way the effectiveness of surgical planing could be increased, resulting in a better surgical outcome.

Computer Aided Surgery, Vol.4 (1999), 5, pp. 281-285

Endonasal sinus surgery requires a great amount of training before it can be adequately performed. The complicated anatomy involved, the proximity of relevant structures, and the variability of the anatomy due to inborn or iatrogenic variations make several complications possible. Today cadaver dissections are the "gold standard" for surgical training. To overcome the drawbacks of traditional training methods the Fraunhofer Institute for Computer Graphics is currently developing a highly interactive medical simulation system for nasal endoscopy and endonasal sinus surgery in cooperation with the Mainz University Hospital. For the simulation of a rhinoscopic procedure not only the realization of the 3-D interaction and the geometric representation of the anatomical structures is necessary, but also a real-time simulation of the deformation behavior constraint by the instrument collisions. The challenge is to close the gap between a maximal degree of realism and the needed real-time conditions.

Biomedical Journal, (1999), 53, pp.4-6

In der modernen Medizin spielt die Verarbeitung medizinischer Bilddaten eine wichtige Rolle. Bildgebende Verfahren wie Röntgen-, Ultraschall- oder Kernspintomographieaufnahmen liefern große Datenmengen, mit denen eine Diagnose erstellt und die Therapie geplant wird. Hierbei muß der Mediziner aus den zweidimensional präsentierten Bilddaten die dreidimensionale Anatomie erschließen. Um dabei Unterstützung zu geben, ist eine 3D-Rekonstruktion der anatomischen Strukturen von Nutzen. Unter einer 3D-Rekonstruktion versteht man den Vorgang, für eine anatomische Struktur ein dreidimensionales Modell aus den gegebenen Bilddaten abzuleiten. Diese dreidimensionalen Repräsentationen können genutzt werden, den Arzt bei der Diagnose und Therapieplanung zu unterstützen. Eine wichtige Voraussetzung dafür ist eine intuitive benutzerfreundliche Mensch-Maschine Schnittstelle. Um intuitiv durch computergenerierte dreidimensionale Szenen zu navigieren, werden Techniken aus dem Bereich der Virtuellen Realität (VR) eingesetzt. Seit 1993 werden in der Abteilung Visualisierung und Virtuelle Realität des IGD Anwendungen von Virtueller Realität (VR) entwickelt, die in unterschiedlichen Gebieten der Medizin eingesetzt werden können. Die Aktivitäten der Gruppe Medizin und VR liegen hauptsächlich in den folgenden Bereichen: - Anatomische Ausbildung - Chirurgisches Training - Operationsplanung In diesem Artikel werden einige aktuelle Projekte vorgestellt, die Möglichkeiten zeigen, VR in der Medizin einzusetzen.

Westwood, James D. et al.: Medicine Meets Virtual Reality 1998. Proceedings : Art, Science, Technology: Healthcare (R)Evolution TM. Amsterdam: IOS Press, 1998. (Studies in Health Technology and Informatics 50), pp. 13-19

Arthroscopy has already become an irreplacable method in diagnostics. The arthroscope, with optics and light source, and the exploratory probe are inserted into the knee joint through two small incisions underneath the patella. Currently, the skills required for arthroscopy are taught through hands-on clinical experience. As arthroscopies became a more common procedure even in smaller hospitals, it became obvious that special training was necessary to guarantee qualification of the surgeons. On-the-job training proved to be insufficient. Therefore, research groups from the Berufsgenossenschaftliche Unfallklinik Frankfurt am Main approached the Fraunhofer Institute for Computer Graphics to develop a training system for arthroscopy based on virtual reality (VR) techniques. Two main issues are addressed: the three-dimensional (3-D) reconstruction process and the 3-D interaction. To provide the virtual environment, a realistic representation of the region of interest with all relevant anatomatical structures is required. Based on a magnetic resonance image sequence a realistic representation of the knee joint was obtained suitable for computer simulation. Two main components of the VR interface can be distinguished: the 3-D interaction to guide the surgical instruments and the 2-D graphical user interface for visual feedback and control of the session. Moreover, the 3-D interaction has to be realized by means of Virtual Reality techniques providing a simulation of an arthroscope and an intuitive handling of other surgical instruments. Currently, the main drawback of the developed simulator is the missing of haptic perception, especially of force feedback. In cooperation with the Department of Electro-Mechanical Construction at the Technical University Darmstadt a haptic display is designed and built for the VR arthroscopy training simulator. In parallel we developed a concept for the integration of the haptic display in a configurable way.

Kim, Myoung-Hee et al.: Advanced Medical Image Processing. Proceedings 1998. Seoul, 1998, Session III, No. 5

Arthroscopy has already become an irreplaceable method in diagnostics. The arthroscope, with optics and light source, and the exploratory probe are inserted into the knee joint through two small incisions underneath the patella. Currently, the skills required for arthroscopy are taught through hands-on clinical experience. As arthrocopies became a more common procedure even in smaller hospitals, it became obvious that special training was necessary to guarantee qualification of the surgeons. On-the-job training proved to be insufficient. Therefore, research groups from the Berufsgenossenschaftliche Unfallklinik Frankfurt am Main approached the Fraunhofer Institute for Computer Graphics to develop a training system for arthroscopy based on virtual reality (VR) techniques. Two main issues are addressed: the three-dimensional (3-D) recontruction process and the 3-D interaction. To provide the virtual environment, a realistic representation of the region of interest with all relevant anatomical structures is required. Based on a magnetic resonance image sequence a realistic representation of the knee joint was obtained suitable for computer simulation. Two main components of the VR interface can be distinguished: the 3-D interaction to guide the surgical instruments and the 2-D graphical user interface for visual feedback and control of the session. Moreover, the 3-D interaction has to be realized by means of VR techniques providing a simulation of an arthroscope and an intuitive handling of other surgical instruments. Currently, the main drawback of the developed simulator is the missing of haptic perception, especially of force feedback. In cooperation with the Department of Electro-Mechanical Construction at the Technical University Darmstadt a haptic display is designed and built for the VR arthroscopy training simulator. In parallel we developed a concept for the integration of the haptic display in a configurable way.