Ralf Gutbell

Darmstadt, TU, Master Thesis, 2019

In recent years there has been a lot of progress on the task of simultaneous localization and mapping (SLAM) of image sequences from monocular cameras. Latest methods utilize advancements in machine learning to improve the quality of both the camera tracking as well as the reconstruction of the environment. One of these methods called CNN-SLAM uses a neural network to estimate depth maps for keyframes and fuses them with stereo observations from neighboring image frames. To make use of the images within a geo-referenced context they first have to be localized globally. The use of additional sensors to determine the position and orientation of the camera is not only more expensive, but also sensitive to errors. This thesis proposes a real-time system that combines the methods of CNN-SLAM with image based localization within a simple city model. The SLAM algorithm tracks the camera movement while the use of a depth estimation network enables the recovery of the scale of the scene. A genetic algorithm is implemented to quickly refine estimated camera poses by aligning synthetic views of the city model with semantic segmentations of the images. This does not only localize the camera trajectories but also helps to compensate tracking errors caused by the SLAM algorithm. The evaluation showed the systems ability to compute scaled trajectories correctly, to compensate tracking drift and densly reconstruct the scene in the vicinity of the camera. It also revealed the unreliability of image localization without a constraint search space, tracking drift during rotational movement and inaccurate semantic segmentations.

Human Interface and the Management of Information. Interaction, Visualization, and Analytics

International Conference on Human Interface and the Management of Information (HIMI) <20, 2018, Las Vegas, NV, USA>

Lecture Notes in Computer Science (LNCS), 10904

Three-dimensional visualization of maps is becoming an increasingly important issue on the Internet. The growing computing power of consumer devices and the establishment of new technologies like HTML5 and WebGL allow a plug-in free display of 3D geo applications directly in the browser. Existing software solutions like Google Earth or Cesium either lack the necessary customizability or fail to deliver a realistic representation of the world. In this work a browser-based visualization component for geo-information is designed and a prototype is implemented in the gaming engine Unity3D. Unity3D allows translating the implementation to JavaScript and to embed it in the browser with WebGL. A comparison of the prototype with the opensource geo-visualization framework Cesium shows, that while maintaining an acceptable performance an improvement of the visual quality is achieved. Another reason to use a gaming engine as platform for our streaming algorithm is that they usually feature engines for physics, audio, traffic simulations and more, which we want to use in our future work.

Advanced Concepts for Intelligent Vision Systems

International Conference on Advanced Concepts for Intelligent Vision Systems (ACIVS) <18, 2017, Antwerp, Belgium>

Building recognition from images and video streams of mobile devices to texturize and refine an existing 3D city model is an open challenge, since such models most often do not completely represent the actual buildings. We present ways to extract buildings from images enabling improvement of the existing model. The approach is based on edge detection on images to detect walls, pure use of sensor data by creating an overlay to the video stream with the 3D model renderer from current position by a server, and the use of structure from motion algorithms to create point clouds and recognize a building via the support of the device's sensors. We show that we are thus able to texturize and refine an existing 3D city model.

Darmstadt, Hochschule, Master Thesis, 2016

Rendering of geodata is a challenging tasks, because the needed data is distributed on multiple institutions. The 3D Portrayal Service (3DPS) is the first realization of a standard, that is addressing this problem and which provides a solution for a combined visualization of distributed data through the use of the getView definition. The objective of this thesis is to design a framework that enables server site rendering through the getView operation interface, which is provided by the 3DPS standard. Therefore, the development of the framework concept follows the 3DPS standard principles. The framework is tested through the implementation of a prototype, which proves its functionality by simulating a defined use case. It could be shown that the framework concept can be used to implement a prototype, which realizes the getView operation. The framework could provide a solution for a combined visualization of distributed data.

Darmstadt, TU, Master Thesis, 2016

In this thesis, I investigate the problem of building recognition from images and video streams of mobile devices to texturize and refine an existing 3D city model. These city models have different origins and so different levels of detail. Existing approaches for this problem are analyzed and compared. Since no suitable approach was found, a new one has to be created. Supported by GPS and Gyroscope sensor data there are multiple possibilities to analyze images of the video stream on a 2D and 3D basis. Different ways to extract buildings from images are presented including model refinement. These are based on computer vision technologies such as edge detection on images to detect walls, pure use of sensor data by creating an overlay to the video stream with the 3D model renderer from current position by a server and use of structure from motion algorithms to create point clouds and extract a building from them. Each of the different detectors yields in successful results depending on different properties of a building such as e.g. the presence of adjoining buildings or accuracy of the 3D model. The edge detection modes yield to the highest accuracy, followed by the extraction from point clouds and pure sensor data. Also future work is presented to expand the shown approaches.

Proceedings Web3D 2015

International Conference on 3D Web Technology (WEB3D) <20, 2015, Heraklion, Crete, Greece>

We evaluate three Open-Source WebGL frameworks (X3DOM, three.js and Cesium) and investigate their use for geospatial applications in the Web. Over the course of one year we carried out five case studies and developed different software prototypes using these frameworks. We focus on geospatial applications since they require unique features that bring the evaluated WebGL frameworks to their limits. Geospatial data is typically heterogeneous and very large. Geospatial applications require special data management and interaction techniques. We describe the criteria that we defined to test the WebGL solutions and present our experience from working with them. We give a qualitative comparison and finish the paper with conclusions and an outlook on future research perspectives.

Darmstadt, TU, Bachelor Thesis, 2015

Dreidimensionale Kartenvisualisierungen nehmen einen stetig wachsenden Stellenwert im Internet ein. Neben der privaten Nutzung, können sie besonders bei der Planung von Bauprojekten eingesetzt werden, um diese interaktiv und visuell ansprechend zu präsentieren. Steigende Leistungsfähigkeit der Endgeräte und die Etablierung neuer Technologien wie HTML5 und WebGL erlauben dabei eine Plug-in freie Darstellung von 3D Anwendungen direkt im Browser. Bestehenden Softwarelösungen, wie Google Earth oder Cesium, mangelt es entweder an der benötigten Anpassbarkeit oder der realistischen Darstellung der Welt. In dieser Arbeit wird eine browserbasierte Visualisierungskomponente für Geoinformationen konzipiert und ein Prototyp in der Spiele-Engine Unity3D implementiert. Unity5 ermöglicht es die Implementierung in Javascript zu übersetzen und mit WebGL im Browser darzustellen. Ein Vergleich des Prototypen mit Cesium zeigt, dass eine Verbesserung der visuellen Qualität bei gleichzeitig akzeptabler Performance erreicht wird.

Clinical Image-Based Procedures. From Planning to Intervention

International Workshop on Clinical Image-based Procedures (CLIP) <1, 2012, Nice, France>

We investigate a multi-port minimally-traumatic approach for lateral skull base surgery. Our long-term goal is to automatically determine the best combination of three paths from the skull surface to the surgical target. Since this multi-port approach is not yet in clinical use, it is difficult to define what the best combination of paths is. Therefore, we present a planning tool which allows to manually selecting a combination of three paths. A clinician used this planning tool to select the best combination of three paths for three target structures in computed tomography data sets of 20 patients. From this choice we derived initial constraints for an automatic planning of the best combination of paths.

Bildverarbeitung für die Medizin 2012

Workshop Bildverarbeitung für die Medizin (BVM) <15, 2012, Berlin, Germany>

Bei Operationen an der Otobasis ist es wichtig, umliegende Risikostrukturen wie Gesichtsnerv oder Blutgefäße nicht zu verletzen. Bisher legt der Arzt dazu alle Strukturen frei. Unser Forschungsprojekt untersucht nun einen minimal-invasiven Multi-Port-Ansatz. Im Rahmen dieses Projektes haben wir basierend auf SOFA einen Prototypen zur Planung von Bohrpfaden an der Otobasis entwickelt. Damit kann der Arzt sowohl einen Bohrkanal genauer analysieren als auch sich die Menge aller zulässigen Pfade anzeigen lassen, aus der er schließlich die besten Bohrpfade für die Operation auswählt. Dies ist ein erster Schritt, den Arzt bei der Planung von mehreren Bohrkanälen zu unterstützen.

Darmstadt, TU, Master Thesis, 2011

Research in the field of minimally-invasive surgery is important, because it will allow surgeries at lower risk levels and will help to safe money and time as well. Several critical structures are located at the lateral skull base, so the minimally-invasive approach has not been performed in this area yet. Small errors in the drill placement could lead to severe complications for the patient. In this thesis a prototype for multi-port planning is developed, which models the critical structures and considers the placement inaccuracy of the drill. It also allows the surgeon to determine the drill's size and a minimal distance to the critical structures, which the drill paths have to respect. A prototype has been developed, which employs and extends the medical simulation framework SOFA. New models have been created, describing the drill paths and collision components, enabling the collision detection and its visualization. Another feature of the prototype is the option of simulating the drill feed for a single drill path. The simulation models the drill and its movement from the entry point to the surgical target. This option allows the user assess the spatial relation between the instrument and the critical structures. The prototype's visualization features, its results and performance are analyzed for different drill settings, datasets and computer systems. The drill settings are combinations of drill sizes, minimal distance and drill placement inaccuracies. The evaluation shows that the created collision model has been successfully integrated in SOFA's collision pipeline. The implemented visualization features adequately represents the collisions and drill path parameters. The prototype's performance is limited and needs to be improved. Different solutions to achieve that and possible new features are presented.

Darmstadt, TU, Bachelor Thesis, 2009

In der medizinischen Visualisierung werden Ärzte oft mit großen Datensätzen konfrontiert aus denen sie ihre Diagnose stellen müssen. Die anatomische Struktur, welche untersucht werden soll, ist dabei meist von anderen anatomischen Elementen verdeckt und erschwert dem Arzt die Arbeit. Mehrere Arbeitsgruppen, wie zum Beispiel Luft et al. und Ebert et al., beschäftigen sich mittlerweile damit, die Volumen optisch, mit Hilfe von bekannten menschlichen Wahrnehmungsmustern, aufzubereiten, so dass dem Arzt bei der Diagnose geholfen wird. Die vorliegende Arbeit nutzt bekannte Aspekte der menschlichen Wahrnehmungen, wie den Fokus des Auges, um dem Arzt optische Hilfen zu geben. Dabei wird hauptsächlich, abhängig von einem Distanzmaß, die Opazität und Schärfe von Voxeln manipuliert. Die Bestimmung des Fokus geschieht durch den Benutzer, indem er den Fokus im Volumen markiert. Zusätzlich zum Fokus bestimmt der Benutzer noch einen zweiten Radius, welcher als äußerer Rand benutzt wird, um zu bestimmen wie weit außerhalb des Fokus das Volumen noch sichtbar sein soll. Zwischen dem Fokus und dem Radius werden zusätzlich zur manipulierten Opazität die Voxel auch noch verschmiert, um den Effekt zu verstärkten. Die Intensität des Blurrings kann wiederum vom Benutzer gesteuert werden. Die Evaluation mit mehreren verschiedenen Datensätzen ergab, dass der Ansatz ein mächtiges Werkzeug zur Untersuchung sein kann. Die Benutzbarkeit der momentanen Umsetzung ist noch stark von der Größe des Volumens abhängig, da sonst zu hohe Wartezeiten auftreten. Auch sehr homogene Volumen sind eine Schwäche der jetzigen Umsetzung, welche aber lösbare Probleme darstellen und deren Umsetzung Teil der zukünftigen Arbeiten sein wird.