Liste der Fachpublikationen

Ergebnisse 1 - 90 von 90
Show publication details
Hegde, Chaitra; Kuijper, Arjan [1. Gutachter]; Dennstädt, Marco [2. Gutachter]

A Distributed Task Scheduler for Cuttlefish::Web to Optimize the Cost and Runtime

2019

Darmstadt, TU, Master Thesis, 2019

The ability to mass produce customized products by additively layering materials has placed 3D printing in the spotlight of the manufacturing industry. Cuttlefish is a 3D printer driver which generates printable files from a 3D mesh. When executed at scale, the driver consumes a considerable amount of computing resources. This highlights the need for a distributed system that is capable of efficiently scaling up or down depending on the type of input while operating under cost and time constraints. Through this master thesis, an intelligent task scheduler which runs print jobs on suitable computers and optimizes cost and runtime based on the user’s preference is implemented. Several Machine Learning algorithms are evaluated to build the system classification and running time prediction models, and the best performing model is deployed as a service. The realized architecture highlights methods to develop an Intelligent task scheduler. They also form a baseline for Cuttlefish::Web to be used on Cloud Infrastructure.

Show publication details
Ma, Jingting; Wang, Anqi; Lin, Feng; Wesarg, Stefan; Erdt, Marius

A novel robust kernel principal component analysis for nonlinear statistical shape modeling from erroneous data

2019

Computerized Medical Imaging and Graphics

Statistical Shape Models (SSMs) have achieved considerable success in medical image segmentation. A high quality SSM is able to approximate the main plausible variances of a given anatomical structure to guide segmentation. However, it is technically challenging to derive such a quality model because: (1) the distribution of shape variance is often nonlinear or multi-modal which cannot be modeled by standard approaches assuming Gaussian distribution; (2) as the quality of annotations in training data usually varies, heavy corruption will degrade the quality of the model as a whole. In this work, these challenges are addressed by introducing a generic SSM that is able to model nonlinear distribution and is robust to outliers in training data. Without losing generality and assuming a sparsity in nonlinear distribution, a novel Robust Kernel Principal Component Analysis (RKPCA) for statistical shape modeling is proposed with the aim of constructing a low-rank nonlinear subspace where outliers are discarded. The proposed approach is validated on two different datasets: a set of 30 public CT kidney pairs and a set of 49 MRI ankle bones volumes. Experimental results demonstrate a significantly better performance on outlier recovery and a higher quality of the proposed model as well as lower segmentation errors compared to the state-of-the-art techniques.

Show publication details
Zhou, Tong-xue; Zeng, Dong-dong; Kuijper, Arjan

A Template Consensus Method for Visual Tracking

2019

Optoelectronics Letters

Visual tracking is a challenging problem in computer vision. Recently, correlation filter-based trackers have shown to provide excellent tracking performance. Inspired by a sample consensus approach proposed for foreground detection, which classifies a given pixel as foreground or background based on its similarity to recently observed samples, we present a template consensus tracker based on the kernelized correlation filter (KCF). Instead of keeping only one target appearance model in the KCF, we make a feature pool to keep several target appearance models in our method and predict the new target position by searching for the location of the maximal value of the response maps. Both quantitative and qualitative evaluations are performed on the CVPR2013 tracking benchmark dataset. The results show that our proposed method improves the original KCF tracker by 8.17% in the success plot and 8.11% in the precision plot.

Show publication details
Cheng, Wentao; Chen, Kan; Lin, Weisi; Goesele, Michael; Zhang, Xinfeng; Zhang, Yabin

A Two-stage Outlier Filtering Framework for City-Scale Localization using 3D SfM Point Clouds

2019

IEEE Transactions on Image Processing

3D Structure-based localization aims to estimate the 6-DOF camera pose of a query image by means of feature matches against a 3D Structure-from-Motion (SfM) point cloud. For city-scale SfM point clouds with tens of millions of points, it becomes more and more difficult to disambiguate matches. Therefore a 3D Structure-based localization method, which can efficiently handle matches with very large outlier ratios, is needed. We propose a two-stage outlier filtering framework for city-scale localization that leverages both visibility and geometry intrinsics of SfM point clouds. Firstly, we propose a visibility-based outlier filter, which is based on a bipartite visibility graph, to filter outliers on a coarse level. Secondly, we apply a geometry-based outlier filter to generate a set of fine-grained matches with a novel data-driven geometrical constraint for efficient inlier evaluation. The proposed two-stage outlier filtering framework only relies on intrinsic information of a SfM point cloud. It is thus widely applicable to be embedded into existing localization approaches. The experimental results on two real-world datasets demonstrate the effectiveness of the proposed two-stage outlier filtering framework for city-scale localization

Show publication details
Araslanov, Nikita; Rothkopf, Constantin A.; Roth, Stefan

Actor-Critic Instance Segmentation

2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition

IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) <2019, Long Beach, California, USA>

Most approaches to visual scene analysis have emphasised parallel processing of the image elements. However,one area in which the sequential nature of vision is apparent, is that of segmenting multiple, potentially similar andpartially occluded objects in a scene. In this work, we revisit the recurrent formulation of this challenging problemin the context of reinforcement learning. Motivated by thelimitations of the global max-matching assignment of theground-truth segments to the recurrent states, we developan actor-critic approach in which the actor recurrently predicts one instance mask at a time and utilises the gradientfrom a concurrently trained critic network. We formulatethe state, action, and the reward such as to let the criticmodel long-term effects of the current prediction and incorporate this information into the gradient signal. Furthermore, to enable effective exploration in the inherentlyhigh-dimensional action space of instance masks, we learna compact representation using a conditional variationalauto-encoder. We show that our actor-critic model consistently provides accuracy benefits over the recurrent baselineon standard instance segmentation benchmarks.

Show publication details
Han, Xiyu; Lv, Tao; Song, Xiangyu; Nie, Ting; Liang, Huaidan; He, Bin; Kuijper, Arjan

An Adaptive Two-scale Image Fusion of Visible and Infrared Images

2019

IEEE Access

In this paper, we proposed an adaptive two-scale image fusion method using latent low-rank representation (LatLRR). Firstly, both IR and VI images are decomposed into a two-scale representation using LatLRR to generate low-rank parts (the global structure) and saliency parts (the local structure). The algorithm denoises at the same time. Then, the guided filter is used in the saliency parts to make full use of the spatial consistency, which reduces artifacts effectively. With respect to the fusion rule of the low-rank parts, we construct adaptive weights by adopting fusion global-local-topology particle swarm optimization (FGLT-PSO) to obtain more useful information from the source images. Finally, the resulting image is reconstructed by adding the fused low-rank part and the fused saliency part. Experimental results validate that the proposed method outperforms several representative image fusion algorithms on publicly available datasets for infrared and visible image fusion in terms of subjective visual effect and objective assessment.

Show publication details

An Experimental Overview on Electric Field Sensing

2019

Journal of Ambient Intelligence and Humanized Computing

Electric fields exist everywhere. They are influenced by living beings, conductive materials, and other charged entities. Electric field sensing is a passive capacitive measurement technique that detects changes in electric fields and has a very low power consumption. We explore potential applications of this technology and compare it to other measurement approaches, such as active capacitive sensing. Five prototypes have been created that give an overview of the potential use cases and how they compare to other technologies. Our results reveal that electric field sensing can be used for indoor applications as well as outdoor applications. Even a mobile usage is possible due to the low energy consumption of this technology.

Show publication details
Montefiori, Erica; Modenese, Luca; Di Marco, Roberto; Magni-Manzoni, Silvia; Malattia, Clara; Petrarca, Maurizio; Ronchetti, Anna; Tanturri De Horatio, Laura; Dijkhuizen, Pieter van; Wang, Anqi; Wesarg, Stefan; Viceconti, Marco; Mazzà, Claudia

An Image-based Kinematic Model of the Tibiotalar and Subtalar Joints and its Application to Gait Analysis in Children with Juvenile Idiopathic Arthritis

2019

Journal of Biomechanics

In vivo estimates of tibiotalar and the subtalar joint kinematics can unveil unique information about gait biomechanics, especially in the presence of musculoskeletal disorders affecting the foot and ankle complex. Previous literature investigated the ankle kinematics on ex vivo data sets, but little has been reported for natural walking, and even less for pathological and juvenile populations. This paper proposes an MRI-based morphological fitting methodology for the personalised definition of the tibiotalar and the subtalar joint axes during gait, and investigated its application to characterise the ankle kinematics in twenty patients affected by Juvenile Idiopathic Arthritis (JIA). The estimated joint axes were in line with in vivo and ex vivo literature data and joint kinematics variation subsequent to inter-operator variability was in the order of 1°. The model allowed to investigate, for the first time in patients with JIA, the functional response to joint impairment. The joint kinematics highlighted changes over time that were consistent with changes in the patient’s clinical pattern and notably varied from patient to patient. The heterogeneous and patient-specific nature of the effects of JIA was confirmed by the absence of a correlation between a semi-quantitative MRI-based impairment score and a variety of investigated joint kinematics indexes. In conclusion, this study showed the feasibility of using MRI and morphological fitting to identify the tibiotalar and subtalar joint axes in a non-invasive patient-specific manner. The proposed methodology represents an innovative and reliable approach to the analysis of the ankle joint kinematics in pathological juvenile populations.

Show publication details
Lian, Runze; Kuijper, Arjan [Gutachter]; Fu, Biying [Betreuerin]

Anomaly Detection and probable path prediction for Single and Multiperson-Application in Smart Homes

2019

Darmstadt, TU, Master Thesis, 2019

The most popular outdoor positioning system, global positioning system (GPS), does not perform well in indoor environment. Because this system primarily depends on the signal propagation in the air and complex architecture of buildings will interfere with signal propagation, i.e., its indoor positioning performance will be limited by the line-of-sight nature. While the drawbacks of GPS, other indoor positioning techniques (such as Wi-Fi based, RFID based) can provide Location-based-service (LBS) for various applications, which make our life comfortable and smart. As one kind of these sensing and positioning techniques, the passive Electric Field Sensing has numerous advantages compared to the others, e.g., lower power consumption and no personal information and specific positioning tokens required. So it is applied in our Smart Floor system to position and track movement for users, which mainly aims at the elderly care. On the other hand, a passive EFS-based positioning system might be susceptible to disturbance, due to the aliasing effect and noises from environment. To address this problem, I studied and investigated the Anomaly Detection issue in the Machine Learning domain, which aims at discovering proper ML algorithms to improve the positioning and movement prediction performance of our Smart Floor system. In this thesis, I proposed a novel ML algorithm for this goal, namely the Dictionary-based Anomaly Detection Algorithm. Compared with other existing algorithms, this dictionary algorithm exploits not only the normal data but also coupling outliers to obtain our desired results, i.e. indoor positions of users. Furthermore, combining with a customized positioning scheme relying on anchor points, the Dictionary-based indoor Positioning and Movement Prediction approach preformed well in our living laboratory. Moreover as discussion and expansion, the Dictionary-based Anomaly Detection Algorithm is especially practicable in application scenarios where a large amount of outliers and normal data are always at the same time observed.

Show publication details
Wagner, Tim Geronimo; Kuijper, Arjan [1. Prüfer]; Siegmund, Dirk [2. Prüfer]

Applied Food Recognition for Vision-Based Self-Checkout Systems

2019

Darmstadt, TU, Bachelor Thesis, 2019

Food recognition has been around almost as long as object detection itself. However, it is still an immensely complicated task due to the nature of food. The same food can come in different shapes, colors and arrangements. In contrast, different food can look almost identical. Therefore, it is crucial to find efficient systems to evaluate images of food. The main goal of this thesis is to provide a way for canteens to use self-checkout systems. It should be able to identify and differentiate between food items on the basis of pictures of the food. Then, a total price should be calculated and a method of payment is provided. This report solved these problems by building on a master’s thesis [3]. We collected a database that was used to train two neural networks. A CNN based on the Inception architecture achieved equal error rate losses of 9% and is responsible for identifying the main dishes on the user’s tray. Another Faster R-CNN was set up to identify side components with a precision of 99.98%. A prototype was set up that is able to classify food on canteen trays from two images. It was equipped with an efficient camera setup and interfaces to a back-end server that handles classifications. The system is ready to be used in canteens. The following report will describe how the system works and which steps have been taken in order to achieve the given accuracies.

Show publication details

AR Tracking with Hybrid, Agnostic And Browser Based Approach

2019

2019 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR 2019). Proceedings

IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR) <2, 2019, San Diego, CA>

Mobile platform tools are desirable when it comes to practical augmented reality applications. With the convenience and portability that the form factor has to offer, it lays an ideal basic foundation for a feasible use case in industry and commercial applications. Here, we present a novel approach of using the monocular Simultaneous Localization and Mapping (SLAM) [1], [2] information provided by a Cross-Reality (XR) device [3] to augment the linked 3D CAD models. The main objective is to use the tracking technology for an augmented and mixed reality experience by tracking a 3D model and superimposing its respective 3D CAD model data over the images we receive from the camera feed of the XR device without any scene preparation (e.g markers or feature maps). The intent is to conduct a visual analysis and evaluations based on the intrinsic and extrinsic of the model in the visualization system that instant3Dhub [4] has to offer. To achieve this we make use of the Apple’s ARKit to obtain the images, sensor data and SLAM heuristic of client XR device, remote marker-less model based 3D object tracking from monocular RGB image data and hybrid client server architecture. Our approach is agnostic of any SLAM system or Augmented Reality (AR) framework. We make use of the Apple’s ARKit because of the its ease of use, affordability, stability and maturity as a platform and as an integrated system.

Show publication details

Automatic Detection of the Nasal Cavities and Paranasal Sinuses Using Deep Neural Networks

2019

2019 IEEE International Symposium on Biomedical Imaging

IEEE International Symposium on Biomedical Imaging (ISBI) <16, 2019, Venice, Italy>

The nasal cavity and paranasal sinuses present large interpatient variabilities. Additional circumstances like for example, concha bullosa or nasal septum deviations complicate their segmentation. As in other areas of the body a previous multistructure detection could facilitate the segmentation task. In this paper an approach is proposed to individually detect all sinuses and the nasal cavity. For a better delimitation of their borders the use of an irregular polyhedron is proposed. For an accurate prediction the Darknet-19 deep neural network is used which combined with the You Only Look Once method has shown very promising results in other fields of computer vision. 57 CT scans were available of which 85% were used for training and the remaining 15% for validation.

  • 978-1-5386-3640-4
Show publication details
Sachs, Moritz Karl; Unbescheiden, Matthias [Betreuer]; Buxmann, Peter [Betreuer]

Automatisierte Geschäftsmodellanalysen mit Deep Neural Networks

2019

Darmstadt, TU, Master Thesis, 2019

Ein Hauptkriterium für das Investment eines Venture Capital Fonds in ein Start-up ist dessen Geschäftsmodell. Dieses ist im Businessplan enthalten. Das Screening, sowie die Analyse der eingereichten Businesspläne, erfolgt bei den meisten Venture Capital Fonds überwiegend durch Menschen. Mit der vorliegenden Arbeit wird untersucht, inwieweit die Analyse der in den Businessplänen enthaltenen Geschäftsmodelle mit Hilfe von Deep Neural Networks automatisiert werden kann. Ziel war die Entwicklung eines Prototypen, der die in den Businessplänen enthaltenen Geschäftsmodelle automatisch extrahiert und in das Metamodell Startup Navigator überführt. Dem Knowledge Discovery in Databases Prozess folgend wurden hierfür die Businesspläne eines Venture Capital Fonds aufbereitet und damit ein tiefes Convolutional Neural Network, der Multilabel k-Nearest Neighbour Algorithmus, sowie eine Support Vector Machine mit Naive Bayes Features trainiert. Die Ergebnisse des entwickelten Prototypen zeigen, dass die in den Businessplänen enthaltenen Geschäftsmodelle automatisch extrahiert und in das Metamodell Startup Navigator überführt werden können. Es erscheint plausibel, dass mit mehr Trainingsdaten und einer intensiveren Hyperparameteroptimierung die Korrektklassifizierungsrate verbessert werden kann, sodass der Prototyp zum Aufbau eines Geschäftsmodellkorpus genutzt werden könnte.

Show publication details
Cheng, Wentao; Lin, Weisi; Chen, Kan; Zhang, Xinfeng

Cascaded Parallel Filtering for Memory-Efficient Image-Based Localization

2019

2019 International Conference on Computer Vision Workshops. Proceedings

International Conference on Computer Vision (ICCV) <17, 2019, Seoul, Korea>

Image-based localization (IBL) aims to estimate the 6DOF camera pose for a given query image. The camera pose can be computed from 2D-3D matches between a query image and Structure-from-Motion (SfM) models. Despite recent advances in IBL, it remains difficult to simultaneously resolve the memory consumption and match ambiguity problems of large SfM models. In this work, we propose a cascaded parallel filtering method that leverages the feature, visibility and geometry information to filter wrong matches under binary feature representation. The core idea is that we divide the challenging filtering task into two parallel tasks before deriving an auxiliary camera pose for final filtering. One task focuses on preserving potentially correct matches, while another focuses on obtaining high quality matches to facilitate subsequent more powerful filtering. Moreover, our proposed method improves the localization accuracy by introducing a quality-aware spatial reconfiguration method and a principal focal length enhanced pose estimation method. Experimental results on real-world datasets demonstrate that our method achieves very competitive localization performances in a memory-efficient manner.

Show publication details
Al Hajj, Hassan; Mukhopadhyay, Anirban; Lamard, Mathieu; Conze, Pierre-Henri; Roychowdhury, Soumali; Hu, Xiaowei; Marsalkaite, Gabija; Zisimopoulos, Odysseas; Dedmari, Muneer Ahmad; Zhao, Fenqiang; Prellberg, Jonas; Sahu, Manish; Galdran, Adrian; Araujo, Teresa; Vo, Duc My; Panda, Chandan; Dahiya, Navdeep; Kondo, Satoshi; Bian, Zhengbing; Vahdat, Arash; Bialopetravicius, Jonas; Flouty, Evangello; Qiu, Chenhui; Dill, Sabrina; Costa, Pedro; Aresta, Guilherme; Ramamurthy, Senthil; Lee, Sang-Woong; Campilho, Aurelio; Zachow, Stefan; Xia, Shunren; Conjeti, Sailesh; Stoyanov, Danail; Armaitis, Jogundas; Heng, Pheng-Ann; Macready, William G.; Cochener, Beatrice; Quellec, Gwenole

CATARACTS: Challenge on Automatic Tool Annotation for cataRACT Surgery

2019

Medical Image Analysis

Surgical tool detection is attracting increasing attention from the medical image analysis community. The goal generally is not to precisely locate tools in images, but rather to indicate which tools are being used by the surgeon at each instant. The main motivation for annotating tool usage is to design e_cient solutions for surgical workflow analysis, with potential applications in report generation, surgical training and even real-time decision support. Most existing tool annotation algorithms focus on laparoscopic surgeries. However, with 19 million interventions per year, the most common surgical procedure in the world is cataract surgery. The CATARACTS challenge was organized in 2017 to evaluate tool annotation algorithms in the specific context of cataract surgery. It relies on more than nine hours of videos, from 50 cataract surgeries, in which the presence of 21 surgical tools was manually annotated by two experts. With 14 participating teams, this challenge can be considered a success. As might be expected, the submitted solutions are based on deep learning. This paper thoroughly evaluates these solutions: in particular, the quality of their annotations are compared to that of human interpretations. Next, lessons learnt from the di_erential analysis of these solutions are discussed. We expect that they will guide the design of e_cient surgery monitoring tools in the near future.

Show publication details
Abuladze, David; Kuijper, Arjan [1. Gutachten]; Ben Hmida, Helmi [2. Gutachten]

ClickDigital IDE Enrichment with Internet of Things Rule Widgets

2019

Darmstadt, TU, Bachelor Thesis, 2019

In the recent years, the Internet of Things (IoT) has become an important subject for different services and for the society in general, where different companies use IoT Systems in different ways to improve everyday life quality. Examples of use can be smart adjustment of heating systems, logistics, smart homes, device management for IoT System gateway, device management for connected agricultures, connected cars, and many others. Simplification of the communication process between the end user and devices of the IoT Systems has been a point of concern in many computer science studies. One of them is the master thesis elaborated by Zafar [1]. In his research titled “Enhancing User Experience in the Internet of Things systems regarding Smart Rule Management”, Zafar [1] based his work on several friendly Rule Management interfaces and their arguments pro and contra. After having analyzed and compared relevant studies that have been published at that time, Zafar [1] has suggested a number of user-friendly Rules based widgets within ClickDigital toolkit. ClickDigital toolkit is a dashboard that combines and integrates different IoT System Service interfaces. These widgets aim to manage rules, created by the end users, without any technical background. In my Bachelor Present I will extend the ClickDigital Integrated Development Environment (IDE), which is a web toolkit developed by the Fraunhofer-Institut für Graphische Datenverarbeitung. The present Bachelor Thesis focuses on the ClickDigital IDE enrichment with an Internet of Things Rule widgets. Enrichment of the ClickDigital IDE consists of Rule widgets implementation and validation. The main target includes the extension and the enrichment of the existing ClickDigital widgets repository with a set of Rule Management widgets. The analysis of the previous relevant study by Zafar [1] and other sources has been performed, and several improvements have been implemented. It allows resulting in a better and more efficient communication process between end users and devices within ClickDigital toolkit.

Show publication details
Zeng, Dong-dong; Zhu, Ming; Kuijper, Arjan

Combining Background Subtraction Algorithms with Convolutional Neural Network

2019

Journal of Electronic Imaging

Accurate and fast extraction of foreground objects is a key prerequisite for a wide range of computer vision applications, such as object tracking and recognition. Thus, many background subtraction (BGS) methods for foreground object detection have been proposed in recent decades. However, this is still regarded as a tough problem due to a variety of challenges, such as illumination variations, camera jitter, dynamic backgrounds, and shadows. Currently, there is no single method that can handle all the challenges in a robust way. We try to solve this problem from a perspective of combining different state-of-the-art BGS algorithms to create a more robust and more advanced foreground detection algorithm. More specifically, an encoder–decoder fully convolutional neural network architecture is adapted and trained to automatically learn how to leverage the characteristics of different algorithms to fuse the results produced by different BGS algorithms and produce a more precise result. Comprehensive experiments evaluated on the CDnet 2014 dataset demonstrate that the proposed method outperforms all the considered single BGS algorithms. We show that our solution is more efficient than other BGS combination strategies.

Show publication details

D-ID-Net: Two-Stage Domain and Identity Learning for Identity-Preserving Image Generation From Semantic Segmentation

2019

2019 International Conference on Computer Vision Workshops. Proceedings

International Conference on Computer Vision (ICCV) <17, 2019, Seoul, Korea>

Training functionality-demanding AR/VR systems require accurate and robust gaze estimation and tracking solutions. Achieving such a performance requires the availability of diverse eye image data that might only be acquired by the means of image generation. Works addressing the generation of such images did not target realistic and identity-specific images, nor did they address the practicalrelevant case of generation from semantic labels. Therefore, this work proposes a solution to generate realistic and identity-specific images that correspond to semantic labels, given samples of a specific identity. Our proposed solution consists of two stages. In the first stage, a network is trained to transform the semantic label into a corresponding eye image of a generic identity. The second stage is an identity-specific network that induces identity details on the generic eye image. The results of our D-ID-Net solutions shows a high degree of identity-preservation and similarity to the ground-truth images, with an RMSE of 7.235.

Show publication details
Chaudry, Zaki Ullah; Kuijper, Arjan [Betreuer]; Rus, Silvia [Prüfer]

Designing a General Purpose Smart Textile for a Healthy Lifestyle

2019

Darmstadt, TU, Master Thesis, 2019

We live in a world where wearable technology is fast becoming a part of our daily lives. These wearables vary from a wide range of gadgets such as watches, goggles, shoes, textile etc. One thing all of these wearables have in common is health and fitness monitoring in order to make human life better, which is being achieved by the help of monitoring the activities of the users with the help of various sensors incorporated within the wearable technology. In this thesis the main focus is on creating a smart textile which typically could be placed on different forms of furniture being used in daily life such as bed, chair, couch or table, which than monitors the activities and ergonomics of the person using the furniture. Based on the embedded sensor matrix and advanced algorithms the textile would automatically detect the type of furniture it is placed on. The textile will also constitute of capacitive sensors which, separately or in combination with the accelerometer sensors, detect the user activity (by detecting the breathing rhythm, movement or user interaction). When used in combination with a bed this textile with the help of the sensors will be able to detect the movement, breathing rhythm and user interaction for the user. The same prototype can be used by a person sitting on a chair or couch to detect the breathing, movement and interaction. In case of a table the prototype will be able to measure movement and interaction only. Some smart furniture is already available in the market, but the prices are often to high for a average user to be able to afford them and also the use is limited to just one product. With this textile a user can convert existing dumb furniture into smart furniture, hence providing a wide range of usage with the help of just one textile. In result creating a place-able smart textile technology for a better and healthy lifestyle at an affordable price. This textile minimizes the distance between humans and smart living and fills in the gaps which are beyond the capacity of current wearable technology, while staying cost effective and flexible. Therefore being a great addition to the ecosystem of smart living.

Show publication details

2019

Proceedings of the 12th ACM International Conference on PErvasive Technologies Related to Assistive Environments

ACM International Conference on PErvasive Technologies Related to Assistive Environments (PETRA) <12, 2019, Rhodes, Greece>

Smart textiles and garments promise intriguing new possibilities for the wearer. Integrated interaction can create new experiences and sensors can detect relevant information about the wearer. However, this poses an additional challenge for the designer of smart garments, about how to integrate these technologies. In this work, we want to investigate how human intuition and technical knowledge feed into the design of smart garments. Using a jacket that tracks its whereabouts as a use case, we have collected a dataset from 18 test subjects with varying technical knowledge, on what sensor patterns they would create on the garment. Using a specifically created simulation framework, we have evaluated the performance of the created sensor patterns. We observed that many participants intuitively create well-working patterns, while technical knowledge does not play a significant role in the resulting performance.

  • 978-1-4503-6232-0
Show publication details
Mettel, Matthias Ruben; Alekseew, Michael; Stocklöw, Carsten; Braun, Andreas

Designing and Evaluating Safety Services Using Depth Cameras

2019

Journal of Ambient Intelligence and Humanized Computing

Not receiving help in the case of an emergency is one of the most common fears of older adults that live independently at home. Falls are a particularly frequent occurrence and often the cause of serious injuries. In the last years, various ICT solutions for supporting older adults at home have been developed. Based on sensors and services in a smart environment they provide a wide range of services. In this work we have designed and evaluated safety-related services, based on a single Microsoft Kinect that is installed in a user’s home. We created two services to investigate the benefits and limitations of these solutions. The first is a fall detection service that registers falls in real-time, using a novel combination of static and dynamic skeleton tracking. The second is a fall prevention service that detects potentially dangerous objects in the walking path, based on scene analysis in a depth image. We conducted technical and user evaluations for both services, in order to get feedback on the feasibility, limitations, and potential future improvements.

Show publication details
Helfmann, Stefan; Kuijper, Arjan [1. Gutachten]; Rus, Silvia [2. Gutachten]

Designing Smart Home Controls for Elderly

2019

Darmstadt, TU, Master Thesis, 2019

As technology advances, the idea of a "smart home" gets implemented more and more, and offers multiple ways to help people managing their homes. This offers a big opportunity to help elderly people with their daily lives. However, most feel overwhelmed by modern technology and opt out of using it at all. This thesis provides an alternative take on a Smart Home remote control. After collecting, discussing and evaluating several design principles and guidelines we propose two possible concept remotes and corresponding mock-ups, that are subjected to an evaluation by elderly people to choose the one they prefer. This preferred model will be along with proposed improvements made into a functional prototype and, again, be evaluated by the elderly. The final prototype was received positively and additional improvements are proposed.

Show publication details
Damer, Naser; Boller, Viola; Wainakh, Yaza; Boutros, Fadi; Terhörst, Philipp; Braun, Andreas; Kuijper, Arjan

Detecting Face Morphing Attacks by Analyzing the Directed Distances of Facial Landmarks Shifts

2019

Pattern Recognition

German Conference on Pattern Recognition (GCPR) <40, 2018, Stuttgart, Germany>

Lecture Notes in Computer Science (LNCS)
11269

Face morphing attacks create face images that are verifiable to multiple identities. Associating such images to identity documents lead to building faulty identity links, causing attacks on operations like border crossing. Most of previously proposed morphing attack detection approaches directly classified features extracted from the investigated image. We discuss the operational opportunity of having a live face probe to support the morphing detection decision and propose a detection approach that take advantage of that. Our proposed solution considers the facial landmarks shifting patterns between reference and probe images. This is represented by the directed distances to avoid confusion with shifts caused by other variations. We validated our approach using a publicly available database, built on 549 identities. Our proposed detection concept is tested with three landmark detectors and proved to outperform the baseline concept based on handcrafted and transferable CNN features.

  • 978-3-030-12938-5
Show publication details
Jagadishwara, Anitha Sankapura; Mukhopadhyay, Anirban [1. Prüfer]; Kuijper, Arjan [2. Prüfer]

Disentangled Representation of Breast Cancer Images

2019

Darmstadt, TU, Master Thesis, 2019

The motivation behind semi-supervised learning of disentangled representation lies in the difficulty of getting access to an expert-annotated large dataset in medical applications. Learning disentangled representations from visual data where high-level generative factors are defined by users, is of great importance for many computer vision applications. Defining the factors of variation externally allows a user greater flexibility in modifying and interpreting the learnt latent representations of data. Recent works have shown increased interest in using Variational Autoencoders to discover the interpretable representations of data in an semi-supervised way. This has also been used for a wide range of applications such as image search, natural language parsing and speech analysis. In this work, we propose a deep generative model combining Variational Autoencoder (VAE) and Wasserstein Generative adversarial network (WGAN) to learn the disentangled representation of the data. One of the application of this approach is to enhance the readability of mammograms used for breast cancer screening. The proposed approach was partially successful and there is a lot of scope for possible improvements. The accuracy of the proposed approach is limited due to underlying constraints which were out of scope of this thesis. Additionally, due to resource constraints, the quality of images has been compromised in this work. The novelty of this approach, however, lies in learning more semantically meaningful latent space representation which can be reused in various applications such as synthetic data generation, image translation, classification and segmentation tasks. This approach was successfully tested on most popular datasets such as MNIST (hand-written digits) and fashion-MNIST. However, due to the unavailability of ground truth images, a qualitative measure for breast cancer result is missing. It lacks proper evaluation at the moment and could be taken up as future works.

Show publication details
Rosbach, Sascha; James, Vinit; Großjohann, Simon; Homoceanu, Silviu; Roth, Stefan

Driving with Style: Inverse Reinforcement Learning in General-Purpose Planning for Automated Driving

2019

IEEE/RSJ 2019 International Conference on Intelligent Robots and Systems

IEEE/RSJ International Conference on Intelligent Robotics and Systems (IROS) <2019, Macau, China>

Behavior and motion planning play an important role in automated driving. Traditionally, behavior planners instruct local motion planners with predefined behaviors. Due to the high scene complexity in urban environments, unpredictable situations may occur in which behavior planners fail to match predefined behavior templates. Recently, general-purpose planners have been introduced, combining behavior and local motion planning. These general-purpose planners allow behavior-aware motion planning given a single reward function. However, two challenges arise: First, this function has to map a complex feature space into rewards. Second, the reward function has to be manually tuned by an expert. Manually tuning this reward function becomes a tedious task. In this paper, we propose an approach that relies on human driving demonstrations to automatically tune reward functions. This study offers important insights into the driving style optimization of general-purpose planners with maximum entropy inverse reinforcement learning. We evaluate our approach based on the expected value difference between learned and demonstrated policies. Furthermore, we compare the similarity of human driven trajectories with optimal policies of our planner under learned and expert-tuned reward functions. Our experiments show that we are able to learn reward functions exceeding the level of manual expert tuning without prior domain knowledge.

Show publication details
Hiemenz, Benedikt; Krämer, Michel

Dynamic Searchable Symmetric Encryption for Storing Geospatial Data in the Cloud

2019

International Journal of Information Security

We present a dynamic searchable symmetric encryption scheme allowing users to securely store geospatial data in the cloud. Geospatial data sets often contain sensitive information, for example, about urban infrastructures. Since clouds are usually provided by third parties, these data need to be protected. Our approach allows users to encrypt their data in the cloud and make them searchable at the same time. It does not require an initialization phase, which enables users to dynamically add new data and remove existing records. We design multiple protocols differing in their level of security and performance, respectively. All of them support queries containing boolean expressions, as well as geospatial queries based on bounding boxes, for example. Our findings indicate that although the search in encrypted data requires more runtime than in unencrypted data, our approach is still suitable for real-world applications.We focus on geospatial data storage, but our approach can also be applied to applications from other areas dealing with keyword-based searches in encrypted data. We conclude the paper with a discussion on the benefits and drawbacks of our approach.

Show publication details

E-Textile Capacitive Electrodes: Fabric or Thread

2019

Proceedings of the 12th ACM International Conference on PErvasive Technologies Related to Assistive Environments

ACM International Conference on PErvasive Technologies Related to Assistive Environments (PETRA) <12, 2019, Rhodes, Greece>

Back pain is one of the most common illnesses in Western civilizations. Office work and lack of motion can lead to deterioration over time. Many people already use seat cushions to improve their posture during work or leisure. In this work, we present an E-Textile cushion. This seat cushion is equipped with capacitive proximity sensors that track the proximity and motion of the sitting user and distinguish up to 7 postures. Giving a user immediate feedback on the posture can facilitate more healthy behavior. We evaluated a number of different electrode setups, materials, and classification methods, leading to a maximum accuracy of 97.1%.

  • 978-1-4503-6232-0
Show publication details

Efficient slicing of Catmull–Clark solids for 3D printed objects with functionally graded material

2019

Computers & Graphics

In the competition for the volumetric representation most suitable for functionally graded materials in additively manufactured (AM) objects, volumetric subdivision schemes, such as Catmull-Clark (CC) solids, are widely neglected. Although they show appealing properties, e_cient implementations of some fundamental algorithms are still missing. In this paper, we present a fast algorithm for direct slicing of CC-solids generating bitmaps printable by multi-material AMmachines. Our method optimizes runtime by exploiting constant time limit evaluation and other structural characteristics of CCsolids. We compare our algorithm with the state of the art in trivariate trimmed spline representations and show that our algorithm has similar runtime behavior as slicing trivariate splines, fully supporting the benefits of CC-solids.

Show publication details
Silva, Nelson; Blascheck, Tanja; Jianu, Radu; Rodrigues, Nils; Weiskopf, Daniel; Raubal, Martin; Schreck, Tobias

Eye Tracking Support for Visual Analytics Systems

2019

ETRA '19

ACM Symposium on Eye Tracking Research & Applications (ETRA) <11, 2019, Denver, Colorado>

Visual analytics (VA) research provides helpful solutions for interactive visual data analysis when exploring large and complex datasets. Due to recent advances in eye tracking technology, promising opportunities arise to extend these traditional VA approaches. Therefore, we discuss foundations for eye tracking support in VA systems. We first review and discuss the structure and range of typical VA systems. Based on a widely used VA model, we present five comprehensive examples that cover a wide range of usage scenarios. Then, we demonstrate that the VA model can be used to systematically explore how concrete VA systems could be extended with eye tracking, to create supportive and adaptive analytics systems. This allows us to identify general research and application opportunities, and classify them into research themes. In a call for action, we map the road for future research to broaden the use of eye tracking and advance visual analytics.

Show publication details

Eye-MMS: Miniature Multi-Scale Segmentation Network of Key Eye-Regions in Embedded Applications

2019

2019 International Conference on Computer Vision Workshops. Proceedings

International Conference on Computer Vision (ICCV) <17, 2019, Seoul, Korea>

Segmentation of the iris or sclera is an essential processing block in ocular biometric systems. However, humancomputer interaction, as in VR/AR applications, requires multiple region segmentation to enable smoother interaction and eye-tracking. Such application does not only demand highly accurate and generalizable segmentation, it requires such segmentation model to be appropriate for the limited computational power of embedded systems. This puts strict limits on the size of the deployed deep learning models. This work presents a miniature multi-scale segmentation network consisting of inter-connected convolutional modules. We present a baseline multi-scale segmentation network and modify it to reduce its parameters by more than 80 times, while reducing its accuracy by less than 3%, resulting in our Eye-MMS model containing only 80k parameters. This work is developed on the OpenEDS database and is conducted in preparation for the OpenEDS Semantic Segmentation Challenge.

Show publication details
Krispel, Ulrich; Ullrich, Torsten; Tamke, Martin

Formalising Expert Knowledge for Building Information Models: Automated Identification of Electrical Wiring from 3D Scans

2019

Proceedings of International Academic Conference on Places and Technologies

International Academic Conference on Places and Technologies <3, 2016, Belgrade, Serbia>

New computational methods provide means to deduce semantic information from measurements, such as range scans and photographs of building interiors. In this paper, we showcase a method that allows to estimate elements that are not directly observable – ducts and power lines in walls. For this, we combine explicit information, which is deduced by algorithms from measured data, with implicit information that is publicly available: technical standards that restrict the placement of electrical power lines. We present a complete pipeline from measurements to a hypothesis of these power lines within walls. The approach is structured into the following steps: First, a coarse geometry is extracted from input measurements; i.e., the unstructured point cloud which was acquired by laser scanning is transformed into a simplistic building model. Then, visible endpoints of electrical appliances (e.g. sockets, switches) are detected from photos using machine learning techniques and a pre-trained classifier. Afterwards, positions of installation zones in walls are generated. Finally, a hypothesis of non-visible cable ducts is generated, under the assumption that (i) the real configuration obeys the rules of legal requirements and standards and (ii) the configuration connects all endpoints using a minimal amount of resources, i.e. cable length.

Show publication details
Bartschat, Andreas; Allgeier, Stephan; Scherr, Tim; Stegmaier, Johannes; Bohn, Sebastian; Reichert, Klaus-Martin; Kuijper, Arjan; Reischl, Markus; Stachs, Oliver; Köhler, Bernd; Mikut, Ralf

Fuzzy tissue detection for real-time focal control in corneal confocal microscopy

2019

at - Automatisierungstechnik

Corneal confocal laser scanning microscopy is a promising method for in vivo investigation of cellular structures, e. g., of nerve fibers in the sub-basal nerve plexus. During recording, even slight displacements of the focal plane lead to images of adjacent tissue layers. In this work, we propose a closed-loop control of the focal plane. To detect and evaluate the visible tissues, we utilize the Bag of Visual Words approach to implement a customizable image processing pipeline for real-time applications. Furthermore, we show that the proposed model can be trained with small classification datasets and can be applied as a segmentation method. The proposed control loop, including tissue detection, is implemented in a proof-of-concept setup and shows promising results in a first evaluation with a human subject.

Show publication details
Preiner, Reinhold; Boubekeur, Tamy; Wimmer, Michael

Gaussian-Product Subdivision Surfaces

2019

ACM Transactions on Graphics

International Conference on Computer Graphics and Interactive Techniques (SIGGRAPH) <46, 2019, Los Angeles, USA>

Probabilistic distribution models like Gaussian mixtures have shown greatpotential for improving both the quality and speed of several geometricoperators. This is largely due to their ability to model large fuzzy data usingonly a reduced set of atomic distributions, allowing for large compressionrates at minimal information loss. We introduce a new surface model thatutilizes these qualities of Gaussian mixtures for the definition and controlof a parametric smooth surface. Our approach is based on an enrichedmesh data structure, which describes the probability distribution of spatialsurface locations around each vertex via a Gaussian covariance matrix. Byincorporating this additional covariance information, we show how to definea smooth surface via a nonlinear probabilistic subdivision operator based onproducts of Gaussians, which is able to capture rich details at fixed controlmesh resolution. This entails new applications in surface reconstruction,modeling, and geometric compression.

Show publication details
Abrams, Jesse F.; Vashishtha, Anand; Wong, Seth T.; Nguyen, An; Mohamed, Azlan; Wieser, Sebastian; Kuijper, Arjan; Wilting, Andreas; Mukhopadhyay, Anirban

Habitat-Net: Segmentation of Habitat Images Using Deep Learning

2019

Ecological Informatics

Understanding environmental factors that influence forest health, as well as the occurrence and abundance of wildlife, is a central topic in forestry and ecology. However, the manual processing of field habitat data is time-consuming and months are often needed to progress from data collection to data interpretation. To shorten the time to process the data we propose here Habitat-Net: a novel deep learning application based on Convolutional Neural Networks (CNN) to segment habitat images of tropical rainforests. Habitat-Net takes color images as input and after multiple layers of convolution and deconvolution, produces a binary segmentation of the input image. We worked on two different types of habitat datasets that are widely used in ecological studies to characterize the forest conditions: canopy closure and understory vegetation. We trained the model with 800 canopy images and 700 understory images separately and then used 149 canopy and 172 understory images to test the performance of Habitat-Net. We compared the performance of Habitat-Net to the performance of a simple threshold based method, manual processing by a second researcher and a CNN approach called U-Net, upon which Habitat-Net is based. Habitat-Net, U-Net and simple thresholding reduced total processing time to milliseconds per image, compared to 45 s per image for manual processing. However, the higher mean Dice coefficient of Habitat-Net (0.94 for canopy and 0.95 for understory) indicates that accuracy of Habitat-Net is higher than that of both the simple thresholding (0.64, 0.83) and U-Net (0.89, 0.94). Habitat-Net will be of great relevance for ecologists and foresters, who need to monitor changes in their forest structures. The automated workflow not only reduces the time, it also standardizes the analytical pipeline and, thus, reduces the degree of uncertainty that would be introduced by manual processing of images by different people (either over time or between study sites).

Show publication details
Kügler, David; Krumb, Henry John; Bredemann, Judith; Stenin, Igor; Kristin, Julia; Klenzner, Thomas; Schipper, Jörg; Schmitt, Robert; Sakas, Georgios; Mukhopadhyay, Anirban

High-precision Evaluation of Electromagnetic Tracking

2019

International Journal of Computer Assisted Radiology and Surgery

Purpose: Navigation in high-precision minimally invasive surgery (HP-MIS) demands high tracking accuracy in the absence of line of sight (LOS). Currently, no tracking technology can satisfy this requirement. Electromagnetic tracking (EMT) is the best tracking paradigm in the absence of LOS despite limited accuracy and robustness. Novel evaluation protocols are needed to ensure high-precision and robust EMT for navigation in HP-MIS. Methods: We introduce a novel protocol for EMT measurement evaluation featuring a high-accuracy phantom based on LEGO, which is calibrated by a coordinate measuring machine to ensure accuracy. Our protocol includes relative sequential positions and an uncertainty estimation of positioning. We show effects on distortion compensation using a learned interpolation model. Results: Our high-precision protocol clarifies properties of errors and uncertainties of EMT for high-precision use cases. For EMT errors reaching clinically relevant 0.2 mm, our design is 5–10 times more accurate than previous protocols with 95% confidence margins of 0.02 mm. This high-precision protocol ensures the performance improvement in compensated EMT by 0.05 mm. Conclusion: Our protocol improves the reliability of EMT evaluations because of significantly lower protocol-inherent uncertainties. To reduce patient risk in HP-MIS and to evaluate magnetic field distortion compensation, more high-accuracy protocols such as the one proposed here are required.

Show publication details
Zhou, Wei; Ma, Calwen; Yao, Tong; Chang, Peng; Zhang, Qi; Kuijper, Arjan

Histograms of Gaussian Normal Distribution for 3D Feature Matching in Cluttered Scenes

2019

The Visual Computer

3D feature descriptors provide essential information to find given models in captured scenes. In practical applications, these scenes often contain clutter. This imposes severe challenges on the 3D object recognition leading to feature mismatches between scenes and models. As such errors are not fully addressed by the existing methods, 3D feature matching still remains a largely unsolved problem. We therefore propose our Histograms of Gaussian Normal Distribution (HGND) for capturing salient feature information on a local reference frame (LRF) that enables us to solve this problem. We define a LRF on each local surface patch by using the eigenvectors of the scatter matrix. Different from the traditional local LRF-based methods, our HGND descriptor is based on the combination of geometrical and spatial information without calculating the distribution of every point and its geometrical information in a local domain. This makes it both simple and efficient. We encode the HGND descriptors in a histogram by the geometrical projected distribution of the normal vectors. These vectors are based on the spatial distribution of the points.We use three public benchmarks, the Bologna, the UWA and the Ca’ Foscari Venezia dataset, to evaluate the speed, robustness, and descriptiveness of our approach. Our experiments demonstrate that the HGND is fast and obtains a more reliable matching rate than state-of-the-art approaches in cluttered situations.

Show publication details
Subasioglu, Meltem; Damer, Naser [Supervisor]; Kaschube, Matthias [Supervisor]

Humans vs Machines: A Comparison of Human and Machine Learning Performance in Inferring Professions from Facial Images

2019

Frankfurt am Main, Univ., Master Thesis, 2019

Convolutional neural networks have seen a rise in the computer vision community in the recent years, even surpassing human accuracies in face recognition tasks. However, research regarding different face classification and face clustering experiments, which do not focus on the discrimination of face images according to unique individuals in the dataset, are still very limited. Many psychological studies indicate that humans are capable in inferring leaders, status and competence from facial images. This work focused on the question whether humans are indeed capable in telling profession – given by the branch of the profession and the career status – from face images and whether current state-of-the-art machine learning approaches are able to do the same. Furthermore, the performance of humans and machine learning systems were compared to give insights in the underlying processes. The results indicate that both human and machine learning models are capable to infer professions from facial images with better than chance accuracies. Both humans and machine learning systems perform almost equally well in these tasks, however, performance differences in individual tasks indicate that humans and machine learning algorithms solve the same tasks while relying on different cues in the face images.

Show publication details

Implementing Secure Applications in Smart City Clouds Using Microservices

2019

Future Generation Computer Systems

Smart Cities make use of ICT technology to address the challenges of modern urban management. The cloud provides an efficient and cost-effective platform on which they can manage, store and process data, as well as build applications performing complex computations and analyses. The quickly changing requirements in a Smart City require flexible software architectures that let these applications scale in a distributed environment such as the cloud. Smart Cities have to deal with huge amounts of data including sensitive information about infrastructure and citizens. In order to leverage the benefits of the cloud, in particular in terms of scalability and cost-effectiveness, this data should be stored in a public cloud. However, in such an environment, sensitive data needs to be encrypted to prevent unauthorized access. In this paper, we present a software architecture design that can be used as a template for the implementation of Smart City applications. The design is based on the microservice architectural style, which provides properties that help make Smart City applications scalable and flexible. In addition, we present a hybrid approach to securing sensitive data in the cloud. Our architecture design combines a public cloud with a trusted private environment. To store data in a cost-effective manner in the public cloud, we encrypt metadata items with CP-ABE (Ciphertext-Policy Attribute-Based Encryption) and actual Smart City data with symmetric encryption. This approach allows data to be shared across multiple administrations and makes efficient use of cloud resources. We show the applicability of our design by implementing a web-based application for urban risk management. We evaluate our architecture based on qualitative criteria, benchmark the performance of our security approach, and discuss it regarding honest-but-curious cloud providers as well as attackers trying to access user data through eavesdropping. Our findings indicate that the microservice architectural style fits the requirements of scalable Smart City applications while the proposed security approach helps prevent unauthorized access.

Show publication details
Stoll, Christian; Kuijper, Arjan [Prüfer]; Fu, Biying [Betreuerin]

Indoor Localization using Particle Filter approach for Single and Multiperson Application in Smart Home

2019

Darmstadt, TU, Bachelor Thesis, 2019

The variation of electric potential, caused by movements, can be used to detect activities in an indoor environment. To detect those variations a sensor system is required. In this thesis, we perform indoor localization, by using a smart floor system, which is able to detect and measure the change in electric potential. We develop a system, which uses three different particle filter approach to process the data from the smart floor system, in order to be able to perform indoor localization for single- and multi-person applications. This system is able to perform all steps necessary to solve this task. It processes the data, sent by the smart floor system, by updating the particles, clusters those particles and it finally assigns the centers of those clusters as positions to people, which are located on the smart floor. We will describe a theoretical approach to particle filter algorithms, and the clustering algorithm DBSCAN. Afterwards, we will describe and discuss our design descisions, as well as our implementation in detail and then evaluate the three proposed algorithms by examining their applicability in practical scenarios in a model apartment and their precision in a laboratory environment. Both those examinations and evaluations will be split into two parts, according to the split task of single- and multi-person applications. Finally, we will discuss those results and point out some properties which are specific for the approaches, we take at our particle filter algorithms and describe future adaptions, we consider necessary for a potential practical application, for the system to be more flexible in interaction and localizing different numbers of people.

Show publication details

Integrating Server-based Simulations into Web-based Geo-applications

2019

Eurographics 2019. Short Papers

Annual Conference of the European Association for Computer Graphics (Eurographics) <40, 2019, Genoa, Italy>

In this work, we present a novel approach for combining fluid simulations running on a GPU server with terrain rendered by a web-based 3D GIS system. We introduce a hybrid rendering approach, combining server-side and client-side rendering, to interactively display the results of a shallow water simulation on client devices using web technology. To display water and terrain in unison, we utilize image merging based on depth values.We extend it to deal with numerical and compression artifacts as well as Level-of-detail rendering and use Depth Image Based Rendering to counteract network latency.

Show publication details
Chegini, Mohammad; Bernard, Jürgen; Berger, Philip; Sourin, Alexei; Andrews, Keith; Schreck, Tobias

Interactive labelling of a multivariate dataset for supervised machine learning using linked visualisations, clustering, and active learning

2019

Visual Informatics

Supervised machine learning techniques require labelled multivariate training datasets. Many approaches address the issue of unlabelled datasets by tightly coupling machine learning algorithmswith interactive visualisations. Using appropriate techniques, analysts can play an active role in ahighly interactive and iterative machine learning process to label the dataset and create meaningfulpartitions. While this principle has been implemented either for unsupervised, semi-supervised, orsupervised machine learning tasks, the combination of all three methodologies remains challenging.In this paper, a visual analytics approach is presented, combining a variety of machine learningcapabilities with four linked visualisation views, all integrated within the mVis (multivariate Visualiser)system. The available palette of techniques allows an analyst to perform exploratory data analysis ona multivariate dataset and divide it into meaningful labelled partitions, from which a classifier canbe built. In the workflow, the analyst can label interesting patterns or outliers in a semi-supervisedprocess supported by active learning. Once a dataset has been interactively labelled, the analyst cancontinue the workflow with supervised machine learning to assess to what degree the subsequentclassifier has effectively learned the concepts expressed in the labelled training dataset. Using a noveltechnique called automatic dimension selection, interactions the analyst had with dimensions of themultivariate dataset are used to steer the machine learning algorithms.A real-world football dataset is used to show the utility of mVis for a series of analysis and labellingtasks, from initial labelling through iterations of data exploration, clustering, classification, and activelearning to refine the named partitions, to finally producing a high-quality labelled training datasetsuitable for training a classifier. The tool empowers the analyst with interactive visualisations includingscatterplots, parallel coordinates, similarity maps for records, and a new similarity map for partitions

Show publication details
Matthiesen, Moritz; Rojtberg, Pavel [1. Gutachten]; Kuijper, Arjan [2. Gutachten]

Interpolation von Kalibrier daten für Zoom und Autofokus Kameras

2019

Darmstadt, TU, Bachelor Thesis, 2019

In dieser Arbeit wird das Problem betrachtet, dass für jede neue Kameraeinstellung eine neue Kalibrierung vorgenommen werden muss.Ziel dabei ist Kalibrierdaten an bestimmten Kameraeinstellungen zu erstellen, um mithilfe vondiesen die Kalibrierdaten von anderen Kameraeinstellungen herzuleiten. Dabei werden die Kalibrierdaten betrachtet und es wird versucht Beziehungen zwischen den einzelnen Parametern der Kalibrierung herzuleiten. Um diese zu ermitteln wird zwischen verschiedenen Parametern der Kalibrierung interpoliert.

Show publication details
Distergoft, Alexander; Kuijper, Arjan [1. Gutachten]; Mukhopadhyay, Anirban [2. Gutachten]

Interpreting Adversarial Examples in Medical Imaging

2019

Darmstadt, TU, Master Thesis, 2019

Deep neural networks (DNNs) have been achieving high accuracy on many important tasks like image classification, detection or segmentation. Yet, recent discoveries have shown a high degree of susceptibility for these deep-learning algorithms under attack. DNNs seem to be vulnerable to small amounts of non-random noise, created by perturbing the input to output mapping of the network. These perturbations can severely affect the performance of DNNs and thus endanger systems where such models are employed. The purpose of this thesis is to examine adversarial examples in clinical settings, be it digitally created or physical ones. For this reason we studied the performance of DNNs under the following three attack scenarios: 1. We hypothesize that adversarial examples might occur from incorrect mapping of the image space to the lower dimensional generation manifold. The hypothesis is tested by creating a proxy task of a pose estimation of surgical tools in its simplest form. For this we define a clear decision boundary. We use exhaustive search on a synthetic toy dataset to localize possible resions of successful one-pixel-attacks in image space. 2. We design a small scale prospective evaluation on how Deep-learning (DL) dermoscopy systems perform under physical world attacks. The publically available Physical Attacks on Dermoscopy Dataset (PADv1) is used for this evaluation. The introduced susceptibility and robustness values reveal that such attacks lead to accuracy loss across popular state-of-the-art DL-architectures. 3. As a pilot study to understand the vulnerabilities of DNNs that perform under regression tasks we design a set of auxiliary tasks that are used to create adversarial examples for non-classification-models. We train auxiliary networks on augmented datasets to satisfy the defined auxiliary tasks and create adversarial examples that might influence the decision of a regression model without knowing about the underlying system or hyperparameters.

Show publication details
Braun, Andreas; Zander-Walz, Sebastian; Majewski, Martin; Kuijper, Arjan

Investigating Large Curved Interaction Devices

2019

Personal and Ubiquitous Computing

Large interactive surfaces enable novel forms of interaction for their users, particularly in terms of collaborative interaction. During longer interactions, the ergonomic factors of interaction systems have to be taken into consideration. Using the full interaction space may require considerable motion of the arms and upper body over a prolonged period of time, potentially causing fatigue. In this work, we present Curved, a large-surface interaction device, whose shape is designed based on the natural movement of an outstretched arm. It is able to track one or two hands above or on its surface by using 32 capacitive proximity sensors. Supporting both touch and mid-air interaction can enable more versatile modes of use. We use image processing methods for tracking the user's hands and classify gestures based on their motion. Virtual reality is a potential use case for such interaction systems and was chosen for our demonstration application. We conducted a study with ten users to test the gesture tracking performance, as well as user experience and user preference for the adjustable system parameters.

Show publication details
Boller, Viola; Pesavento, Marius [1. Gutachten]; Damer, Naser [2. Gutachten]

Investigating the Use of Deeply Calculated Flows and Dynamic Routed Network Capsules in Face Presentation Attack Detection

2019

Darmstadt, TU, Master Thesis, 2019

Identifying an individual based on their facial characteristics is convenient and is gaining more and more importance. Only a normal camera is needed for data capturing, the individual characteristic can’t be stolen or forgot like a password, and it’s a non-intrusive capturing method, which enables identification from distances. With increasing usage of face recognition systems, also the amount of sophisticated attacks increase. Therefore vulnerabilities of such a recognition system are used. Especially the attack directly at the sensor, by presenting fake biometrics at the sensor, can be performed with low efforts and costs, and without detailed knowledge about the biometric system. That’s why this kind of attacks have caused wide attention in the biometric community. The attack is called presentation attack and there are many detection algorithms developed to solve this issue. Most existing presentation attack detection algorithms work great, when deployed in the same environment as they are trained in. But different environmental conditions are still challenging to solve. This thesis deals with the presentation attack issue by developing an approach, which combines deep optical flows and a capsule neural network. Optical flows have already proven to be a good preprocessing step, because only motion information is kept and environmental conditions won’t have a big impact on it. Deep optical flows are produced by end-to-end trained convolutional neural networks for the optical flow calculation. Capsule neural networks train neurons in a more general way by forwarding information of found pattern’s poses in the network additionally to it’s probabilities. Forwarding is achieved by a so called routing-by-agreement method, which allows to take relationships between the found patterns into account. For presentation attack detection development not enough data is available to train a deep neural network from scratch. This could be overcome by using a capsule neural network for classification. A deep understanding can be learned, needing less data. The contribution of this thesis is the combination of deep optical flows and a capsule neural network, which hasn’t been tested so far. This is realized by stacking multiple optical flows over the time and downsample this information, thus that the capsule neural network is able to train a classification. This pipeline is structurally optimized by testing different optical flow sizes, data normalization and multiple class training. Also a new downsampling and time stacking method is developed to keep outliers in the reduced data and order a stack according to the contained data information. This has proven to help the network to classify. In the end, the behavior of the network on various attacks is analyzed. In this thesis, performance improvements are achieved by structure optimizations and a novel downsampling and time stacking method. The structural improvements increased the model performance up to 12% and the new downsampling and time stacking method leads to an average performance improvement of 7.56% for two databases. However, the achieved performance does not outperform recent state-of-the-art presentation attack detection methods and the analysis over different attacks reveals that the developed structure adapts weakly to changes in the attack performance or to new attacks. Nevertheless, stable results for different environmental conditions are achieved.

Show publication details
Fellner, Dieter W. [Hrsg.]; Welling, Daniela [Red.]; Ackeren, Janine van [Red.]; Bergstedt, Bettina [Red.]; Krüger, Kathrin [Red.]; Prasche, Svenja [Betreuer]

Jahresbericht 2018

2019

Das Fraunhofer IGD hat seine Forschungsaktivitäten vor Kurzem in vier Leitthemen gebündelt, welche die Basis seiner Arbeit bilden und verschiedene Themen abteilungsübergreifend miteinander verknüpfen. Eines dieser Leitthemen ist "Visual Computing as a Service - Die Plattform für angewandtes Visual Computing". Die Basis dieser universellen Plattform für Visual-Computing Lösungen ist gelegt und wird kontinuierlich erweitert. Dieser technologische Ansatz bildet die Grundlage für die weiteren Leitthemen. In der "Individuellen Gesundheit - Digitale Lösungen für das Gesundheitswesen" werden die Daten betrachtet, die in der personalisierten Medizin anfallen - mithilfe der Visual-Computing-Technologien des Instituts. Im Leitthema "Intelligente Stadt - Innovativ, digital und nachhaltig" ist die Fragestellung, wie man den Lebenszyklus urbaner Prozesse unterstützen kann. Und im Leitthema "Digitalisierte Arbeit - Der Mensch in der Industrie 4.0" geht es erster Linie um die Unterstützung des Menschen in der durch die Digitalisierung veränderten Produktion.

Show publication details
Antony, Niklas; Urban, Bodo [Gutachter]; Bieber, Gerald [Betreuer]

Konzeption und Entwicklung einer Kommunikationsstruktur für autonom-mobile Assistenzsysteme

2019

Rostock, Univ., Bachelor Thesis, 2019

In Deutschland herrscht ein Mangel an Arbeitskräften in Krankenhäusern und Pflegeheimen, welcher durch die steigende Lebenserwartung stetig zunimmt. Mithilfe von autonom-mobilen Robotern lassen sich Pflegekräfte bei ihrer Arbeit entlasten. Lokalisierung und Zielfindung sind die wesentlichen Funktionalitäten dieser Roboter. In der vorliegenden Arbeit wird ein Konzept für ein autonom-mobiles Assistenzsystem mit darauf aufbauendem Prototypen vorgestellt, das QR-Codes als Orientierungspunkte verwendet. Die Einsatzumgebung beschränkt sich auf Innenräume. Hierbei ist ein Smartphone über Bluetooth mit dem mobilen Roboter verbunden, das mit der Rück-Kamera an die Raumdecke gerichtet ist und die dort befestigten QR-Codes ausliest. Die QR-Codes speichern ausreichend Informationen über ihre Umgebung, sodass es dem Roboter möglich ist, ausschließlich mithilfe dieser Informationen und ohne Suchalgorithmus einen zugewiesenen Zielpunkt zu erreichen. Um Kollisionen zu vermeiden, ist der Roboter zusätzlich mit einem Ultraschallsensor ausgestattet. Die Ergebnisse aus Versuchen mit dem Prototypen werden vorgestellt sowie Möglichkeiten der Weiterentwicklung erwähnt.

Show publication details
Zielinski, Michael; Urban, Bodo [Gutachter]; Nonnemann, Lars [Betreuer]

Koordiniertes Workflow Management mit Hilfe von Brushing & Linking Mechanismen

2019

Rostock, Univ., Bachelor Thesis, 2019

Die Art und Weise wie Datenmengen untersucht werden, erlebte in den letzten Jahren einige Veränderungen. Da die Menge der Daten mit der Zeit immer weiter anwächst, müssen verschiedene Analysewerkzeuge zur Untersuchung der Daten verwendet werden. Durch die gleichzeitige Nutzung mehrerer Werkzeuge entsteht jedoch die Problematik, dass unterschiedliche Aspekte wie beispielsweise die Position oder die Skalierung der Werkzeugansichten koordiniert werden müssen. Die Koordination dieser Werkzeuge reicht dabei vom sequentiellen Ausführen der Werkzeuge bis zur Nutzung von anwendungsspezifischen Analysesystemen. Weiterhin haben sich oftmals Workflows zur Analyse der Datenmengen etabliert. Im Kontext eines Workflows werden die Analysewerkzeuge in vielen Fällen jedoch nicht ausreichend miteinander koordiniert, sodass die Datenübergabe oder das Starten und Beenden von Analysewerkzeugen manuell durchgeführt werden muss. Dies führt zu erhöhtem Arbeitsaufwand auf Seiten der Datenanalytiker. Ziel dieser Arbeit ist es, basierend auf dem aktuellen Stand der Technik Konzepte zur workflowbasierten Datenanalyse zu entwickeln, um die Koordination unterschiedlicher Werkzeuge in analytischen Prozessen zu unterstützen. Dabei wird die Aufteilung des Informationsraumes auf Daten- und Visualisierungsebene untersucht. Weiterhin nutzt der entwickelte Ansatz Brushing & Linking zur Verknüpfung mehrerer Ansichten durch anwendungsübergreifende Interaktion. Die konzeptionell entwickelten Ansätze werden prototypisch in das vom Fraunhofer IGD entwickelte Health@Hand-System eingebunden. Dieses System ist ein virtueller Leitstand zur Visualisierung von Vitaldaten, damit beispielsweise Anomalien festgestellt werden können, sodass bei Störfällen frühzeitig interveniert wird. Um repetitive Arbeitsprozesse automatisieren zu können, wird der workflow-basierte Ansatz dieser Arbeit in der Implementierung in Health@Hand verwendet. Dabei wird ein gegebenes Szenario zur Detektion von kardiovaskulären Symptomen und Anomalien umgesetzt. Diese Arbeit gliedert sich in sieben Kapitel. Im nachfolgenden zweiten Kapitel werden grundlegende Techniken zum Aufteilen und Navigieren von Ansichten im Darstellungs- sowie Informationsraum vorgestellt. Darauf folgt in Kapitel 3 die Beschreibung von aktuellen Systemen, welche jeweils unterschiedliche Probleme der Ansichts- und Werkzeugkoordination lösen. Anschließend wird in Kapitel 4 eine Problem- und Anforderungsanalyse zur workflow-basierten Datenanalyse durchgeführt. Auf Basis dessen wird in den Kapiteln 5 und 6 das entwickelte Konzept eines Systems zur workflow-basierten Datenanalyse beschrieben und die prototypische Implementierung dessen in Health@Hand vorgestellt. Die Arbeit schließt schlussendlich mit einem Fazit der entwickelten Ansätze sowie einem Ausblick der noch auszuführenden Arbeiten ab.

Show publication details
Kaymak, Yasin; Kuijper, Arjan [Betreuer]; Ben Hmida, Helmi [Betreuer]

Künstliche Intelligenz Widget für ClickDigital

2019

Darmstadt, TU, Bachelor Thesis, 2019

Die in den letzten Jahren vorangetriebene Umsetzung der Vision Internet der Dinge(IoT) hat auch zu einem massiven Zuwachs von IoT-Geräten gesorgt. Dadurch hat im IoT-Umfeld der Aspekt Sicherheit immer mehr an Bedeutung gewonnen. Das Gewährleisten einer reibungslosen und sicheren Nutzung der IoT-Geräte würde auch das Fortsetzten der Vision IoT unterstützen. Eine Gegenmaßnahme Sicherheit zu gewährleisten wären Anomalie-Erkennungsalgorithmen. Diese Algorithmen sind in der Lage, die von den IoT-Geräten erzeugten Unmengen von Daten (Big Data) nach Fehlverhalten zu analysieren, um überhaupt die Möglichkeit anzubieten, Gegenmaßnahmen zu treffen. In dieser Arbeit wird ein Algorithmus vorgestellt, der die von IoT-Geräten erzeugten Daten nach Fehlverhalten untersuchen kann. Zudem wird dieser Algorithmus Teil der Entwicklungsumgebung ClickDigital.

Show publication details
Uecker, Marc; Kügler, David [1. Prüfer]; Kuijper, Arjan [2. Prüfer]

LEMBAS: Latent Embedding Based Architecture Search

2019

Darmstadt, TU, Bachelor Thesis, 2019

In the field of computer vision, deep learning is an essential element of many applications. Its success has long extended to various industries, far beyond machine learning research. Many state-of-the-art approaches succeed by developing custom neural network architectures for specific tasks. However, most practical applications borrow pre-defined architectures from standard image classification benchmarks. While this approach is useful if the benchmark task closely resembles the target task, it does not guarantee good performance in other circumstances. Therefore, designing custom architectures would likely improve the performance of many applications. Neural network design requires experience, statistical knowledge, and experimentation. Many developers lack expertise, computational resources, or time for this process. This is especially true for researchers outside the machine learning field. One solution to this problem is presented by neural architecture search. It is an automated approach to neural architecture design. Existing methods deliver good performance, but often require datacenter-scale computation power. In this thesis, the process of automatic neural architecture search is investigated in a portable, low- resource setting. The presented methods are an adaptable, low-cost solution for searching neural architectures on custom tasks. This work also provides an extensible codebase, which natively scales to various hardware configurations.

Show publication details
Schulz, Hans-Jörg; Röhlig, Martin; Nonnemann, Lars; Aehnelt, Mario; Diener, Holger; Urban, Bodo; Schumann, Heidrun

Lightweight Coordination of Multiple Independent Visual Analytics Tools

2019

IVAPP 2019. Proceedings

International Conference on Information Visualization Theory and Applications (IVAPP) <10, 2019, Prague, Czech Republic>

With the advancement of Visual Analytics (VA) and its spread into various application fields comes along a specialization of methods and tools. This adds complexity and requires extra effort when devising domain-dependent VA solutions, as for every new domain question a new specialized tool or framework must be developed. In this paper, we investigate the possibility of using and re-using existing tools – domain-dependent and general-purpose – by loosely coupling them into specialized VA tool ensembles as needed. We call such coupling among independent tools lightweight coordination, as it is minimally-invasive, pair-wise, and opportunistic in utilizing whichever interface a VA tool offers. We propose the use of lightweight coordination for managing the workflow, the data flow, and the control flow among VA tools, and we show how it can be supported with suitable setups of the multiple tool UIs involved. This concept of lightweight coordination is exemplified with a health care scenario, where an ensemble of independent VA tools is used in a concerted way to pursue the visual analysis of a patient’s troublesome vital data.

  • 978-989-758-354-4
Show publication details
Montefiori, Erica; Modenese, Luca; Di Marco, Roberto; Magni-Manzoni, Silvia; Malattia, Clara; Petrarca, Maurizio; Ronchetti, Anna; Tanturri De Horatio, Laura; Dijkhuizen, Pieter van; Wang, Anqi; Wesarg, Stefan; Viceconti, Marco; Mazzà, Claudia

Linking Joint Impairment and Gait Biomechanics in Patients with Juvenile Idiopathic Arthritis

2019

Annals of Biomedical Engineering

Juvenile Idiopathic Arthritis (JIA) is a paediatric musculoskeletal disease of unknown aetiology, leading to walking alterations when the lower-limb joints are involved. Diagnosis of JIA is mostly clinical. Imaging can quantify impairments associated to inflammation and joint damage. However, treatment planning could be better supported using dynamic information, such as joint contact forces (JCFs). To this purpose, we used a musculoskeletal model to predict JCFs and investigate how JCFs varied as a result of joint impairment in eighteen children with JIA. Gait analysis data and magnetic resonance images (MRI) were used to develop patient-specific lower-limb musculoskeletal models, which were evaluated for operator-dependent variability (< 3.6°, 0.05 N kg21 and 0.5 BW for joint angles, moments, and JCFs, respectively). Gait alterations and JCF patterns showed high between-subjects variability reflecting the pathology heterogeneity in the cohort. Higher joint impairment, assessed with MRI-based evaluation, was weakly associated to overall joint overloading. A stronger correlation was observed between impairment of one limb and overload of the contralateral limb, suggesting risky compensatory strategies being adopted, especially at the knee level. This suggests that knee overloading during gait might be a good predictor of disease progression and gait biomechanics should be used to inform treatment planning.

Show publication details
Pandikow, Lars; Kuijper, Arjan [1. Gutachten]; Gutbell, Ralf [2. Gutachten]

Localization and Mapping of Monocular Cameras in Large Urban Environments using Virtual City Models

2019

Darmstadt, TU, Master Thesis, 2019

In recent years there has been a lot of progress on the task of simultaneous localization and mapping (SLAM) of image sequences from monocular cameras. Latest methods utilize advancements in machine learning to improve the quality of both the camera tracking as well as the reconstruction of the environment. One of these methods called CNN-SLAM uses a neural network to estimate depth maps for keyframes and fuses them with stereo observations from neighboring image frames. To make use of the images within a geo-referenced context they first have to be localized globally. The use of additional sensors to determine the position and orientation of the camera is not only more expensive, but also sensitive to errors. This thesis proposes a real-time system that combines the methods of CNN-SLAM with image based localization within a simple city model. The SLAM algorithm tracks the camera movement while the use of a depth estimation network enables the recovery of the scale of the scene. A genetic algorithm is implemented to quickly refine estimated camera poses by aligning synthetic views of the city model with semantic segmentations of the images. This does not only localize the camera trajectories but also helps to compensate tracking errors caused by the SLAM algorithm. The evaluation showed the systems ability to compute scaled trajectories correctly, to compensate tracking drift and densly reconstruct the scene in the vicinity of the camera. It also revealed the unreliability of image localization without a constraint search space, tracking drift during rotational movement and inaccurate semantic segmentations.

Show publication details

Minutiae-Based Gender Estimation for Full and Partial Fingerprints of Arbitrary Size and Shape

2019

Computer Vision - ACCV 2018

Asian Conference on Computer Vision (ACCV) <14, 2018, Perth, Australia>

Lecture Notes in Computer Science (LNCS)
11361

Since fingerprints are one of the most widely deployed biometrics, accurate fingerprint gender estimation can positively affect several applications. For example, in criminal investigations, gender classification may significantly minimize the list of potential subjects. Previous work mainly offered solutions for the task of gender classification based on complete fingerprints. However, partial fingerprint captures are frequently occurring in many applications, including forensics and the fast growing field of consumer electronics. Due to its huge variability in size and shape, gender estimation on partial fingerprints is a challenging problem. Therefore, in this work we propose a flexible gender estimation scheme by building a gender classifier based on an ensemble of minutiae. The outputs of the single minutia gender predictions are combined by a novel adjusted score fusion approach to obtain an enhanced gender decision. Unlike classical solutions this allows to deal with unconstrained fingerprint parts of arbitrary size and shape. We performed investigations on a publicly available database and our proposed solution proved to significantly outperform state-of-the-art approaches on both full and partial fingerprints. The experiments indicate a reduction in the gender estimation error by 19.34% on full fingerprints and 28.33% on partial captures in comparison to previous work.

  • 978-3-030-20886-8
Show publication details
Lengauer, Stefan; Komar, Alexander; Labrada, Arniel; Karl, Stephan; Trinkl, Elisabeth; Preiner, Reinhold; Bustos, Benjamin; Schreck, Tobias

Motif-driven Retrieval of Greek Painted Pottery

2019

GCH 2019

Eurographics Workshop on Graphics and Cultural Heritage (GCH) <17, 2019, Sarajevo, Bosnia and Herzegovina>

The analysis of painted pottery is instrumental for understanding ancient Greek society and human behavior of past cultures in Archaeology. A key part of this analysis is the discovery of cross references to establish links and correspondences. However, due to the vast amount of documented images and 3D scans of pottery objects in today’s domain repositories, manual search is very time consuming. Computer aided retrieval methods are of increasing importance. Mostly, current retrieval systems for this kind of cultural heritage data only allow to search for pottery of similar vessel’s shape. However, in many cases important similarity cues are given by motifs painted on these vessels. We present an interactive retrieval system that makes use of this information to allow for a motif-driven search in cultural heritage repositories. We address the problem of unsupervised motif extraction for preprocessing and the shape-based similarity search for Greek painted pottery. Our experimental evaluation on relevant repository data demonstrates effectiveness of our approach on examples of different motifs of interests.

Show publication details
Chegini, Mohammad; Andrews, Keith; Schreck, Tobias; Sourin, Alexei

Multiple Linked-View Exploration on Large Displays Facilitated by a Secondary Handheld Device

2019

International Workshop on Advanced Image Technology (IWAIT) 2019

International Workshop on Advanced Image Technology (IWAIT) <2019, Singapore>

Proceedings of SPIE
11049

Large displays are capable of visualising a large amount of data on multiple views including scatterplots and parallel coordinates and are often present in meeting rooms. They can be used to interact with a dataset and foster discussion among team members. Although some of these large screens have multi-touch capabilities, in many cases it is cumbersome to have to stand close to the display in order to interact with it. One of the solutions is to use a small handheld display to interact with the large display. This paper discusses how traditional interactions such as selection, brushing, and linking can be performed using a secondary handheld device. As a proof of concept, a system including scatterplots and parallel coordinates views is implemented. The interactions are straightforward and are useful for any interactive visual analysis application on a large display with wireless connectivity.

Show publication details

Nocturnal Respiration Pattern of healthy people as a hint for sleep state detection

2019

Proceedings of the 12th ACM International Conference on PErvasive Technologies Related to Assistive Environments

ACM International Conference on PErvasive Technologies Related to Assistive Environments (PETRA) <12, 2019, Rhodes, Greece>

ACM International Conference Proceedings Series (ICPS)
01608

Sleep state detection is important to distinguish between a healthy sleep and sleep disorders. Common sleep state analysis methods consist of identifying signals of EEG, EOG, or EMG etc. that can only be assessed in sleep laboratories. The respiration rate and pattern are also affected by the sleep states but are not included in the sleep state analysis method. Since sleep is very important for the recreation of humans, we assume that sleep is mirroring the strain of the day and the general health condition. In our research, we identified a certain respiration rate pattern during sleep in 5 out of 17 healthy persons that might be an identifier for sleep states or for interactions of daytime activity and sleep. Therefore, we introduce this new respiration pattern as “pumping breathing” and compare it with other known respiration patterns.

Show publication details
Debiasi, Luca; Damer, Naser; Moseguí Saladié, Alexandra; Rathgeb, Christian; Scherhag, Ulrich; Busch, Christoph; Kirchbuchner, Florian; Uhl, Andreas

On the Detection of GAN-Based Face Morphs Using Established Morph Detectors

2019

Image Analysis and Processing - ICIAP 2019

International Conference of Image Analysis and Processing (ICIAP) <20, 2019, Trento, Italy>

Lecture Notes in Computer Science (LNCS)
11752

Face recognition systems (FRS) have been found to be highly vulnerable to face morphing attacks. Due to this severe security risk, morph detection systems do not only need to be robust against classical landmark-based face morphing approach (LMA), but also future attacks such as neural network based morph generation techniques. The focus of this paper lies on an experimental evaluation of the morph detection capabilities of various state-of-the-art morph detectors with respect to a recently presented novel face morphing approach, MorGAN, which is based on Generative Adversarial Networks (GANs). In this work, existing detection algorithms are confronted with different attack scenarios: known and unknown attacks comprising different morph types (LMA and MorGAN). The detectors’ performance results are highly dependent on the features used by the detection algorithms. In addition, the image quality of the morphed face images produced with the MorGAN approach is assessed using well-established no-reference image quality metrics and compared to LMA morphs. The results indicate that the image quality of MorGAN morphs is more similar to bona fide images compared to classical LMA morphs.

  • 978-3-030-30644-1
Show publication details

Ontology in Holonic Cooperative Manufacturing: A Solution to Share and Exchange the Knowledge

2019

Knowledge Discovery, Knowledge Engineering and Knowledge Management

International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management (IC3K) <9, 2017, Funchal, Madeira, Portugal>

Communications in Computer and Information Science (CCIS)
976

Cooperative manufacturing is a new trend in industry, which depends on the existence of a collaborative robot. A collaborative robot is usually a light-weight robot which is capable of operating safely with a human co-worker in a shared work environment. During this cooperation, a vast amount of information is exchanged between the collaborative robot and the worker. This information constructs the cooperative manufacturing knowledge, which describes the production components and environment. In this research, we propose a holonic control solution, which uses the ontology concept to represent the cooperative manufacturing knowledge. The holonic control solution is implemented as an autonomous multi-agent system that exchanges the manufacturing knowledge based on an ontology model. Ultimately, the research illustrates and implements the proposed solution over a cooperative assembly scenario, which involves two workers and one collaborative robot, whom cooperate together to assemble a customized product.

  • 978-3-030-15639-8
Show publication details
Wientapper, Folker; Kuijper, Arjan [Referent]; Fellner, Dieter W. [Referent]; Stricker, Didier [Betreuer]

Optimal Spatial Registration of SLAM for Augmented Reality

2019

Darmstadt, TU., Diss., 2019

Augmented reality (AR) is a paradigm that aims at fusing the perceived real environment of a human with digital information located in 3D space. Typically, virtual 3D graphics are overlayed into the captured images of a moving camera or directly into the user's field-of-view by means of optical see-through displays (OST). For a correct perspective and view-dependent alignment of the visualization, it is required to solve various static and dynamic geometric registration problems in order to create the impression that the virtual and the real world are seamlessly interconnected.The advances during the last decade in the field of simultaneous localization and mapping (SLAM) represent an important contribution to this general problem. It is now possible to reconstruct the real environment and to simultaneously capture the dynamic movements of a camera from the images without having to instrument the environment in advance. However, SLAM in general can only partly solve the entire registration problem, because the retrieved 3D scene geometry and the calculated motion path are spatially related only with regard to an arbitrarily selected coordinate system. Without a proper reconciliation of coordinate systems (spatial registration), the real world of the human observer still remains decoupled from the virtual world. Existing approaches for solving this problem either require the availability of a virtual 3D model that represents a real object with sufficient accuracy (model-based tracking), or they rely on use-case specific assumptions and additional sensor data (such as GPS signals or the Manhattan-world assumption). Therefore, these approaches are bound to these additional prerequisites, which limit the general applicability. The circumstance that automated registration is desirable but not always possible, creates the need for techniques that allow a user to specify connections between the real and the virtual world when setting up AR applications, so that it becomes possible to support and control the process of registration. These techniques must be complemented with numerical algorithms that optimally exploit the provided information to obtain precise registration results.Within this context, the present thesis provides the following contributions.* We propose a novel, closed-form (non-iterative) algorithm for calculating a Euclidean or a similarity transformation. The presented algorithm is a generalization of recent state-of-the-art solvers for computing the camera pose based on 2D measurement points in the image (perspective-n-point problem) - a fundamental problem in computer vision that has attracted research for many decades. The generalization consists in extending and unifying these algorithms, so that they can handle other types of input correspondences than originally designed for. With this algorithm, it becomes possible to perform a rigid registration of SLAM systems to a target coordinate system based on heterogeneous and partially indeterminate input data.* We address the global refinement of structure and motion parameters by means of iterative sparse minimization (bundle adjustment or BA), which has become a standard technique inside SLAM systems. We propose a variant of BA in which information about the virtual domain is integrated as constraints by means of an optimization-on-manifold approach. This serves for compensating low-frequency deformations (non-rigid registration) of the estimated camera path and the reconstructed scene geometry caused by measurement error accumulation and the ill-conditionedness of the BA problem.* We present two approaches in which a user can contribute with his knowledge for registering a SLAM system. In a first variant, the user can place markers in the real environment with predefined connections to the virtual coordinate system. Precise positioning of the markers is not required, rather they can be placed arbitrarily on surfaces or along edges, which notably facilitates the preparative effort. During run-time, the dispersed information is collected and registration is accomplished automatically. In a second variant, the user is given the possibility to mark salient points in an image sequence during a preparative preprocessing step and to assign corresponding points in the virtual 3D space via a simple point-and-click metaphor. The result of this preparative phase is a precisely registered and ready-to-use reference model for camera tracking at run-time.* Finally, we propose an approach for geometric calibration of optical see-trough displays. We present a parametric model, which allows to dynamically adapt the rendering of virtual 3D content to the current viewpoint of the human observer, including a pre-correction of image aberrations caused by the optics or irregularly curved combiners. In order to retrieve its parameters, we propose a camera-based approach, in which elements of the real and the virtual domain are simultaneously observed. The calibration procedure was developed for a head-up display in a vehicle. A prototypical extension to head-mounted displays is also presented.

Show publication details
Fauser, Johannes; Stenin, Igor; Kristin, Julia; Klenzner, Thomas; Schipper, Jörg; Mukhopadhyay, Anirban

Optimizing Clearance of Bézier Spline Trajectories for Minimally-Invasive Surgery

2019

Medical Image Computing and Computer Assisted Intervention - MICCAI 2019

International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI) <22, 2019, Shenzhen, China>

Lecture Notes in Computer Science (LNCS)
11768

Preoperative planning of nonlinear trajectories is a key element in minimally-invasive surgery. Interpolating between start and goal of an intervention while circumnavigating risk structures provides the necessary feasible solutions for such procedure. While recent research shows that Rapidly-exploring Random Trees (RRT) on B´ezier Splines efficiently solve this task, access paths computed by this method do not provide optimal clearance to surrounding anatomy. We propose an approach based on sequential convex optimization that rearranges B´ezier Splines computed by an RRT-connect, thereby achieving locally optimal clearance to risk structures. Experiments on real CT data of patients demonstrate the applicability of our approach on two scenarios: catheter insertion through the aorta and temporal bone surgery. We compare distances to risk structures along computed trajectories with the state of the art solution and show that our method results in clinically safer paths.

  • 978-3-030-32254-0
Show publication details
Getto, Roman; Fellner, Dieter W. [Betreuer]; Schreck, Tobias [Betreuer]

Parametric Procedural Models for 3D Object Retrieval, Classification and Parameterization

2019

Darmstadt, TU., Diss., 2019

The amount of 3D objects has grown over the last decades, but we can expect that it will grow much further in the future. 3D objects are also becoming more and more accessible to non-expert users. The growing amount of available 3D data is welcome for everyone working with this type of data, as the creation and acquisition of many 3D objects is still costly. However, the vast majority of available 3D objects are only present as pure polygon meshes. We arguably can not assume to get meta-data and additional semantics delivered together with 3D objects stemming from non-expert or 3D scans of real objects from automatic systems. For this reason content-based retrieval and classification techniques for 3D objects has been developed.Many systems are based on the completely unsupervised case. However, previous work has shown that there are strong possibilities of highly increasing the performance of these tasks by using any type of previous knowledge. In this thesis I use procedural models as previous knowledge. Procedural models describe the construction process of a 3D object instead of explicitly describing the components of the surface. These models can include parameters into the construction process to generate variations of the resulting 3D object. Procedural representations are present in many domains, as these implicit representations are vastly superior to any explicit representation in terms of content generation, flexibility and reusability. Therefore, using a procedural representation always has the potential of outclassing other approaches in many aspects. The usage of procedural models in 3D object retrieval and classification is not highly researched as this powerful representation can be arbitrary complex to create and handle. In the 3D object domain, procedural models are mostly used for highly regularized structures like buildings and trees.However, Procedural models can deeply improve 3D object retrieval and classification, as this representation is able to offer a persistent and reusable full description of a type of object. This description can be used for queries and class definitions without any additional data. Furthermore, the initial classification can be improved further by using a procedural model: A procedural model allows to completely parameterize an unknown object and further identify characteristics of different class members. The only drawback is that the manual design and creation of specialized procedural models itself is very costly. In this thesis I concentrate on the generalization and automation of procedural models for the application in 3D object retrieval and 3D object classification.For the generalization and automation of procedural models I propose to offer different levels of interaction for a user to fulfill the possible needs of control and automation. This thesis presents new approaches for different levels of automation: the automatic generation of procedural models from a single exemplary 3D object. The semi-automatic creation of a procedural model with a sketch-based modeling tool. And the manual definition a procedural model with restricted variation space. The second important step is the insertion of parameters into the procedural model, to define the variations of the resulting 3D object. For this step I also propose several possibilities for the optimal level of control and automation: An automatic parameter detection technique. A semi-automatic deformation based insertion. And an interface for manually inserting parameters by choosing one of the offered insertion principles. It is also possible to manually insert parameters into the procedures if the user needs the full control on the lowest level.To enable the usage of procedural models directly for 3D object retrieval and classification techniques I propose descriptor-based and deep learning based approaches. Descriptors measure the difference of 3D objects. By using descriptors as comparison algorithm, we can define the distance between procedural models and other objects and order these by similarity. The procedural models are sampled and compared to retrieve an optimal object retrieval list. We can also directly use procedural models as data basis for a retraining of a convolutional neural network. By deep learning a set of procedural models we can directly classify new unknown objects without any further large learning database. Additionally, I propose a new multi-layered parameter estimation approach using three different comparison measures to parameterize an unknown object. Hence, an unknown object is not only classified with a procedural model but the approach is also able to gather new information about the characteristics of the object by using the procedural model for the parameterization of the unknown object.As a result, the combination of procedural models with the tasks of 3D object retrieval and classification lead to a meta concept of a holistically seamless system of defining, generating, comparing, identifying, retrieving, recombining, editing and reusing 3D objects.

Show publication details

Performing Indoor Localization with Electric Potential Sensing

2019

Journal of Ambient Intelligence and Humanized Computing

Location-based services or smart home applications all depend on an accurate indoor positioning system. Basically one divides these systems into token-based and token-free localization systems. In this work, we focus on the token-free system based on smart floor technology. Smart floors can typically be built using pressure sensors or capacitive sensors. However, these set-ups are often hard to deploy as mechanical or electrical features are required below the surface and even harder to replace when detected a sensor malfunctioning. Therefore we present a novel indoor positioning system using an uncommon form of passive electric field sensing (EPS), which detects the electric potential variation caused by body movement. The EPS-based smart floor set-up is easy to install by deploying a grid of passive electrode wires underneath any non-conductive surfaces. Easy maintenance is also ensured by the fact that the sensors are not placed underneath the surface, but on the side. Due to the passive measuring nature, low power consumption is achieved as opposed to active capacitive measurement. Since we do not collect image data as in visual-based systems and all sensor data is processed locally, we preserve the user’s privacy. The proposed architecture achieves a high position accuracy and an excellent spatial resolution. Based on our evaluation conducted in our living lab, we measure a mean positioning error of only 12.7 cm.

Show publication details
Jabhe, Ali Khesrau; Kuijper, Arjan [1. Prüfer]; Kügler, David [2. Prüfer]

Physical World Attacks in Medical Imaging

2019

Darmstadt, TU, Bachelor Thesis, 2019

During this decade Deep Learning (DL) has become a game changer in the area of MedicalImaging. More and more tasks whether classification, detection, segmentation, etc. are beingsolved by DL algorithms with significantly high performance gains compared to traditionalmethods. However, Deep Neural Network (DNN)-based classification models have been foundto be vulnerable to adversarial attacks - maliciously and digitally manipulated image data -resulting in misclassification of the model.While only observed in Medical Imaging, adversarial examples define an established securitythreat to computer vision tasks. Furthermore, recent research has demonstrated theexistence of adversarial examples in the physical world - perturbations by physical artifactscausing misclassification instead of digitally perturbed data fed into the model. Since the existenceof adversarial examples in the physical world is not yet explored in Medical Imagingand trends are towards DL methods starting to get approved for clinical practice, this problemneeds more attention.Using the example of skin lesion classification, as it is likely to be the next aspirant for afully automated medical product, this work throws light on the potential threat adversarialexamples in the physical world pose to patients, physicians, and the healthcare system by generatinga new dataset "Physical Attacks in Dermoscopy" (PADv1) and evaluate susceptibilityand robustness of five state-of-the-art DNN-based skin lesion classifiers under physical attack.All architectures were trained on the HAM10000 dataset, a comprehensive challenge-verifiedbenchmark dataset for Machine Learning. Results show that on average the accuracy of thesearchitectures drops by 30.8%, and robustness decreases by 50% which means that every seconddiagnosis is fooled. Furthermore, a visualization of Gradient-weighted class activationmaps is provided to help understand or even interpret a models decision-making on clean andattacked image data.

Show publication details
Fauser, Johannes; Chadda, Romol; Goergen, Yannik; Hessinger, Markus; Motzki, Paul; Stenin, Igor; Kristin, Julia; Klenzner, Thomas; Schipper, Jörg; Seelecke, Stefan; Werthschützky, Roland; Kupnik, Mario; Mukhopadhyay, Anirban

Planning for Flexible Surgical Robots via Bézier Spline Translation

2019

IEEE Robotics and Automation Letters

In a minimally invasive surgery, new flexible instruments enable safer and easier access to difficult-to-reach anatomical regions. However, their introduction into the clinical workflow requires robust replanning because navigation errors during surgery render initially planned trajectories infeasible. Such replanning requires to regularly solve an expensive two-point boundary value problem (BVP) that connects the target pose of the instrument with the currently measured one. We propose a hybrid planning scheme that features both robust and safe replanning. This two-step approach first solves the BVP and then transforms the result to circular arcs that fit the motion of our instruments' models. We exploit implicitly defined Bézier splines as a robust method for interpolation in the first step. A novel geometric translation of these splines, then, provides a convenient solution for movement along circular arcs. We consider two example applications: 1) planning for a drilling unit in temporal bone surgery; and 2) guidewires in catheter insertion. Evaluation on real patient data of both temporal bone and aorta show that our proposed hybrid two-step approach achieves, on average, 55% higher replanning rates and provides 31% larger clearance to risk structures, thus improving trajectory quality with regard to clinical safety.

Show publication details

Presenting a Data Imputation Concept to Support the Continuous Assessment of Human Vital Data and Activities

2019

Proceedings of the 12th ACM International Conference on PErvasive Technologies Related to Assistive Environments

ACM International Conference on PErvasive Technologies Related to Assistive Environments (PETRA) <12, 2019, Rhodes, Greece>

ACM International Conference Proceedings Series (ICPS)
01608

Data acquisition of mobile tracking devices often suffers from invalid and non-continuous input data streams. This issue especially occurs with current wearables tracking the user’s activity and vital data. Typical reasons include the short battery life and the fact that the body-worn tracking device may be doffed. Other reasons, such as technical issues, can corrupt the data and render it unusable. In this paper, we introduce a data imputation concept which complements and thus fixes incomplete datasets by using a new merging approach that is particularly suitable for assessing activities and vital data. Our technique enables the dataset to become coherent and comprehensive so that it is ready for further analysis. In contrast to previous approaches, our technique enables the controlled creation of continuous data sets that also contain information on the level of uncertainty for possible reconversions, approximations, or later analysis.

Show publication details
Lenhart, Malte; Kuijper, Arjan [Betreuer]; Wilmsdorff, Julian von [Betreuer]

Prototyping Platform for Capacitive and Passive Electrical Field Sensing

2019

Darmstadt, TU, Bachelor Thesis, Jahr

In this thesis the Linoc prototyping toolkit is presented. It is built around two capac- itive and two Electric Potential Sensing (EPS) groups providing unobtrusive proximity detection in the field of Human Computer Interface (HCI). The toolkits focus lies on its usability in order to be adapted in future research and novel use cases. Its strength is the possibility to change its configuration at run time. A common obstacle in the beginning of a project is the time required to familiarize with present tools and systems, before the actual project can be attended to. This can be made worse by dependencies on previous work, often not fully documented and without training given from the original designer. Good toolkits can help to overcome this problem by providing a layer of abstraction and allowing to work on a higher level. If the toolkit however requires too much time to familiarize or behaves too restrictive, its goal has been missed and no benefits are generated. To access the quality of this thesis’ work, the Linoc toolkit was evaluated in terms of three different aspects: demonstration, usage and technical performance. A usage study found good reception, a steep learning curve and an interest to use the toolkit in the future. Technical benchmarks show a detectable range equal to its predecessors and in a demonstration it was shown that the toolkit can actually be used in projects.

Show publication details
Bulduk, Botan; Stork, André [1. Gutachten]; Luu, Thu Huong [2. Gutachten]

Prozedurale innere Strukturen für den 3D-Druck

2019

Darmstadt, TU, Bachelor Thesis, 2019

In dieser Arbeit wird eine neue Methode zur Erstellung von inneren Strukturen für den 3D-Druck vorgestellt. Diese basiert auf Skeletten mit denen sich Volumen generieren lassen können.

Show publication details
Mukhopadhyay, Anirban; Kügler, David; Bucher, Andreas; Fellner, Dieter W.; Vogl, Thomas J.

Putting Trust First in the Translation of AI for Healthcare

2019

Ercim News

From screening diseases to personalised precision treatments, AI is showing promise in healthcare. But how comfortable should we feel about giving black box algorithms the power to heal or kill us?

Show publication details

Real-time texturing for 6D object instance detection from RGB Images

2019

Adjunct Proceedings of the 2019 IEEE International Symposium on Mixed and Augmented Reality

IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct) <18, 2019, Beijing, China>

For objected detection, the availability of color cues strongly influences detection rates and is even a prerequisite for many methods. However, when training on synthetic CAD data, this information is not available. We therefore present a method for generating a texture-map from image sequences in real-time. The method relies on 6 degree-of-freedom poses and a 3D-model being available. In contrast to previous works this allows interleaving detection and texturing for upgrading the detector on-the-fly. Our evaluation shows that the acquired texture-map significantly improves detection rates using the LINEMOD [5] detector on RGB images only. Additionally, we use the texture-map to differentiate instances of the same object by surface color.

Show publication details
Urban, Philipp; Tanksale, Tejas Madan; Brunton, Alan; Vu, Bui Minh; Nakauchi, Shigeki

Redefining A in RGBA: Towards a Standard for Graphical 3D Printing

2019

ACM Transactions on Graphics

Advances inmultimaterial 3D printing have the potential to reproduce various visual appearance attributes of an object in addition to its shape. Since many existing 3D file formats encode color and translucency by RGBA texturesmapped to 3Dshapes, RGBA information is particularly important for practical applications. In contrast to color (encoded by RGB), which is specified by the object’s reflectance, selected viewing conditions, and a standard observer, translucency (encoded by A) is neither linked to any measurable physical nor perceptual quantity. Thus, reproducing translucency encoded by A is open for interpretation. In this article, we propose a rigorous definition for A suitable for use in graphical 3D printing, which is independent of the 3D printing hardware and software, and which links both optical material properties and perceptual uniformity for human observers. By deriving our definition from the absorption and scattering coefficients of virtual homogenous reference materials with an isotropic phase function, we achieve two important properties. First, a simple adjustment of A is possible, which preserves the translucency appearance if an object is rescaled for printing. Second, determining the value of A for a real (potentially non-homogenous) material, can be achieved by minimizing a distance function between light transport measurements of this material and simulated measurements of the reference materials. Such measurements can be conducted by commercial spectrophotometers used in graphic arts. Finally, we conduct visual experiments employing the method of constant stimuli, and we derive from them an embedding of A into a nearly perceptually uniform scale of translucency for the reference materials.

Show publication details
Boutros, Fadi; Kuijper, Arjan [1. Gutachten]; Damer, Naser [2. Gutachten]

Reducing Ethnic Bias of Face Recognition by Ethnic Augmentation

2019

Darmstadt, TU, Master Thesis, 2019

Automated face recognition has gained wider deployment ground after the recent accuracy gains achieved by deep learning techniques. Despite the rapid performance gains, face recognition still suffers from very critical issues. One of the recently uncovered, and very sensitive, challenges is the ethnicity bias in face recognition decision. This is the case, unfortunately, even in the latest commercial and academic face recognition system. In 2018, the National Institute of Standards and Technology (NIST) published the latest report regarding the evaluation result of commercial face recognition solutions from several major biometric vendors. The report specifically evaluated and demonstrated the variance of the error rates of the evaluated solutions based on demographic variations. This thesis is one of the first research efforts in addressing the decision bias challenge in biometrics. It builds its hypothesis on the strong assumption that ethnicity bias is caused by the relative under-representation of certain ethnicities in training databases. This work introduces a novel ethnicity-driven data augmentation approach to reduce biometric bias. The proposed approach successfully utilize a generative image model to generate new face images that preserve the identity of their source images while partially transforming their ethnicity to the targeted ethnicity group. A large-scale ethnicity-labeled face images database is created in order to develop and evaluate the proposed approach. To achieve that, part of this thesis focused on creating an ethnicity classifier to annotate face images, achieving accuracy in the state-of-the-art range. The proposed ethnicity-driven face generative model is developed based on the ethnicity labeled images to generate realistically and high-resolution face images, depending on a limit amount of training data. More importantly, the thesis proves that the proposed augmentation approach strongly preserves the identity of the input images and partially transforms the ethnicity. The augmented images are used as part of the training data of a face recognition model. The achieved verification results prove that the proposed ethnicity augmentation methods significantly and consistently reduced the ethnicity bias of the face recognition model. For examples, the ERR was reduced from 0.159 to 0.130 when verifying inter-ethnicity samples of Black individuals on a model trained on Asian individual images, respectively before and after applying the proposed training data augmentation. Moreover, the overall performance of the face recognition model was improved. However, this improvement was more significant, as intended, in the targeted ethnicity groups.

Show publication details
Fuchs, Moritz; Kuijper, Arjan [Betreuer]; Noll, Matthias [Betreuer]

Registrierung eines Biopsieroboters zu bildbasierten Planungsdaten

2019

Darmstadt, TU, Bachelor Thesis, 2019

Das Setzen eines perkutanen Zugangs durch einen Roboter ist erstrebenswert, um die Genauigkeit von Eingriffen, wie Biopsien, thermale und non-thermale Ablation, zu verbessern. Für das dafür verwendete Robotersystem ist es notwendig, die Transformation des Roboters innerhalb eines CT zu bestimmen. Der Roboter hat hierfür ein Registrierungswerkzeug, dass auch die Nadelführung beinhaltet, an seinem Endeffektor angebracht. Das Registrierungswerkzeug beinhaltet vier Kugeln, die als Marker für die Ermittlung der Transformation dienen. In dieser Arbeit wurde sich damit beschäftigt die Kugeln zu lokalisieren. Es wird hier zu ein zweistufiges Verfahren verwendet, dass zunächst die eine Region of Interest(ROI) lokalisiert um anschließend die genau Transformation zu ermitteln. Für die Ermittlung der genauen Transformation wurden Optimierungsverfahren getestet, als auch Merkmal-Extraaktionstechniken. Als Optimierungsverfahren werden der Amoeba Algorithmus und der Gradientenabstieg, in Verbindung mit verschieden Metriken und Modellen getestet. Jedoch sind die Ergebnisse der Optimierer abhängig von der Initialisierung. Eine fehlerhafte Initialisierung konnte von den Optimierern nicht ausgeglichen werden. Bei der Merkmal-Extraaktionstechnik werden die Mittelpunkte der vier Kugeln bestimmt und zu einem Modell registriert. Zur Bestimmung der Kugelmittelpunkte wurde die Hough-Transformation verwendet. Es lässt damit eine zuverlässige Genauigkeit von DICE = 0.95879 auf den alten Registrierungswerkzeugen erreichen. Durch die bessere Fertigung und Bildqualität ist auf neueren Registrierungswerkzeugen ein DICE = 0.98515 möglich. Die Translationsabweichung ist dabei auf Sub-Voxel-Spacing genau.

Show publication details
Frank, Sebastian; Kuijper, Arjan

Robust Driver Foot Tracking and Foot Gesture Recognition Using Capacitive Proximity Sensing

2019

Journal of Ambient Intelligence and Smart Environments

Nowadays, there is an increasing trend towards automated driving. This is supported by both driver assistance systems getting more and more available and powerful, and research for car manufacturing industries. As a consequence, driver hands and feet are less involved in vehicle control. Increasing automation will even let them become idle. Recent gesture recognition mainly focuses on hand interaction. This work focuses on possibilities for feet gesture interaction. Many gesture recognition systems rely on computing intensive, privacy concerns causing video systems. Furthermore, these systems require a line of sight and therefore visible interior design integration. The proposed system shows that invisibly integrated capacitive proximity sensors can do the job, too. They do not cause privacy issues and they can be integrated under non-conductive materials. Therefore, there is no visible interior design impact. The proposed solution distinguishes between four feet gestures. There is no limitation to feet movement. Further, an evaluation including six participants and a vehicle legroom mockup shows the system function. This work contributes to the basis of driver foot gesture recognition pointing to further applications and more comprehensive investigations.

Show publication details

Seamless and Non-repetitive 4D Texture Variation Synthesis and Real-time Rendering for Measured Optical Material Behavior

2019

Computational Visual Media

We show how to overcome the single weakness of an existing fully automatic system for acquisition of spatially varying optical material behavior of real object surfaces. While the expression of spatially varying material behavior with spherical dependence on incoming light as a 4D texture (an ABTF material model) allows flexible mapping onto arbitrary 3D geometry, with photo-realistic rendering and interaction in real time, this very method of texture-like representation exposes it to common problems of texturing, striking in two disadvantages. Firstly, non-seamless textures create visible artifacts at boundaries. Secondly, even a perfectly seamless texture causes repetition artifacts due to their organised placement in large numbers over a 3D surface. We have solved both problems through our novel texture synthesis method that generates a set of seamless texture variations randomly distributed over the surface at shading time. When compared to regular 2D textures, the inter-dimensional coherence of the 4D ABTF material model poses entirely new challenges to texture synthesis, which includes maintaining the consistency of material behavior throughout the 4D space spanned by the spatial image domain and the angular illumination hemisphere. In addition, we tackle the increased memory consumption caused by the numerous variations through a fitting scheme specifically designed to reconstruct the most prominent effects captured in the material model.

Show publication details
Wang, Anqi; Franke, Andreas; Wesarg, Stefan

Semi-automatic Segmentation of JIA-induced Inflammation in MRI Images of Ankle Joints

2019

Medical Imaging 2019: Image Processing

SPIE Medical Imaging Symposium <2019, San Diego, CA, USA>

Proceedings of SPIE
10949

The autoimmune disease Juvenile Idiopathic Arthritis (JIA) affects children of under 16 years and leads to the symptom of inflamed synovial membranes in affected joints. In clinical practice, characteristics of these inflamed membranes are used to stage the disease progression and to predict erosive bone damage. Manual outlining of inflammatory regions in each slide of a MRI dataset is still the gold standard for detection and quantification, however, this process is very tiresome and time-consuming. In addition, the inter- and intra-observer variability is a known problem of human annotators. We have developed the first method to detect inflamed regions in and around major joints in the human ankle. First, we use an adapted coupled shape model framework to segment the ankle bones in a MRI dataset. Based on these segmentations, joints are defined as locations where two bones are particularly close to each other. A number of potential inflammation candidates are generated using multi-level thresholding. Since it is known that inflamed synovial membranes occur in the proximity of joints, we filter out structures with similar intensities such as vessels and tendons sheaths using not only a vesselness filter, but also their distance to the joints and their size. The method has been evaluated on a set of 10 manually annotated clinical MRI datasets and achieved the following results: Precision 0.6785 ± 0.1584, Recall 0.5388 ± 0.1213, DICE 0.5696 ± 0.0976.

Show publication details
Damer, Naser; Saladie, Alexandra Moseguí; Zienert, Steffen; Wainakh, Yaza; Kirchbuchner, Florian; Kuijper, Arjan; Terhörst, Philipp

To Detect or not to Detect: The Right Faces to Morph

2019

The 12th IAPR International Conference On Biometrics

IAPR International Conference on Biometrics (ICB) <12, 2019, Crete, Greece>

Recent works have studied the face morphing attack detection performance generalization over variations in morphing approaches, image re-digitization, and image source variations. However, these works assumed a constant approach for selecting the images to be morphed (pairing) across their training and testing data. A realistic variation in the pairing protocol in the training data can result in challenges and opportunities for a stable attack detector. This work extensively study this issue by building a novel database with three different pairing protocols and two different morphing approaches. We study the detection generalization over these variations for single image and differential attack detection, along with handcrafted and CNNbased features. Our observations included that training an attack detection solution on attacks created from dissimilar face images, in contrary to the common practice, can result in an overall more generalized detection performance. Moreover, we found that differential attack detection is very sensitive to variations in morphing and pairing protocols.

Show publication details
Fauser, Johannes; Stenin, Igor; Bauer, Markus; Hsu, Wei-Hung; Kristin, Julia; Klenzner, Thomas; Schipper, Jörg; Mukhopadhyay, Anirban

Toward an Automatic Preoperative Pipeline for Image-guided Temporal Bone Surgery

2019

International Journal of Computer Assisted Radiology and Surgery

International Conference on Information Processing in Computer-Assisted Interventions (IPCAI) <10, 2019, Rennes, France>

Purpose: Minimally invasive surgery is often built upon a time-consuming preoperative step consisting of segmentation and trajectory planning. At the temporal bone, a complete automation of these two tasks might lead to faster interventions and more reproducible results, benefiting clinical workflow and patient health. Methods: We propose an automatic segmentation and trajectory planning pipeline for image-guided interventions at the temporal bone. For segmentation, we use a shape regularized deep learning approach that is capable of automatically detecting even the cluttered tiny structures specific for this anatomy.We then perform trajectory planning for both linear and nonlinear interventions on these automatically segmented risk structures. Results: We evaluate the usability of segmentation algorithms for planning access canals to the cochlea and the internal auditory canal on 24 CT data sets of real patients. Our new approach achieves similar results to the existing semiautomatic method in terms of Dice but provides more accurate organ shapes for the subsequent trajectory planning step. The source code of the algorithms is publicly available. Conclusion: Automatic segmentation and trajectory planning for various clinical procedures at the temporal bone are feasible. The proposed automatic pipeline leads to an efficient and unbiased workflow for preoperative planning.

Show publication details

Underwater Color Restoration Using U-Net Denoising Autoencoder

2019

Proceedings of the 11th International Symposium Image and Signal Processing and Analysis

International Symposium on Image and Signal Processing and Analysis (ISPA) <11, 2019, Dubrovnik, Croatia>

Visual inspection of underwater structures by vehicles, e.g. remotely operated vehicles (ROVs), plays an important role in scientific, military, and commercial sectors. However, the automatic extraction of information using software tools is hindered by the characteristics of water which degrade the quality of captured videos. As a contribution for restoring the color of underwater images, Underwater Denoising Autoencoder (UDAE) model is developed using a denoising autoencoder with U-Net architecture. The proposed network takes into consideration the accuracy and the computation cost to enable realtime implementation on underwater visual tasks using end-toend autoencoder network. Underwater vehicles perception is improved by reconstructing captured frames; hence obtaining better performance in underwater tasks. Related learning methods use generative adversarial networks (GANs) to generate color corrected underwater images, and to our knowledge this paper is the first to deal with a single autoencoder capable of producing same or better results. Moreover, image pairs are constructed for training the proposed network, where it is hard to obtain such dataset from underwater scenery. At the end, the proposed model is compared to a state-of-the-art method.

Show publication details
Hashisho, Yousif; Lukas, Uwe von [Gutachter]; Staadt, Oliver [Gutachter]; Albadawi, Mohamad [Supervising Advisor]; Krause, Tom [Supervising Advisor]

Underwater Image Enhancement Using Autoencoders

2019

Rostock, Univ., Master Thesis, 2019

Visual inspection of underwater structures by vehicles, e.g. remotely operated vehicles (ROVs), plays an important role in scientific, military, and commercial aspects. However, the automatic extraction of information using software tools is hindered by the optical characteristics of water which degrade the quality of the videos. As a contribution for enhancing underwater images, we develop an algorithm using a single denoising autoencoder to restore the color and remove the disturbances such as marine snow from underwater images. Marine snow in some images is only partially removed using the proposed network; however, we show the reason behind this failure. Related learning methods use generative adversarial networks (GANs) for generating color corrected underwater images, and to our knowledge this thesis is the first to deal with a single autoencoder capable of producing same or better results. Moreover, underwater aligned image pairs are established for the training of the proposed network, where it is hard to obtain such dataset from underwater scenery. The objective is to increase the accuracy and reliability on automatic underwater operations that rely on robotic perception without human interference by improving the quality of the captured frames. At the end, the proposed network is evaluated using Mean Squared Error (MSE), Peak Signal-to- Noise Ratio (PSNR), and Structural Similarity Index Metric (SSIM) quality metrics. Additionally, we compare our experiment with a related method. The proposed network takes into consideration the computation cost and the accuracy to have real-time implementation on visual-driven tasks using a single autoencoder.

Show publication details

Unsupervised Privacy-enhancement of Face Representations Using Similarity-sensitive Noise Transformations

2019

Applied Intelligence

Face images processed by a biometric system are expected to be used for recognition purposes only. However, recent work presented possibilities for automatically deducing additional information about an individual from their face data. By using soft-biometric estimators, information about gender, age, ethnicity, sexual orientation or the health state of a person can be obtained. This raises a major privacy issue. Previous works presented supervised solutions that require large amount of private data in order to suppress a single attribute. In this work, we propose a privacy-preserving solution that does not require these sensitive information and thus, works in an unsupervised manner. Further, our approach offers privacy protection that is not limited to a single known binary attribute or classifier. We do that by proposing similarity-sensitive noise transformations and investigate their effect and the effect of dimensionality reduction methods on the task of privacy preservation. Experiments are done on a publicly available database and contain analyses of the recognition performance, as well as investigations of the estimation performance of the binary attribute of gender and the continuous attribute of age. We further investigated the estimation performance of these attributes when the prior knowledge about the used privacy mechanism is explicitly utilized. The results show that using this information leads to significantly enhancement of the estimation quality. Finally, we proposed a metric to evaluate the trade-off between the privacy gain and the recognition loss for privacy-preservation techniques. Our experiments showed that the proposed cosine-sensitive noise transformation was successful in reducing the possibility of estimating the soft private information in the data, while having significantly smaller effect on the intended recognition performance.

Show publication details
Niemeyer, Frank; Dolereit, Tim; Neumann, Matthias; Albiez, Jan; Vahl, Matthias; Geist, Michael

Untersuchungen von optischen Scansystemen zur geometrischen Erfassung von Unterwasserstrukturen

2019

Hydrographische Nachrichten

Dieser Beitrag beschäftigt sich mit optischen Erfassungs- und Scansystemen für den Bereich unter Wasser. Dabei wurde eine Einteilung in photogrammetrische, trigonometrische und impulsbasierte Systeme vorgenommen. Es wurden Messungen mit verschiedenen Systemen im Schleppkanal des Lehrstuhls für Strömungstechnik der Universität Rostock durchgeführt. Stellvertretend für photogrammetrische Erfassungssysteme kam das Stereokamerasystem vom Fraunhofer-Institut für Graphische Datenverarbeitung zum Einsatz. Das ULS-200-Scansystem von 2GRobotics und das SeaVision- Lasersystem von Kraken Robotik standen stellvertretend für trigonometrische Scansysteme zur Verfügung. Ein impulsbasiertes Scansystem stand für die Untersuchungen nicht bereit. Zusätzlich wurde das SeaVision-Scansystem in der Ostsee nahe des künstlichen Riffs bei Nienhagen unter realistischen Bedingungen getestet. Die Messungen spiegeln das derzeitige Potenzial optischer Messsysteme für den Bereich unter Wasser wider. Vor- und Nachteile der Systeme werden diskutiert.

Show publication details

User Guidance for Interactive Camera Calibration

2019

Virtual, Augmented and Mixed Reality: Multimodal Interaction

International Conference Virtual Augmented and Mixed Reality (VAMR) <11, 2019, Orlando, FL, USA>

Lecture Notes in Computer Science (LNCS)
11574

For building a Augmented Reality (AR) pipeline, the most crucial step is the camera calibration as overall quality heavily depends on it. In turn camera calibration itself is influenced most by the choice of camera-to-pattern poses – yet currently there is only little research on guiding the user to a specific pose. We build upon our novel camera calibration framework that is capable to generate calibration poses in real-time and present a user study evaluating different visualization methods to guide the user to a target pose. Using the presented method even novel users are capable to perform a precise camera calibration in about 2 min.

  • 978-3-030-21606-1
Show publication details
Bernard, Jürgen; Sessler, David; Kohlhammer, Jörn; Ruddle, Roy A.

Using Dashboard Networks to Visualize Multiple Patient Histories: A Design Study on Post-operative Prostate Cancer

2019

IEEE Transactions on Visualization and Computer Graphics

In this design study, we present a visualization technique that segments patients' histories instead of treating them as raw event sequences, aggregates the segments using criteria such as the whole history or treatment combinations, and then visualizes the aggregated segments as static dashboards that are arranged in a dashboard network to show longitudinal changes. The static dashboards were developed in nine iterations, to show 15 important attributes from the patients' histories. The final design was evaluated with five non-experts, five visualization experts and four medical experts, who successfully used it to gain an overview of a 2,000 patient dataset, and to make observations about longitudinal changes and differences between two cohorts. The research represents a step-change in the detail of large-scale data that may be successfully visualized using dashboards, and provides guidance about how the approach may be generalized.

Show publication details

Virtual Reality in Media and Technology

2019

Digital Transformation
Fraunhofer-Forschungsfokus

Virtual and Augmented Reality technologies have by now become established in numerous engineering areas of application. Also in the cultural and media fields interactive three-dimensional content is being increasingly made available for information purposes, and used in scientific research. On the one hand, this development is accelerated by current advances in smartphones, tablets and head-mounted displays. These support complex 3D applications in mobile application scenarios, and enable us to capture our real physical environment using multimodal sensors in order to correlate it with the digital 3D world. On the other hand, new automated digitization technologies such as CultLab3D of the Fraunhofer Institute for Computer Graphics Research IGD allow the production of the necessary digital replicas of real objects, quickly, economically and of high quality.

  • 978-3-662-58133-9
Show publication details
Riaz, Muhammad Ali; Kuijper, Arjan [1. Prüfer]; Burkhardt, Dirk [2. Prüfer]

Visual Trend Analysis on Condensed Expert Data beside Research Library Data for Enhanced Insights

2019

Darmstadt, TU, Master Thesis, 2019

In the present age of information, we live amidst seas of digital text documents including academic publications, white papers, news articles, patents, newspapers. To tackle the issue of the ever-increasing amount of text documents, researchers from the field of text mining and information visualization have developed tools and techniques to facilitate text analysis. In the context of visual trend analysis on text data, the use of well-structured patent data and public digital libraries are quite established. However, both sources of information have their limitations. For instance, the registration process for patents takes at least one year, which makes the extracted insights not suitable to research on present scenarios. In contrast to patent data, the digital libraries are up-to-date but provide high-level insights, only limited to broader research domains, and the data usage is almost restricted on meta information, such as title, author names and abstract, and they do not provide full text. For a certain type of detailed analysis such as competitor analysis or portfolio analysis, data from digital libraries is not enough, it would also make sense to analyze the full-text. Even more, it can be beneficial to analyze only a limited dataset that is filtered by an expert towards a very specific field, such as additive printing or smart wearables for medical observations. Sometimes also a mixture of both digital library data and manually collected documents is relevant to be able to validate a certain trend, where one gives a big picture and other gives a very condensed overview of the present scenario. The thesis aims, therefore, to focus on such manually collected documents by experts that can be defined as condensed data. So, the major goal of this thesis is to conceptualize and implement a solution that enables the creation and analysis of such a condensed data set and compensate therewith the limitations of digital library data analysis. As a result, a visual trend analysis system for analyzing text documents is presented, it utilizes the best of both state-of-the-art text analytics and information visualization techniques. In a nutshell, the presented trend analysis system does two things. Firstly, it is capable of extracting raw data from text documents in the form of unstructured text and meta-data, convert it into structured and analyzable formats, extract trends from it and present it with appropriate visualizations. Secondly, the system is also capable of performing gap-analysis tasks between two data sources, which in this case is digital library data and data from manually collected text documents (Condensed Expert Data). The proposed visual trend analysis system can be used by researchers for analyzing the research trends, organizations to identify current market buzz and industry trends, and many other use-cases where text data is the primary source of valuable information.

Show publication details
Lehmann, Markus; Kuijper, Arjan [1. Gutachten]; Bernard, Jürgen [2. Gutachten]

Visual-Interactive Combination of Selection Strategies to Improve Data Labeling Processes

2019

Darmstadt, TU, Master Thesis, 2019

Labeling training data is an important task in Machine Learning for building effective and efficient classifiers. There are different approaches to gather labeled instances for a particular data set. The two most important fields of strategies are Active Learning and Visual-Interactive Labeling. In previous work, these strategies were examined and compared, resulting in a set of atomic labeling strategies. Additionally, a quasi-optimal strategy was analyzed in order to infer knowledge from its behavior. This investigation resulted in two main insights. First, the labeling process consists of different phases. Second, the performance of a strategy depends on the data set and its characteristics.In this work, we propose a toolkit which enables users to create novel labeling strategies. First, we present multiple visual interfaces users can employ to examine the space of existing algorithms. Then, we introduce a definition of ensembles users can build upon in order to combine existing strategies to novel strategies. Multiple methods to measure the quality of labeling strategies are provided to users, enabling them to examine the gap between their strategies and existing strategies. The different phases of the labeling process are included in the toolkit in order to allow users to always apply the most appropriate strategy in each phase. During the entire process, users are supported by automated guidance in the improvement of their strategies.We evaluate our concept from different perspectives in order to assess its quality. Overall, we observe that our approach enables users to build ensemble strategies which outperform existing strategies. The insights from this work can be applied to develop novel concepts towards ensemble building as well as to improve the generalization of strategies to other data sets.

Show publication details
Reinemuth, Heiko; Kuijper, Arjan [1. Prüfer]; Bernard, Jürgen [2. Prüfer]

Visual-Interactive Labeling of Multivariate Time Series to Support Semi-Supervised Machine Learning

2019

Darmstadt, TU, Master Thesis, 2019

The labeling of multivariate time series is an essential requirement of data-centric decisionmaking processes in many time-oriented application domains. The basic idea of labeling is to assign (semantic) meaning to specific sections or time steps of the time series and to the time series as a whole, accordingly. Hence, weather phenomena can be characterized, EEG signals can be studied, or movement patterns can be marked in sensor data. In the context of this work a visual-interactive labeling tool was developed that allows nonexpert users to assign semantic meaning to any multivariate time series in an effective and efficient way. Enabling experts as well as non-experts to label multivariate time series in a visual-interactive way has never been proposed in the information visualization and visual analytics research communities before. This thesis combines active learning methods, a visual analytics approach, and novel visual-interactive interfaces to achieve an intuitive data exploration and labeling process for users. Visual guidance based on data analysis and model-based predictions empowers users to select and label meaningful instances from the time series. This user-side selection and labeling task can be taken over by an automated model or data-based process. Visual representations of labeling quality and novel interfaces allow for additional user-side refinement.

Show publication details
Bernard, Jürgen; Hutter, Marco; Reinemuth, Heiko; Pfeifer, Hendrik; Bors, Christian; Kohlhammer, Jörn

Visual-Interactive Preprocessing of Multivariate Time Series Data

2019

Computer Graphics Forum

Eurographics Conference on Visualization (EuroVis) <21, 2019, Porto, Portugal>

Pre-processing is a prerequisite to conduct effective and efficient downstream data analysis. Pre-processing pipelines often require multiple routines to address data quality challenges and to bring the data into a usable form. For both the construction and the refinement of pre-processing pipelines, human-in-the-loop approaches are highly beneficial. This particularly applies to multivariate time series, a complex data type with multiple values developing over time. Due to the high specificity of this domain, it has not been subject to in-depth research in visual analytics. We present a visual-interactive approach for preprocessing multivariate time series data with the following aspects. Our approach supports analysts to carry out six core analysis tasks related to pre-processing of multivariate time series. To support these tasks, we identify requirements to baseline toolkits that may help practitioners in their choice. We characterize the space of visualization designs for uncertainty-aware pre-processing and justify our decisions. Two usage scenarios demonstrate applicability of our approach, design choices, and uncertainty visualizations for the six analysis tasks. This work is one step towards strengthening the visual analytics support for data pre-processing in general and for uncertainty-aware pre-processing of multivariate time series in particular.

Show publication details

Visualizing Time Series Consistency for Feature Selection

2019

Journal of WSCG

International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision (WSCG) <27, 2019, Plzen, Czech Republic>

Feature selection is an effective technique to reduce dimensionality, for example when the condition of a system is to be understood from multivariate observations. The selection of variables often involves a priori assumptions about underlying phenomena. To avoid the associated uncertainty, we aim at a selection criterion that only considers the observations. For nominal data, consistency criteria meet this requirement: a variable subset is consistent, if no observations with equal values on the subset have different output values. Such a model-agnostic criterion is also desirable for forecasting. However, consistency has not yet been applied to multivariate time series. In this work, we propose a visual consistency-based technique for analyzing a time series subset’s discriminating ability w.r.t. characteristics of an output variable. An overview visualization conveys the consistency of output progressions associated with comparable observations. Interaction concepts and detail visualizations provide a steering mechanism towards inconsistencies. We demonstrate the technique’s applicability based on two real-world scenarios. The results indicate that the technique is open to any forecasting task that involves multivariate time series, because analysts could assess the combined discriminating ability without any knowledge about underlying phenomena.