Liste der Fachpublikationen

Ergebnisse 1 - 130 von 130
Show publication details
Ahmad, Salmah; Kuijper, Arjan [1. Gutachten]; Schufrin, Marija [2. Gutachten]

A Design Study on the Joy of Use in Information Visualization for Cybersecurity Analysis for Home Users

2019

Darmstadt, TU, Master Thesis, 2019

In today's society, most people are connected to the internet at least once a day, while the overall awareness for cyber security is still low. Especially average users without much expertise in IT can find it hard to connect to the domain of cyber security. This thesis takes the approach of Joy of Use to increase engagement and enjoyment for users. For this, several concepts relating to Joy of Use are described and aspects of Joy of Use are compiled. On the basis of an existing information visualization interface for the exploration of network traffic for home users, this thesis examines which strategy can best increase the Joy of Use. To work on closing the gap between home user and the domain of cyber security, three concepts are developed on top of the existing interface. These concepts utilize Joy of Use methods to increase enjoyment of a user. A preliminary online study is conducted to characterize the target user. This helps in developing the concepts while keeping the target user's needs in mind. Each of these concepts focuses on a strategy to increase Joy of Use in relation to finding unsecure connections in one's own network traffic. They are then used to develop prototypes to evaluate in a design study. The design study is conducted as an on-site user study where participants have to evaluate their emotional state and the perceived Joy of Use after using each prototype. The results are statistically evaluated. This thesis concludes that utilizing Joy of Use techniques in presenting abstruse topics can help home users to gain awareness and initiative in learning about cyber security.

Show publication details
Hegde, Chaitra; Kuijper, Arjan [1. Gutachter]; Dennstädt, Marco [2. Gutachter]

A Distributed Task Scheduler for Cuttlefish: Web to Optimize the Cost and Runtime

2019

Darmstadt, TU, Master Thesis, 2019

The ability to mass produce customized products by additively layering materials has placed 3D printing in the spotlight of the manufacturing industry. Cuttlefish is a 3D printer driver which generates printable files from a 3D mesh. When executed at scale, the driver consumes a considerable amount of computing resources. This highlights the need for a distributed system that is capable of efficiently scaling up or down depending on the type of input while operating under cost and time constraints. Through this master thesis, an intelligent task scheduler which runs print jobs on suitable computers and optimizes cost and runtime based on the user’s preference is implemented. Several Machine Learning algorithms are evaluated to build the system classification and running time prediction models, and the best performing model is deployed as a service. The realized architecture highlights methods to develop an Intelligent task scheduler. They also form a baseline for Cuttlefish::Web to be used on Cloud Infrastructure.

Show publication details
Ma, Jingting; Wang, Anqi; Lin, Feng; Wesarg, Stefan; Erdt, Marius

A novel robust kernel principal component analysis for nonlinear statistical shape modeling from erroneous data

2019

Computerized Medical Imaging and Graphics

Statistical Shape Models (SSMs) have achieved considerable success in medical image segmentation. A high quality SSM is able to approximate the main plausible variances of a given anatomical structure to guide segmentation. However, it is technically challenging to derive such a quality model because: (1) the distribution of shape variance is often nonlinear or multi-modal which cannot be modeled by standard approaches assuming Gaussian distribution; (2) as the quality of annotations in training data usually varies, heavy corruption will degrade the quality of the model as a whole. In this work, these challenges are addressed by introducing a generic SSM that is able to model nonlinear distribution and is robust to outliers in training data. Without losing generality and assuming a sparsity in nonlinear distribution, a novel Robust Kernel Principal Component Analysis (RKPCA) for statistical shape modeling is proposed with the aim of constructing a low-rank nonlinear subspace where outliers are discarded. The proposed approach is validated on two different datasets: a set of 30 public CT kidney pairs and a set of 49 MRI ankle bones volumes. Experimental results demonstrate a significantly better performance on outlier recovery and a higher quality of the proposed model as well as lower segmentation errors compared to the state-of-the-art techniques.

Show publication details
Kraft, Dimitri; Knaack, Franziska; Bader, Rainer; Portwich, Rene; Eichstaedt, Peter; Bieber, Gerald

A Survey on Vibration and Sound Analysis for Disease Detection of Knee and Hip Joints

2019

iWOAR 2019

International Workshop on Sensor-based Activity Recognition (iWOAR) <6, 2019, Rostock, Germany>

ACM International Conference Proceedings Series

The knee is the largest joint in the human body. Unfortunately, some hips or knee joints suffer on inflammation, misalignment, degeneration, trauma as well as diseases like arthritis or osteoporosis. Modern medicine can measure the joint condition or, if the joint is worn out, even exchange the joint with an implant. Endoprosthetic implants are artificial devices that replaces a weak body part such as osteoarthritic knee or hip joints. The lifespan of joint endoprostheses are also limited and depend on several factors, and it varies for each patient. In most cases total knee or hip endoprostheses need to be replaced after approximately 15 to 20 years, but some implants need an exchange after a few years due to several causes. Current methods to examine the condition of joint endoprostheses and natural joints are X-ray, Computed tomography (CT) and Magnetic Resonance Imaging (MRI). In rare cases implant integrated sensors were used. The usage of these methods and the analysis of the assessed data require medical and data experts. However, a vague estimation of the joint condition can also be performed by external vibration and sound analysis of the endoprosthesis and natural joint during movements. This paper describes several approaches of external vibration and sound analysis as a survey

Show publication details
Zhou, Tong-xue; Zeng, Dong-dong; Kuijper, Arjan

A Template Consensus Method for Visual Tracking

2019

Optoelectronics Letters

Visual tracking is a challenging problem in computer vision. Recently, correlation filter-based trackers have shown to provide excellent tracking performance. Inspired by a sample consensus approach proposed for foreground detection, which classifies a given pixel as foreground or background based on its similarity to recently observed samples, we present a template consensus tracker based on the kernelized correlation filter (KCF). Instead of keeping only one target appearance model in the KCF, we make a feature pool to keep several target appearance models in our method and predict the new target position by searching for the location of the maximal value of the response maps. Both quantitative and qualitative evaluations are performed on the CVPR2013 tracking benchmark dataset. The results show that our proposed method improves the original KCF tracker by 8.17% in the success plot and 8.11% in the precision plot.

Show publication details
Pfeifer, Hendrik; Kuijper, Arjan [1. Prüfer]; Cibulski, Lena [2. Prüfer]

A Visual Analytics Approach to Sensor Analysis for End-of-Line Testing

2019

Darmstadt, TU, Master Thesis, 2019

End-of-Line testing is the final step of modern production lines that assures the quality of produced units before they are shipped to customers. Automatically deciding between functional and defective units as well as classifying the type of defect are main objectives. In this thesis, a dataset consisting of three phase internal rotor engine simulations is used to outline opportunities and challenges of Visual Analytics for End-of-Line testing. At first the simulation data is visually analyzed to understand the influence of the simulation input parameters. Afterwards features are extracted from the signals using discrete Fourier transform (DFT) and discrete Wavelet transform (DWT) to represent the different simulations. Principal Component Analysis (PCA) is applied to further reduce the dimensionality of the data to finally apply K-Means to cluster the datasets and also perform a classification using a support vector machine (SVM). It is discussed which methods are beneficial for the End-of-Line testing domain and how they can be integrated to improve the overall testing process.

Show publication details
Han, Xiyu; Lv, Tao; Song, Xiangyu; Nie, Ting; Liang, Huaidan; He, Bin; Kuijper, Arjan

An Adaptive Two-scale Image Fusion of Visible and Infrared Images

2019

IEEE Access

In this paper, we proposed an adaptive two-scale image fusion method using latent low-rank representation (LatLRR). Firstly, both IR and VI images are decomposed into a two-scale representation using LatLRR to generate low-rank parts (the global structure) and saliency parts (the local structure). The algorithm denoises at the same time. Then, the guided filter is used in the saliency parts to make full use of the spatial consistency, which reduces artifacts effectively. With respect to the fusion rule of the low-rank parts, we construct adaptive weights by adopting fusion global-local-topology particle swarm optimization (FGLT-PSO) to obtain more useful information from the source images. Finally, the resulting image is reconstructed by adding the fused low-rank part and the fused saliency part. Experimental results validate that the proposed method outperforms several representative image fusion algorithms on publicly available datasets for infrared and visible image fusion in terms of subjective visual effect and objective assessment.

Show publication details
Janßen, René; Zabel, Jakob; Lukas, Uwe von; Labrenz, Matthias

An artificial neural network and Random Forest identify glyphosate-impacted brackish communities based on 16S rRNA amplicon MiSeq read counts

2019

Marine pollution bulletin

Machine learning algorithms can be trained on complex data sets to detect, predict, or model specific aspects. Aim of this study was to train an artificial neural network in comparison to a Random Forest model to detect induced changes in microbial communities, in order to support environmental monitoring efforts of contamination events. Models were trained on taxon count tables obtained via next-generation amplicon sequencing of water column samples originating from a lab microcosm incubation experiment conducted over 140 days to determine the effects of glyphosate on succession within brackish-water microbial communities. Glyphosate-treated assemblages were classified correctly; a subsetting approach identified the taxa primarily responsible for this, permitting the reduction of input features. This study demonstrates the potential of artificial neural networks to predict indicator species for glyphosate contamination. The results could empower the development of environmental monitoring strategies with applications limited to neither glyphosate nor amplicon sequence data.

Show publication details

An Experimental Overview on Electric Field Sensing

2019

Journal of Ambient Intelligence and Humanized Computing

Electric fields exist everywhere. They are influenced by living beings, conductive materials, and other charged entities. Electric field sensing is a passive capacitive measurement technique that detects changes in electric fields and has a very low power consumption. We explore potential applications of this technology and compare it to other measurement approaches, such as active capacitive sensing. Five prototypes have been created that give an overview of the potential use cases and how they compare to other technologies. Our results reveal that electric field sensing can be used for indoor applications as well as outdoor applications. Even a mobile usage is possible due to the low energy consumption of this technology.

Show publication details
Montefiori, Erica; Modenese, Luca; Di Marco, Roberto; Magni-Manzoni, Silvia; Malattia, Clara; Petrarca, Maurizio; Ronchetti, Anna; Tanturri De Horatio, Laura; Dijkhuizen, Pieter van; Wang, Anqi; Wesarg, Stefan; Viceconti, Marco; Mazzà, Claudia

An Image-based Kinematic Model of the Tibiotalar and Subtalar Joints and its Application to Gait Analysis in Children with Juvenile Idiopathic Arthritis

2019

Journal of Biomechanics

In vivo estimates of tibiotalar and the subtalar joint kinematics can unveil unique information about gait biomechanics, especially in the presence of musculoskeletal disorders affecting the foot and ankle complex. Previous literature investigated the ankle kinematics on ex vivo data sets, but little has been reported for natural walking, and even less for pathological and juvenile populations. This paper proposes an MRI-based morphological fitting methodology for the personalised definition of the tibiotalar and the subtalar joint axes during gait, and investigated its application to characterise the ankle kinematics in twenty patients affected by Juvenile Idiopathic Arthritis (JIA). The estimated joint axes were in line with in vivo and ex vivo literature data and joint kinematics variation subsequent to inter-operator variability was in the order of 1°. The model allowed to investigate, for the first time in patients with JIA, the functional response to joint impairment. The joint kinematics highlighted changes over time that were consistent with changes in the patient’s clinical pattern and notably varied from patient to patient. The heterogeneous and patient-specific nature of the effects of JIA was confirmed by the absence of a correlation between a semi-quantitative MRI-based impairment score and a variety of investigated joint kinematics indexes. In conclusion, this study showed the feasibility of using MRI and morphological fitting to identify the tibiotalar and subtalar joint axes in a non-invasive patient-specific manner. The proposed methodology represents an innovative and reliable approach to the analysis of the ankle joint kinematics in pathological juvenile populations.

Show publication details
Lian, Runze; Kuijper, Arjan [Gutachter]; Fu, Biying [Betreuerin]

Anomaly Detection and probable path prediction for Single and Multiperson-Application in Smart Homes

2019

Darmstadt, TU, Master Thesis, 2019

The most popular outdoor positioning system, global positioning system (GPS), does not perform well in indoor environment. Because this system primarily depends on the signal propagation in the air and complex architecture of buildings will interfere with signal propagation, i.e., its indoor positioning performance will be limited by the line-of-sight nature. While the drawbacks of GPS, other indoor positioning techniques (such as Wi-Fi based, RFID based) can provide Location-based-service (LBS) for various applications, which make our life comfortable and smart. As one kind of these sensing and positioning techniques, the passive Electric Field Sensing has numerous advantages compared to the others, e.g., lower power consumption and no personal information and specific positioning tokens required. So it is applied in our Smart Floor system to position and track movement for users, which mainly aims at the elderly care. On the other hand, a passive EFS-based positioning system might be susceptible to disturbance, due to the aliasing effect and noises from environment. To address this problem, I studied and investigated the Anomaly Detection issue in the Machine Learning domain, which aims at discovering proper ML algorithms to improve the positioning and movement prediction performance of our Smart Floor system. In this thesis, I proposed a novel ML algorithm for this goal, namely the Dictionary-based Anomaly Detection Algorithm. Compared with other existing algorithms, this dictionary algorithm exploits not only the normal data but also coupling outliers to obtain our desired results, i.e. indoor positions of users. Furthermore, combining with a customized positioning scheme relying on anchor points, the Dictionary-based indoor Positioning and Movement Prediction approach preformed well in our living laboratory. Moreover as discussion and expansion, the Dictionary-based Anomaly Detection Algorithm is especially practicable in application scenarios where a large amount of outliers and normal data are always at the same time observed.

Show publication details
Wagner, Tim Geronimo; Kuijper, Arjan [1. Prüfer]; Siegmund, Dirk [2. Prüfer]

Applied Food Recognition for Vision-Based Self-Checkout Systems

2019

Darmstadt, TU, Bachelor Thesis, 2019

Food recognition has been around almost as long as object detection itself. However, it is still an immensely complicated task due to the nature of food. The same food can come in different shapes, colors and arrangements. In contrast, different food can look almost identical. Therefore, it is crucial to find efficient systems to evaluate images of food. The main goal of this thesis is to provide a way for canteens to use self-checkout systems. It should be able to identify and differentiate between food items on the basis of pictures of the food. Then, a total price should be calculated and a method of payment is provided. This report solved these problems by building on a master’s thesis [3]. We collected a database that was used to train two neural networks. A CNN based on the Inception architecture achieved equal error rate losses of 9% and is responsible for identifying the main dishes on the user’s tray. Another Faster R-CNN was set up to identify side components with a precision of 99.98%. A prototype was set up that is able to classify food on canteen trays from two images. It was equipped with an efficient camera setup and interfaces to a back-end server that handles classifications. The system is ready to be used in canteens. The following report will describe how the system works and which steps have been taken in order to achieve the given accuracies.

Show publication details
Zimmermann, Verena; Gerber, Paul; Marky, Karola; Böck, Leon; Kirchbuchner, Florian

Assessing Users’ Privacy and Security Concerns of Smart Home Technologies

2019

i-com

Smart Home technologies have the potential to increase the quality of life, home security and facilitate elderly care. Therefore, they require access to a plethora of data about the users’ homes and private lives. Resulting security and privacy concerns form a relevant barrier to adopting this promising technology. Aiming to support end users’ informed decision-making through addressing the concerns we first conducted semi-structured interviews with 42 potential and little-experienced Smart Home users. Their diverse concerns were clustered into four themes that center around attacks on Smart Home data and devices, the perceived loss of control, the tradeoff between functionality and security, and user-centric concerns as compared to concerns on a societal level. Second, we discuss measures to address the four themes from an interdisciplinary perspective. The paper concludes with recommendations for addressing user concerns and for supporting developers in designing user-centered Smart Home technologies.

Show publication details

Automatic Detection of the Nasal Cavities and Paranasal Sinuses Using Deep Neural Networks

2019

2019 IEEE International Symposium on Biomedical Imaging

IEEE International Symposium on Biomedical Imaging (ISBI) <16, 2019, Venice, Italy>

The nasal cavity and paranasal sinuses present large interpatient variabilities. Additional circumstances like for example, concha bullosa or nasal septum deviations complicate their segmentation. As in other areas of the body a previous multistructure detection could facilitate the segmentation task. In this paper an approach is proposed to individually detect all sinuses and the nasal cavity. For a better delimitation of their borders the use of an irregular polyhedron is proposed. For an accurate prediction the Darknet-19 deep neural network is used which combined with the You Only Look Once method has shown very promising results in other fields of computer vision. 57 CT scans were available of which 85% were used for training and the remaining 15% for validation.

  • 978-1-5386-3640-4
Show publication details
Mannl, Felix; Sakas, Georgios [Prüfer]; Oyarzun Laura, Cristina [Betreuerin]

Automatische Erkennung von Pneumonie auf Röntgen-Thorax mit Deep Learning

2019

Darmstadt, TU, Bachelor Thesis, 2019

Pneumonie ist eine weit verbreitete und einer der gefahrlichsten Krankheiten weltweit [35]. Das verbreitetste Verfahren fur die Diagnose von Pneumonie, ist das Erstellen von Thorax-Rontgen Aufnahmen [36]. Die Analyse dieser Aufnahmen ist eine schwere Aufgabe, fur die Experten benotigt werden. Um das Analyseverfahren zu vereinfachen und die Experten zu unterstutzen, soll computergestutzte Detektierung aushelfen. Im Rahmen dieser Bachelorarbeit ’Automatische Erkennung von Pneumonie auf Rontgen-Thorax mit Deep Learning’ wird eine Methode zur automatischen Detektierung von Pneumonie anhand von Thorax- Rontgen Aufnahmen vorgestellt. Die ausgewahlte Architektur YOLO [39] basiert auf Deep Learning. Sie wurde mit Hilfe der von [5] bereitgestellten Daten trainiert und getestet.

Show publication details
Sachs, Moritz Karl; Unbescheiden, Matthias [Betreuer]; Buxmann, Peter [Betreuer]

Automatisierte Geschäftsmodellanalysen mit Deep Neural Networks

2019

Darmstadt, TU, Master Thesis, 2019

Ein Hauptkriterium für das Investment eines Venture Capital Fonds in ein Start-up ist dessen Geschäftsmodell. Dieses ist im Businessplan enthalten. Das Screening, sowie die Analyse der eingereichten Businesspläne, erfolgt bei den meisten Venture Capital Fonds überwiegend durch Menschen. Mit der vorliegenden Arbeit wird untersucht, inwieweit die Analyse der in den Businessplänen enthaltenen Geschäftsmodelle mit Hilfe von Deep Neural Networks automatisiert werden kann. Ziel war die Entwicklung eines Prototypen, der die in den Businessplänen enthaltenen Geschäftsmodelle automatisch extrahiert und in das Metamodell Startup Navigator überführt. Dem Knowledge Discovery in Databases Prozess folgend wurden hierfür die Businesspläne eines Venture Capital Fonds aufbereitet und damit ein tiefes Convolutional Neural Network, der Multilabel k-Nearest Neighbour Algorithmus, sowie eine Support Vector Machine mit Naive Bayes Features trainiert. Die Ergebnisse des entwickelten Prototypen zeigen, dass die in den Businessplänen enthaltenen Geschäftsmodelle automatisch extrahiert und in das Metamodell Startup Navigator überführt werden können. Es erscheint plausibel, dass mit mehr Trainingsdaten und einer intensiveren Hyperparameteroptimierung die Korrektklassifizierungsrate verbessert werden kann, sodass der Prototyp zum Aufbau eines Geschäftsmodellkorpus genutzt werden könnte.

Show publication details
Zeng, Dong-dong; Chen, Xiang; Zhu, Ming; Goesele, Michael; Kuijper, Arjan

Background Subtraction With Real-Time Semantic Segmentation

2019

IEEE Access

Accurate and fast foreground (FG) object extraction is very important for object tracking and recognition in video surveillance. Although many background subtraction (BGS) methods have been proposed in the recent past, it is still regarded as a tough problem due to the variety of challenging situations that occur in real-world scenarios. In this paper, we explore this problem from a new perspective and propose a novel BGS framework with the real-time semantic segmentation. Our proposed framework consists of two components, a traditional BGS segmenter B and a real-time semantic segmenter S. The BGS segmenter B aims to construct background (BG) models and segments FG objects. The real-time semantic segmenter S is used to re_ne the FG segmentation outputs as feedbacks for improving the model updating accuracy. B and S work in parallel on two threads. For each input frame It , the BGS segmenter B computes a preliminary FG/BG mask Bt . At the same time, the real-time semantic segmenter S extracts the object-level semantics St . Then, some speci_c rules are applied on Bt and St to generate the _nal detection Dt . Finally, the re_ned FG/BG mask Dt is fed back to update the BG model. The comprehensive experiments evaluated on the CDnet 2014 dataset demonstrate that our proposed method achieves the state-of-the-art performance among all unsupervised BGS methods while operating at the real-time and even performs better than some deep learning-based supervised algorithms. In addition, our proposed framework is very _exible and has the potential for generalization.

Show publication details
Schaufelberger, Max; Stork, André [1. Gutachten]; Grasser, Tim [2. Gutachten]

Bestimmung adjungierter Sensitivitäten als Grundlage einer prototypischen Topologieoptimierung

2019

Darmstadt, TU, Bachelor Thesis, 2019

Diese Arbeit behandelt die Bestimmung von Sensitivitäten zur Lösung eines Optimierungsproblems. Ziel ist es dabei, mithilfe von Topologieoptimierung, eine optimale Struktur zu finden, die unter mechanischer Belastung eine Zielfunktion minimiert. Die Strukturbildung verläuft über einen elementbasierten Ansatz der Dichte. Der Optimierung unterliegt eine Nebenbedingung an die maximale Masse, die eine optimale Lösung unterschreiten muss. Für die Bestimmung des mechanischen Verhaltens wird eine Finite-Elemente-Analyse durchgeführt. Die Sensitivitäten sind dabei die Ableitungen der Zielfunktion nach den Designparametern. Die Sensitivitäten werden über einen adjungierten Ansatz hergeleitet und über Automatisches Differenzieren bestimmt. Bei der Bestimmung mit automatischem Differenzieren wird eine Graphenfärbung verwendet, um die Durchläufe des Vorwärtsmodus zu reduzieren. Die Errechneten Sensitivitäten werden auf Plausibilität untersucht und für die Lösung verschiedener Topologieoptimierungen eingesetzt.

Show publication details
Gutsch, Stefan; Kuijper, Arjan [1. Review]; Lücke-Tieke, Hendrik [2. Review]

Bridging the Gap between Analysis and Presentation

2019

Darmstadt, TU, Master Thesis, 2019

Creating reports or presentations to communicate insights from a data analysis is common in various professional areas. While both data analysis as well as reporting are already offered by sophisticated tools, those processes are mostly independent and miss further interconnection. In this work, possibilities to seamlessly integrate the analysis and presentation as well as aid the user with the linearisation of results, are evaluated. This integration was realised in form of interim steps or views, and compared regarding the Usability and User Experience. In contrast to the hypothesis, the results indicate that the implemented measures do not significantly increase the Usability or User Experience. Nevertheless, observed trends in the experiment, alternative supportive approaches and user feedbacks favouring the implemented approaches, could form the basis of future research.

Show publication details

calibDB: Enabling Web Based Computer Vision Through On-the-fly Camera Calibration

2019

Proceedings Web3D 2019

International Conference on 3D Web Technology (WEB3D) <24, 2019, Los Angeles, CA, USA>

For many computer vision applications, the availability of camera calibration data is crucial as overall quality heavily depends on it. While calibration data is available on some devices through Augmented Reality (AR) frameworks like ARCore and ARKit, for most cameras this information is not available. Therefore, we propose a web based calibration service that not only aggregates calibration data, but also allows calibrating new cameras on-the-fly. We build upon a novel camera calibration framework that enables even novice users to perform a precise camera calibration in about 2 minutes. This allows general deployment of computer vision algorithms on the web, which was previously not possible due to lack of calibration data.

Show publication details

Change Detection and Blob Tracking of Fish in Underwater Scenarios

2019

Computer Vision, Imaging and Computer Graphics – Theory and Applications

International Joint Conference on Computer Vision and Computer Graphics Theory and Applications (VISIGRAPP) <12, 2017, Porto, Portugal>

In this paper, the difficult task of detecting fishes in underwater scenarios is analyzed with a special focus on crowded scenes where the differentiation between separate fishes is even more challenging. An extension for the Gaussian Switch Model is developed for the detection which applies an intelligent update scheme to create more accurate background models even for difficult scenes. To deal with very crowded areas in the scene we use the Flux Tensor to create a first coarse segmentation and only update areas that are with high certainty background. The spatial coherency is increased by the N2Cut, which is a Ncut adaption to change detection. More relevant information are gathered with a novel blob tracker that uses a specially developed energy function and handling of errors during the change detection. This method keeps the generality of the whole approach so that it can be used for any moving object. The proposed algorithm enabled us to get very accurate underwater segmentations as well as precise results in tracking scenarios.

Show publication details
Abuladze, David; Kuijper, Arjan [1. Gutachten]; Ben Hmida, Helmi [2. Gutachten]

ClickDigital IDE Enrichment with Internet of Things Rule Widgets

2019

Darmstadt, TU, Bachelor Thesis, 2019

In the recent years, the Internet of Things (IoT) has become an important subject for different services and for the society in general, where different companies use IoT Systems in different ways to improve everyday life quality. Examples of use can be smart adjustment of heating systems, logistics, smart homes, device management for IoT System gateway, device management for connected agricultures, connected cars, and many others. Simplification of the communication process between the end user and devices of the IoT Systems has been a point of concern in many computer science studies. One of them is the master thesis elaborated by Zafar [1]. In his research titled “Enhancing User Experience in the Internet of Things systems regarding Smart Rule Management”, Zafar [1] based his work on several friendly Rule Management interfaces and their arguments pro and contra. After having analyzed and compared relevant studies that have been published at that time, Zafar [1] has suggested a number of user-friendly Rules based widgets within ClickDigital toolkit. ClickDigital toolkit is a dashboard that combines and integrates different IoT System Service interfaces. These widgets aim to manage rules, created by the end users, without any technical background. In my Bachelor Present I will extend the ClickDigital Integrated Development Environment (IDE), which is a web toolkit developed by the Fraunhofer-Institut für Graphische Datenverarbeitung. The present Bachelor Thesis focuses on the ClickDigital IDE enrichment with an Internet of Things Rule widgets. Enrichment of the ClickDigital IDE consists of Rule widgets implementation and validation. The main target includes the extension and the enrichment of the existing ClickDigital widgets repository with a set of Rule Management widgets. The analysis of the previous relevant study by Zafar [1] and other sources has been performed, and several improvements have been implemented. It allows resulting in a better and more efficient communication process between end users and devices within ClickDigital toolkit.

Show publication details
Zeng, Dong-dong; Zhu, Ming; Kuijper, Arjan

Combining Background Subtraction Algorithms with Convolutional Neural Network

2019

Journal of Electronic Imaging

Accurate and fast extraction of foreground objects is a key prerequisite for a wide range of computer vision applications, such as object tracking and recognition. Thus, many background subtraction (BGS) methods for foreground object detection have been proposed in recent decades. However, this is still regarded as a tough problem due to a variety of challenges, such as illumination variations, camera jitter, dynamic backgrounds, and shadows. Currently, there is no single method that can handle all the challenges in a robust way. We try to solve this problem from a perspective of combining different state-of-the-art BGS algorithms to create a more robust and more advanced foreground detection algorithm. More specifically, an encoder–decoder fully convolutional neural network architecture is adapted and trained to automatically learn how to leverage the characteristics of different algorithms to fuse the results produced by different BGS algorithms and produce a more precise result. Comprehensive experiments evaluated on the CDnet 2014 dataset demonstrate that the proposed method outperforms all the considered single BGS algorithms. We show that our solution is more efficient than other BGS combination strategies.

Show publication details
Funk, Felix Garcia; Kuijper, Arjan [1. Review]

Connected Fermat Spirals for Layered Fabrication with FDM-Printers

2019

Darmstadt, TU, Bachelor Thesis, 2019

This work discusses the implementation process of the algorithm presented in ConnectedFermat Sprials for Layered Fabrication[8]. Layered fabrication is an addaptive manufactoring process that describes a set of techniques to construct 3D objects. In comparison tocasting, molding or milling, these techniques are used to create unique individual partslayer by layer using a 3D printer. Sometimes, these parts would not even be generatableby conventional approaches. The algorithm[8] is designed for fused deposition modeling(FDM) in particular but without limitation. FDM is a technique that uses heated filamentand a movable nozzle to create lines of viscous, fast hardening printing material. Theselines form a 2D layer representing the shape of the object at the respective cross-sectionand can also be refered as the tool path. By creating the next layer on top of the currentone, the printer eventually generates the final 3D Object. The composition of the toolpath directly influences the quality of the resulting product as well as the overall printingtime[8]. There are different approaches to create such a tool path and using connectedfermat spirals is one of them. To explain the motivation behind the Fermat Spiral approach and discuss the method as well as some disagreements on the used definitionsand possible workarrounds, the next chapter lays the groundwork and defines some general concepts. Afterwards, the algorithm and the implementation process is describedand finally the results are discussed in the conclusion. The overall implementation process took longer than expected and did not come to a satisfying completion. The reasonsfor that are described and evaluated in the following chapters. The original goal to validate and compare printing results from the underlying algorithm was not reached. Thenew aim of this work is to describe the obstacles found during the implementation process and present solutions as well as an outlook for more possible extensions. It shouldlay the basic groundwork for future investigations and implementations regarding thistopic.

Show publication details
Rehman, Ahmed Abdul; Kuijper, Arjan [1. Prüfer]; Burkhardt, Dirk [2. Prüfer]

Contrasted Data from Science and Web for Advanced Visual Trend Analytics

2019

Darmstadt, TU, Master Thesis, 2019

With more publicly accessible digital libraries accessible, a plethora of digital research data is now available for gaining insights into actual and upcoming technology trends. These trends are essential to researchers, business analysts, and decision-makers for making strategic decisions and setting strategic goals. Appropriate processing and graphical analysis methods are required in order to extract meaningful information from the data. In particular, the combination of data mining approaches together with visual analytics leads to real beneficial applications to support decision making in e.g. innovation or technology management. The data from digital libraries is only limited to research and overlooks the market aspects e.g if the trend is not important for key business players, it is irrelevant for the market. This importance of market aspects creates a demand for validation approaches based on market data. Most of the current market data can be found publically on websites and social networks, e.g. as news from enterprises or on tech review sites or on tech blogs. Therefore, it makes sense to consider this public and social media data as contrasting data to the research digital library data that can be used to validate technology trends. The goal of this thesis is to enable trend analysis on public and social web data and compare it with retrieved trends based on research library data to enable validation of trends. To achieve this goal a model is proposed that acquires public/social web and digital library data based on user-defined scope called a "campaign", which is then visually transformed from raw data into interactive visualizations passing through different stages of data management, enrichment, transformation, and visual mapping. These interactive visualizations can either be used in insight analysis to gain trend insights for an individual data source or they can be used in comparative analysis with the goal of validating trends from two contrasting data sources.

Show publication details
Mallat, Khawla; Damer, Naser; Boutros, Fadi; Kuijper, Arjan; Dugelay, Jean-Luc

Cross-spectrum thermal to visible face recognition based on cascaded image synthesis

2019

The 12th IAPR International Conference On Biometrics

IAPR International Conference on Biometrics (ICB) <12, 2019, Crete, Greece>

Face synthesis from thermal to visible spectrum is fundamental to perform cross-spectrum face recognition as it simplifies its integration in existing commercial face recognition systems and enables manual face verification. In this paper, a new solution based on cascaded refinement networks is proposed. This method generates visible-like colored images of high visual quality without requiring large amounts of training data. By employing a contextual loss function during training, the proposed network is inherently scale and rotation invariant. We discuss the visual perception of the generated visible-like faces in comparison with recent works. We also provide an objective evaluation in terms of cross-spectrum face recognition, where the generated faces were compared against a gallery in visible spectrum using two state-of-the-art deep learning based face recognition systems. When compared to the recently published TV-GAN solution, the performance of the face recognition systems, OpenFace and LightCNN, was improved by a 42.48% (i.e. from 10.76% to 15.37%) and a 71.43% (i.e. from 33.606% to 57.612%), respectively.

Show publication details
Chaudry, Zaki Ullah; Kuijper, Arjan [Betreuer]; Rus, Silvia [Prüfer]

Designing a General Purpose Smart Textile for a Healthy Lifestyle

2019

Darmstadt, TU, Master Thesis, 2019

We live in a world where wearable technology is fast becoming a part of our daily lives. These wearables vary from a wide range of gadgets such as watches, goggles, shoes, textile etc. One thing all of these wearables have in common is health and fitness monitoring in order to make human life better, which is being achieved by the help of monitoring the activities of the users with the help of various sensors incorporated within the wearable technology. In this thesis the main focus is on creating a smart textile which typically could be placed on different forms of furniture being used in daily life such as bed, chair, couch or table, which than monitors the activities and ergonomics of the person using the furniture. Based on the embedded sensor matrix and advanced algorithms the textile would automatically detect the type of furniture it is placed on. The textile will also constitute of capacitive sensors which, separately or in combination with the accelerometer sensors, detect the user activity (by detecting the breathing rhythm, movement or user interaction). When used in combination with a bed this textile with the help of the sensors will be able to detect the movement, breathing rhythm and user interaction for the user. The same prototype can be used by a person sitting on a chair or couch to detect the breathing, movement and interaction. In case of a table the prototype will be able to measure movement and interaction only. Some smart furniture is already available in the market, but the prices are often to high for a average user to be able to afford them and also the use is limited to just one product. With this textile a user can convert existing dumb furniture into smart furniture, hence providing a wide range of usage with the help of just one textile. In result creating a place-able smart textile technology for a better and healthy lifestyle at an affordable price. This textile minimizes the distance between humans and smart living and fills in the gaps which are beyond the capacity of current wearable technology, while staying cost effective and flexible. Therefore being a great addition to the ecosystem of smart living.

Show publication details

2019

Proceedings of the 12th ACM International Conference on PErvasive Technologies Related to Assistive Environments

ACM International Conference on PErvasive Technologies Related to Assistive Environments (PETRA) <12, 2019, Rhodes, Greece>

Smart textiles and garments promise intriguing new possibilities for the wearer. Integrated interaction can create new experiences and sensors can detect relevant information about the wearer. However, this poses an additional challenge for the designer of smart garments, about how to integrate these technologies. In this work, we want to investigate how human intuition and technical knowledge feed into the design of smart garments. Using a jacket that tracks its whereabouts as a use case, we have collected a dataset from 18 test subjects with varying technical knowledge, on what sensor patterns they would create on the garment. Using a specifically created simulation framework, we have evaluated the performance of the created sensor patterns. We observed that many participants intuitively create well-working patterns, while technical knowledge does not play a significant role in the resulting performance.

  • 978-1-4503-6232-0
Show publication details
Mettel, Matthias Ruben; Alekseew, Michael; Stocklöw, Carsten; Braun, Andreas

Designing and Evaluating Safety Services Using Depth Cameras

2019

Journal of Ambient Intelligence and Humanized Computing

Not receiving help in the case of an emergency is one of the most common fears of older adults that live independently at home. Falls are a particularly frequent occurrence and often the cause of serious injuries. In the last years, various ICT solutions for supporting older adults at home have been developed. Based on sensors and services in a smart environment they provide a wide range of services. In this work we have designed and evaluated safety-related services, based on a single Microsoft Kinect that is installed in a user’s home. We created two services to investigate the benefits and limitations of these solutions. The first is a fall detection service that registers falls in real-time, using a novel combination of static and dynamic skeleton tracking. The second is a fall prevention service that detects potentially dangerous objects in the walking path, based on scene analysis in a depth image. We conducted technical and user evaluations for both services, in order to get feedback on the feasibility, limitations, and potential future improvements.

Show publication details
Helfmann, Stefan; Kuijper, Arjan [1. Gutachten]; Rus, Silvia [2. Gutachten]

Designing Smart Home Controls for Elderly

2019

Darmstadt, TU, Master Thesis, 2019

As technology advances, the idea of a "smart home" gets implemented more and more, and offers multiple ways to help people managing their homes. This offers a big opportunity to help elderly people with their daily lives. However, most feel overwhelmed by modern technology and opt out of using it at all. This thesis provides an alternative take on a Smart Home remote control. After collecting, discussing and evaluating several design principles and guidelines we propose two possible concept remotes and corresponding mock-ups, that are subjected to an evaluation by elderly people to choose the one they prefer. This preferred model will be along with proposed improvements made into a functional prototype and, again, be evaluated by the elderly. The final prototype was received positively and additional improvements are proposed.

Show publication details
Damer, Naser; Boller, Viola; Wainakh, Yaza; Boutros, Fadi; Terhörst, Philipp; Braun, Andreas; Kuijper, Arjan

Detecting Face Morphing Attacks by Analyzing the Directed Distances of Facial Landmarks Shifts

2019

Pattern Recognition

German Conference on Pattern Recognition (GCPR) <40, 2018, Stuttgart, Germany>

Lecture Notes in Computer Science (LNCS)
11269

Face morphing attacks create face images that are verifiable to multiple identities. Associating such images to identity documents lead to building faulty identity links, causing attacks on operations like border crossing. Most of previously proposed morphing attack detection approaches directly classified features extracted from the investigated image. We discuss the operational opportunity of having a live face probe to support the morphing detection decision and propose a detection approach that take advantage of that. Our proposed solution considers the facial landmarks shifting patterns between reference and probe images. This is represented by the directed distances to avoid confusion with shifts caused by other variations. We validated our approach using a publicly available database, built on 549 identities. Our proposed detection concept is tested with three landmark detectors and proved to outperform the baseline concept based on handcrafted and transferable CNN features.

  • 978-3-030-12938-5
Show publication details
Jakob, David; Kuijper, Arjan [1. Gutachten]; Wilmsdorff, Julian von [2. Gutachten]

Development of a Smart-Connection-Surface

2019

Darmstadt, TU, Bachelor Thesis, 2019

Wired and wireless connection methods both have their drawbacks. Wired methods need to be manually locked together, while wireless connections are not as energy-efficient. In this thesis a new connection method in the form of a smart-connection-surface and a matching mobile adapter was explored to improve the shortcomings of existing technologies. A geometrical model was developed and applied to find and simulate connector geometries needed for this. Physical prototypes of all necessary parts of the connection were then implemented and tested for their compliance with the simulation. Due to the mechanical preciseness of the fabricated connectors the success rate for establishing a connection was found to be only 39%. Nevertheless this work serves as a proof of concept for the proposed type of connection method.

Show publication details
Jagadishwara, Anitha Sankapura; Mukhopadhyay, Anirban [1. Prüfer]; Kuijper, Arjan [2. Prüfer]

Disentangled Representation of Breast Cancer Images

2019

Darmstadt, TU, Master Thesis, 2019

The motivation behind semi-supervised learning of disentangled representation lies in the difficulty of getting access to an expert-annotated large dataset in medical applications. Learning disentangled representations from visual data where high-level generative factors are defined by users, is of great importance for many computer vision applications. Defining the factors of variation externally allows a user greater flexibility in modifying and interpreting the learnt latent representations of data. Recent works have shown increased interest in using Variational Autoencoders to discover the interpretable representations of data in an semi-supervised way. This has also been used for a wide range of applications such as image search, natural language parsing and speech analysis. In this work, we propose a deep generative model combining Variational Autoencoder (VAE) and Wasserstein Generative adversarial network (WGAN) to learn the disentangled representation of the data. One of the application of this approach is to enhance the readability of mammograms used for breast cancer screening. The proposed approach was partially successful and there is a lot of scope for possible improvements. The accuracy of the proposed approach is limited due to underlying constraints which were out of scope of this thesis. Additionally, due to resource constraints, the quality of images has been compromised in this work. The novelty of this approach, however, lies in learning more semantically meaningful latent space representation which can be reused in various applications such as synthetic data generation, image translation, classification and segmentation tasks. This approach was successfully tested on most popular datasets such as MNIST (hand-written digits) and fashion-MNIST. However, due to the unavailability of ground truth images, a qualitative measure for breast cancer result is missing. It lacks proper evaluation at the moment and could be taken up as future works.

Show publication details
Hiemenz, Benedikt; Krämer, Michel

Dynamic Searchable Symmetric Encryption for Storing Geospatial Data in the Cloud

2019

International Journal of Information Security

We present a dynamic searchable symmetric encryption scheme allowing users to securely store geospatial data in the cloud. Geospatial data sets often contain sensitive information, for example, about urban infrastructures. Since clouds are usually provided by third parties, these data need to be protected. Our approach allows users to encrypt their data in the cloud and make them searchable at the same time. It does not require an initialization phase, which enables users to dynamically add new data and remove existing records. We design multiple protocols differing in their level of security and performance, respectively. All of them support queries containing boolean expressions, as well as geospatial queries based on bounding boxes, for example. Our findings indicate that although the search in encrypted data requires more runtime than in unencrypted data, our approach is still suitable for real-world applications.We focus on geospatial data storage, but our approach can also be applied to applications from other areas dealing with keyword-based searches in encrypted data. We conclude the paper with a discussion on the benefits and drawbacks of our approach.

Show publication details

E-Textile Capacitive Electrodes: Fabric or Thread

2019

Proceedings of the 12th ACM International Conference on PErvasive Technologies Related to Assistive Environments

ACM International Conference on PErvasive Technologies Related to Assistive Environments (PETRA) <12, 2019, Rhodes, Greece>

Back pain is one of the most common illnesses in Western civilizations. Office work and lack of motion can lead to deterioration over time. Many people already use seat cushions to improve their posture during work or leisure. In this work, we present an E-Textile cushion. This seat cushion is equipped with capacitive proximity sensors that track the proximity and motion of the sitting user and distinguish up to 7 postures. Giving a user immediate feedback on the posture can facilitate more healthy behavior. We evaluated a number of different electrode setups, materials, and classification methods, leading to a maximum accuracy of 97.1%.

  • 978-1-4503-6232-0
Show publication details

Echtzeit 3D-Erfassung von Personen und deren Blickrichtung ohne Smartglasses

2019

Go-3D 2019: Mit 3D Richtung Maritim 4.0

Go-3D <9, 2019, Rostock, Germany>

Intelligente Assistenzsysteme halten immer weiter Einzug in die aktuellen Arbeitsprozesse. Ein großes Anwendungsgebiet ist dabei die passive, visuelle Assistenz. Mit ihrer Hilfe können zusätzliche Informationen bereitgestellt oder Anleitungen für Handlungsabfolgen gegeben werden, ohne das System aktiv eingreifen zu lassen. Dabei sind insbesondere Virtual-Reality (VR)- und Augmented-Reality (AR)-Systeme interessant. Für die Darstellung einer Erweiterten Realität oder einer vollständig virtuellen Welt ist die Erfassung der Position und Blickrichtung des Benutzers notwendig. Viele VR/AR-Systeme nutzen hierfür Mobilgeräte mit Lagesensoren oder spezielle Display Brillen, die mit besonderen Sensoren ausgestattet sind. In alltäglichen Arbeitssituationen ist es im Allgemeinen nicht praktikabel. Dieser Artikel zeigt ein echtzeitfähiges, kamerabasiertes Verfahren, mit dem bewegte Personen in komplexen Umgebungen erfasst und so ein AR- oder VR-Display auf diese Perspektive eingestellt werden kann. Dabei ist insbesondere die Anforderung an die Robustheit des Verfahrens in unterschiedlichen Umgebungsbedingungen hervorzuheben.

Show publication details

Efficient slicing of Catmull–Clark solids for 3D printed objects with functionally graded material

2019

Computers & Graphics

In the competition for the volumetric representation most suitable for functionally graded materials in additively manufactured (AM) objects, volumetric subdivision schemes, such as Catmull-Clark (CC) solids, are widely neglected. Although they show appealing properties, e_cient implementations of some fundamental algorithms are still missing. In this paper, we present a fast algorithm for direct slicing of CC-solids generating bitmaps printable by multi-material AMmachines. Our method optimizes runtime by exploiting constant time limit evaluation and other structural characteristics of CCsolids. We compare our algorithm with the state of the art in trivariate trimmed spline representations and show that our algorithm has similar runtime behavior as slicing trivariate splines, fully supporting the benefits of CC-solids.

Show publication details
Tang, Chong; Lukas, Uwe von; Vahl, Matthias; Wang, Shuo; Tan, Min; Wang, Yu

Efficient underwater image and video enhancement based on Retinex

2019

Signal, Image and Video Processing

The Retinex models the human visual system to perceive natural colors, which could improve the contrast and sharpness of the degraded image and also provide color constancy and dynamic range simultaneously. This endows the Retinex exceeding advantages for enhancing the underwater image. Based on the multi-scale Retinex, an efficient enhancement method for underwater image and video is presented in this paper. Firstly, the image is pre-corrected to equalize the pixel distribution and reduce the dominating color. Then, the classical multi-scale Retinex with intensity channel is applied to the pre-corrected images for further improving the contrast and the color. In addition, multi-down-sampling and infinite impulse response Gaussian filtering are adopted to increase processing speed. Subsequently, the image is restored from logarithmic domain and the illumination of the restored image is compensated based on statistical properties. Finally, the color is selectively preserved by the inverted gray world method depending on imaging conditions and application requirements. Five kinds of typical underwater images with green, blue, turbid, dark and colorful backgrounds and two underwater videos are enhanced and evaluated on Jetson TX2, respectively, to verify the effectiveness of the proposed method.

Show publication details
Borowski, Tim; Barnikol, Maximilian; Stork, André [Betreuer]

Effiziente und präzise geometrische Simulation von 3-Achs-Fräsoperationen

2019

Darmstadt, TU, Master Thesis, 2019

Show publication details

End-to-end Color 3D Reproduction of Cultural Heritage Artifacts: Roseninsel Replicas

2019

GCH 2019

Eurographics Workshop on Graphics and Cultural Heritage (GCH) <17, 2019, Sarajevo, Bosnia and Herzegovina>

Planning exhibitions of cultural artifacts is always challenging. Artifacts can be very sensitive to the environment and therefore their display can be risky. One way to circumvent this is to build replicas of these artifacts. Here, 3D digitization and reproduction, either physical via 3D printing or virtual, using computer graphics, can be the method of choice. For this use case we present a workflow, from photogrammetric acquisition in challenging environments to representation of the acquired 3D models in different ways, such as online visualization and color 3D printed replicas. This work can also be seen as a first step towards establishing a workflow for full color end-to-end reproduction of artifacts. Our workflow was applied on cultural artifacts found around the “Roseninsel” (Rose Island), an island in Lake Starnberg (Bavaria), in collaboration with the Bavarian State Archaeological Collection in Munich. We demonstrate the results of the end-to-end reproduction workflow leading to virtual replicas (online 3D visualization, virtual and augmented reality) and physical replicas (3D printed objects). In addition, we discuss potential optimizations and briefly present an improved state-of-the-art 3D digitization system for fully autonomous acquisition of geometry and colors of cultural heritage objects.

Show publication details
Li, Muyu; He, Xin; Wei, Zhonghui; Wang, Jun; Mu, Zhiya; Kuijper, Arjan

Enhanced Multiple-Object Tracking Using Delay Processing and Binary-Channel Verification

2019

Applied Sciences : open access journal

Tracking objects over time, i.e., identity (ID) consistency, is important when dealing with multiple object tracking (MOT). Especially in complex scenes with occlusion and interaction of objects this is challenging. Significant improvements in single object tracking (SOT) methods have inspired the introduction of SOT to MOT to improve the robustness, that is, maintaining object identities as long as possible, as well as helping alleviate the limitations from imperfect detections. SOT methods are constantly generalized to capture appearance changes of the object, and designed to efficiently distinguish the object from the background. Hence, simply extending SOT to a MOT scenario, which consists of a complex scene with spatially mixed, occluded, and similar objects, will encounter problems in computational efficiency and drifted results. To address this issue, we propose a binary-channel verification model that deeply excavates the potential of SOT in refining the representation while maintaining the identities of the object. In particular, we construct an integrated model that jointly processes the previous information of existing objects and new incoming detections, by using a unified correlation filter through the whole process to maintain consistency. A delay processing strategy consisting of the three parts—attaching, re-initialization, and re-claiming—is proposed to tackle drifted results caused by occlusion. Avoiding the fuzzy appearance features of complex scenes in MOT, this strategy can improve the ability to distinguish specific objects from each other without contaminating the fragile training space of a single object tracker, which is the main cause of the drift results. We demonstrate the effectiveness of our proposed approach on the MOT17 challenge benchmarks. Our approach shows better overall ID consistency performance in comparison with previous works.

Show publication details
Fährmann, Daniel; Kuijper, Arjan [1. Review]; Terhörst, Philipp [2. Review]

Enhancing the Privacy of Face Recognition and its Representations

2019

Darmstadt, TU, Master Thesis, 2019

For these reasons, this work aims at preventing unauthorized deduction of private softbiometriccharacteristics from image representations. Latent features should be extractedfrom facial images, so that sparse feature representations are obtained. The featurerepresentations should be transformed in a way, that the predictive performance of softbiometricestimators is reduced. Biometric systems should still be able to recognize anindividual using the transformed representations.These objectives are achieved by the main contribution, the Thomson loss, that is presentedin this work. By using the Thomson loss a neural network learns a transformation that canbe applied to feature representations of facial images. After the feature representationshave been transformed, even non-binary soft-biometric estimators cannot make reliablepredictions anymore.

Show publication details
Dahlke, Jannik Oliver; Fellner, Dieter W. [1. Review]; Erdt, Marius [2. Review]

Enhancing the Texture Quality of 3D Building Models

2019

Darmstadt, TU, Master Thesis, 2019

3D-City Models have many applications, e.g urban emergency simulations, city planning, 3D city visualization for movies, games and more. Textures play an important role in realizing photorealistic and immersive renderings. Often these textures are of low quality due to the complexity of mass scale texture acquisition. One severe artefact is the occlusion of building facades through neighbouring roofs and trees. This thesis will explore a method that combines recent development in deep learning inpainting methods with domain specific knowledge about building facades. This method is able to remove an occlusion from a building facade by filling it with visually coherent regular facade elements. A state-of-the-art deep learning inpainting method has been extended by using segmentation information as additional input and additional loss metrics among other extensions. These proposed adaptions are able to improve upon current state-of-the-art methods by reducing blurriness, checkerboard artefacts, color smear and by increasing structural integrity of the building facades especially for inpainting large images.

Show publication details
Landesberger, Tatiana von [Program Chair]; Turkay, Cagatay [Betreuer]; Kohlhammer, Jörn [Steering Committee]; Keim, Daniel A. [Betreuer]; Fellner, Dieter W. [Proceedings Production Ed.]

EuroVA 2019

2019

International EuroVis Workshop on Visual Analytics (EuroVA) <10, 2019, Porto, Portugal>

  • 978-3-03868-087-1
Show publication details
Fix, Niklas; Kuijper, Arjan [1. Prüfer]; Basgier, Dennis [2. Prüfer]

Extending Mixed Reality Interaction Using External Sensor Systems

2019

Darmstadt, TU, Bachelor Thesis, 2019

Microsoft's HoloLens is a Mixed Reality supporting head mounted display (HMD) device which is available on the market since 2016. Its first generation model supports some fundamental hand gestures by itself. However, permanent tracking of the finger and joint positions is not supported. This drastically limits the user's ability to interact with the Mixed Reality environment. Phenomenons like hand occlusion further restrict the user's feeling of immersion. However, a combination of the HoloLens with modern hand-tracking systems like the Leap Motion Controller (LMC) could remove these limitations. The final goal of this work was to develop a generic framework for the registration of one or more external sensors with a MR device. The framework was demonstratively implemented for LMC and HoloLens. Based on it, a prototype was developed which renders the finger joint coordinates tracked by the LMC at their actual real-world position, i.e. where the user's hand is located, in a holographic application. For this purpose, a point transformation pipeline has been worked out which maps the LMC's tracking data onto their correspondent coordinates in holographic space. As a prerequisite, a setup of a LMC mounted on top of the HoloLens was used. In order to obtain accurate results, computing the pose of both devices relative to each other precisely is crucial. In this work, the pose estimation has been achieved using a two-step method. This approach is based on standard camera calibration methods using a chessboard pattern as calibration object. Due to the different fields of view of both device's cameras, the joint calibration of both devices turned out to be a major challenge which needed to be solved. The following thesis will describe how the prototype has been worked out, how transformation accuracy was ensured and in which ways it can be improved.

Show publication details
Huynh, Ngoc Anh; Ng, Wee Keong [Supervisor]; Kohlhammer, Jörn [Co-supervisor]

Frequency Analysis and Online Learning in Malware Detection

2019

Singapore, Nanyang Technological University, Diss., 2019

Traditional antivirus products are signature-based solutions, which rely on a static database to perform detection. The weakness of this design is that the signatures may become outdated, resulting in the failure to detect new samples. The other method is behavior-based detection, which aims to identify malware based on their dynamic behavior. Behavior-based detection comes in two approaches. The first approach leverages on common known behaviors of malware such as random domain name generation and periodicity. The second approach aims to directly learn the behavior of malware from data using tools such as graph analytics and machine learning. Behavior-based detection is di cult because we have to deal with intelligent and highly motivated attackers, who can change their strategy to maximize the chance of getting access to computer networks. We narrow our research to the domain of Windows malware detection and we are particularly interested in two approaches of behavior-based detection: periodic behavior and behavior evolution. Periodic behavior refers to the regular activities programmed by attackers such as periodic polling for server connection or periodic update of the victim machine's status. Behavior evolution refers to the change in behavior of malware over time. In the first approach, we aim to exploit the periodic behavior for malware detection. The main analysis tool in this direction is Fourier transform, which is used to convert time-domain signals into frequency domain signals. This idea is motivated by the fact that it is often easier to analyze periodic signals in the frequency domain than in the original time domain. Using Fourier transform, we propose a novel frequency-based periodicity measure to evaluate the regularity of network traffic. Another challenge in this direction is that, other than malware, most automatic services of operating systems also generate periodic signals. To address this challenge, we propose a new visual analytics solution for effective alert verification. In the second approach, we aim to develop adaptive learning algorithms to capture malware samples, whose behavior changes over time. We capitalize on the well-known online machine learning framework of Follow the Regularized Leader (FTRL). Our main contribution in this direction is the usage of an adaptive decaying factor to allow FTRL algorithms to better perform in environments with concept drifts. The decaying factor helps to increasingly discount the contribution of the examples in the past, thereby alleviating the problem of concept drifts. We advance the state of the art in this direction by proposing a new adaptive online algorithm to handle the problem of concept drift in malware detection. Our improved algorithm has also been successfully applied to other non-security domains.

Show publication details
Bartschat, Andreas; Allgeier, Stephan; Scherr, Tim; Stegmaier, Johannes; Bohn, Sebastian; Reichert, Klaus-Martin; Kuijper, Arjan; Reischl, Markus; Stachs, Oliver; Köhler, Bernd; Mikut, Ralf

Fuzzy tissue detection for real-time focal control in corneal confocal microscopy

2019

at - Automatisierungstechnik

Corneal confocal laser scanning microscopy is a promising method for in vivo investigation of cellular structures, e. g., of nerve fibers in the sub-basal nerve plexus. During recording, even slight displacements of the focal plane lead to images of adjacent tissue layers. In this work, we propose a closed-loop control of the focal plane. To detect and evaluate the visible tissues, we utilize the Bag of Visual Words approach to implement a customizable image processing pipeline for real-time applications. Furthermore, we show that the proposed model can be trained with small classification datasets and can be applied as a segmentation method. The proposed control loop, including tissue detection, is implemented in a proof-of-concept setup and shows promising results in a first evaluation with a human subject.

Show publication details
Rizvic, Selma [Program Co-Chair]; Rodriguez Echavarria, Karina [Program Co-Chair]; Fellner, Dieter W. [Proceedings Production Ed.]; Ramic-Brkic, Belma [Local Chair]

GCH 2019

2019

Eurographics Workshop on Graphics and Cultural Heritage (GCH) <17, 2019, Sarajevo, Bosnia and Herzegovina>

The Graphics and Cultural Heritage research community has vast experience in interdisciplinary research and in seeking technical innovation which has a societal application. As such, in this 17th edition of the Workshop on Graphics and Cultural Heritage (GCH 2019) we placed special attention on the role of this research community for proposing novel research which underpins the safeguarding of Cultural Heritage in the digital age while addressing the social, environmental and economic challenges. Taking place at the heart of the Balkans, in the city of Sarajevo, this year’s event explores the role of computer graphics and other digital technologies in the preservation and provision of access to cultural heritage which might be vulnerable from natural and man-made threats such as climate change, economic hardship, violence and neglect. The programme includes a variety of research contributions that address these pressing needs. Novel methods for the digitisation of artefacts are presented, including open and end-to-end processes for 3D documentation and reproductions, capturing complex materials, introducing multispectral imaging processes and finding compression methods for images resulting from digitisation processes. The analysis and classification of cultural heritage material is also presented, including methods for the analysis of historical films, analysis of cracks on painted surfaces, classification of clay statuettes, retrieval of painted pottery and the exploration methods for annotated datasets. Engagement with virtual environments is presented through research conducted on virtual museums, and Augmented Reality (AR) to engage the public and Virtual Reality (VR) environments to enable them to experience seismic simulations. 3D design research includes the design of ancient garments and the extraction of 3D scenes from bas-reliefs. Community engagement with cultural heritage is proposed through storytelling mechanisms using technologies such as AR, VR and 3D printed replicas.

  • 978-3-03868-082-6
Show publication details
Lukas, Uwe von [Hrsg.]; Bauer, Kristine [Hrsg.]; Dolereit, Tim [Hrsg.]; Flach, Guntram [Hrsg.]

Go-3D 2019: Mit 3D Richtung Maritim 4.0

2019

Go-3D <9, 2019, Rostock, Germany>

Die Konferenz Go-3D 2019 präsentiert aktuelle Forschungsergebnisse der 3D-Computergraphik sowie deren praktische Einsatzmöglichkeiten in der Industrie. Fachexperten aus Wissenschaft und Wirtschaft spannen in ihren Beiträgen ein breites Spektrum von der 3D-Erfassung über die Erstellung maritimer 3D-Applikationen bis zu virtuellen Trainingsumgebungen. Highlights sind in diesem Jahr Vorträge über Augmented Reality, Visuelle Assistenz und der Hands-on-Workshop zum Immersiven Lernen. Veranstalter der Konferenz ist das Kompetenznetzwerk "Go-3D - Effiziente Prozesskette für 3D-Computergraphik".

  • 978-3-8396-1499-0
Show publication details
Abrams, Jesse F.; Vashishtha, Anand; Wong, Seth T.; Nguyen, An; Mohamed, Azlan; Wieser, Sebastian; Kuijper, Arjan; Wilting, Andreas; Mukhopadhyay, Anirban

Habitat-Net: Segmentation of Habitat Images Using Deep Learning

2019

Ecological Informatics

Understanding environmental factors that influence forest health, as well as the occurrence and abundance of wildlife, is a central topic in forestry and ecology. However, the manual processing of field habitat data is time-consuming and months are often needed to progress from data collection to data interpretation. To shorten the time to process the data we propose here Habitat-Net: a novel deep learning application based on Convolutional Neural Networks (CNN) to segment habitat images of tropical rainforests. Habitat-Net takes color images as input and after multiple layers of convolution and deconvolution, produces a binary segmentation of the input image. We worked on two different types of habitat datasets that are widely used in ecological studies to characterize the forest conditions: canopy closure and understory vegetation. We trained the model with 800 canopy images and 700 understory images separately and then used 149 canopy and 172 understory images to test the performance of Habitat-Net. We compared the performance of Habitat-Net to the performance of a simple threshold based method, manual processing by a second researcher and a CNN approach called U-Net, upon which Habitat-Net is based. Habitat-Net, U-Net and simple thresholding reduced total processing time to milliseconds per image, compared to 45 s per image for manual processing. However, the higher mean Dice coefficient of Habitat-Net (0.94 for canopy and 0.95 for understory) indicates that accuracy of Habitat-Net is higher than that of both the simple thresholding (0.64, 0.83) and U-Net (0.89, 0.94). Habitat-Net will be of great relevance for ecologists and foresters, who need to monitor changes in their forest structures. The automated workflow not only reduces the time, it also standardizes the analytical pipeline and, thus, reduces the degree of uncertainty that would be introduced by manual processing of images by different people (either over time or between study sites).

Show publication details

Hand-Arm Vibration Estimation Using A Commercial Smartwatch

2019

14th International Conference on Hand-Arm Vibration

International Conference on Hand-Arm Vibration <14, 2019, Bonn, Germany>

Measuring Hand-Arm Vibration (HAV) exposure is important to prevent permanent injuries, such as the WhiteFinger / Raynaud Syndrome. Current measuring solutionsrequire an individual attachment of those work tools thatemit considerable vibrations. These sensing instrumentsare expensive and usually require a setup by experts. Additionally, these attached sensors are bulky and wired,which may further increase the risk of accidents in occupational safety. For an easy use, we propose using aSmartwatch to estimate the HAV doses gathered throughout the day. By utilizing the Smartwatch’s Inertial Measuring Unit (IMU) that is sampling up to 800Hz, we are capable of reconstructing vibrations up to 400Hz. This rangesufficiently covers the majority of harmful HAV loads thatoccurs with work tools. Our approach is an inexpensivesolution that provides a rough estimation to indicate a vibration overload. Our solution does not require the specific tool type or datasheet.

Show publication details

Health@Hand - A Visual Interface for eHealth Monitoring

2019

ICTS4eHealth'19: Fourth International Workshop on ICT Solutions for Health (ICTS4eHealth'19)

IEEE Symposium on Computers and Communications (ISCC) <2019, Barcelona, Spain>

The digitization within the healthcare sector is an extensive topic for multiple ICT technologies, such as Cloud Computing, Internet of Things (IoT), and Artificial Intelligence. Over the past years, there has been an increasing interest in the storage, evaluation and visual representation of medical data. This led to a substantial amount of independent eHealth technologies, that are primarily concerned with information gathering and management. However, the multitude of existing tools makes it is difficult for the user to understand global relationships between different management systems. Therefore, we present Health@Hand an IoT system for monitoring vital and administrative data in the digital twin of an intensive care unit (ICU). The main goal of our system is to summarize and process real-time data in order to assist medical experts such as physician, head-nurses or controllers in their daily tasks. Health@Hand achieves this by providing a visual interface for the management and analysis of different medical data sources.

Show publication details
Zhou, Wei; Ma, Calwen; Yao, Tong; Chang, Peng; Zhang, Qi; Kuijper, Arjan

Histograms of Gaussian Normal Distribution for 3D Feature Matching in Cluttered Scenes

2019

The Visual Computer

3D feature descriptors provide essential information to find given models in captured scenes. In practical applications, these scenes often contain clutter. This imposes severe challenges on the 3D object recognition leading to feature mismatches between scenes and models. As such errors are not fully addressed by the existing methods, 3D feature matching still remains a largely unsolved problem. We therefore propose our Histograms of Gaussian Normal Distribution (HGND) for capturing salient feature information on a local reference frame (LRF) that enables us to solve this problem. We define a LRF on each local surface patch by using the eigenvectors of the scatter matrix. Different from the traditional local LRF-based methods, our HGND descriptor is based on the combination of geometrical and spatial information without calculating the distribution of every point and its geometrical information in a local domain. This makes it both simple and efficient. We encode the HGND descriptors in a histogram by the geometrical projected distribution of the normal vectors. These vectors are based on the spatial distribution of the points.We use three public benchmarks, the Bologna, the UWA and the Ca’ Foscari Venezia dataset, to evaluate the speed, robustness, and descriptiveness of our approach. Our experiments demonstrate that the HGND is fast and obtains a more reliable matching rate than state-of-the-art approaches in cluttered situations.

Show publication details
Wochner, Paul Frederik Franz Ludwig; Walther, Thomas [1. Review]; Damer, Naser [2. Review]; Terhörst, Philipp [3. Review]

How Do Demographic Soft-Biometric Attributes Affect Kinship Verification ?

2019

Darmstadt, TU, Bachelor Thesis, 2019

In recent years, facial kinship verification has received considerable attention due to the easy acquisition of facial images and a large potential application area. Facial kinship verification is defined as the process to determine whether two identities are kin or not by automatically comparing their facial images. Facial kinship verification may have a wide range of potential uses, including aiding in the fight against human trafficking, handling conflicts resulting from the refugee crisis, family album organization, and social media analysis. Other potential applications lie in the academic field, such as genealogical studies and in the identification of the kin of victims or suspects by law enforcement [1], [2]. In Germany, from March 1951 to April 2019, a total of 1995 cases of missing children are unresolved as reported by the Bundeskriminalamt [3]. Due to the significant change in the look of children at adult age, the high similarity of a child’s appearance to their parents, and the much easier acquisition of photos than DNA, facial kinship verification could help resolve these, and similar cases. Unfortunately, the performance of such kinship verification systems is still too underdeveloped to be used for real-world applications [4]. One issue consists of the non-generalizability of currently available data sets to the real-world data distribution [2]. Lopez et al. received an acceptable accuracy on two data sets by only comparing the chrominance [5]. Inspired by this, Dawson et al. built a “From Same Photo” classifier to compete for the kinship verification task by only assigning those pictures as kin which originated from the same photo [6]. As another trait, Guo et al. included gender and age as information in the kinship verification process by only considering this information to determine whether a person of the potential kin pair is older or the age is approximately the same [7]. Although age and ethnicity have not yet been explicitly implemented into the kinship verification process, the present thesis analyzes the impact of gender and these attributes on the kinship verification process. Accordingly, two widely used data sets were labeled manually with gender, age, and ethnicity. The impact of the addition of these traits to the baseline model was then analyzed. These additional traits could improve the accuracy of the kinship verification process slightly. Only a softbiometric classifier, including gender, age, and ethnicity, was built. A significant fraction of the kinship verification process may be explained solely by these attributes because of an inappropriate data set composition. Moreover, an incorrect construction in the two analyzed data sets can be found, which evoked majorly from the same pictures and the same identities in different folds. Understanding the shortcomings of previously conducted research can help future researchers improve their development of the kinship verification process.

Show publication details
Subasioglu, Meltem; Damer, Naser [Supervisor]; Kaschube, Matthias [Supervisor]

Humans vs Machines: A Comparison of Human and Machine Learning Performance in Inferring Professions from Facial Images

2019

Frankfurt am Main, Univ., Master Thesis, 2019

Convolutional neural networks have seen a rise in the computer vision community in the recent years, even surpassing human accuracies in face recognition tasks. However, research regarding different face classification and face clustering experiments, which do not focus on the discrimination of face images according to unique individuals in the dataset, are still very limited. Many psychological studies indicate that humans are capable in inferring leaders, status and competence from facial images. This work focused on the question whether humans are indeed capable in telling profession – given by the branch of the profession and the career status – from face images and whether current state-of-the-art machine learning approaches are able to do the same. Furthermore, the performance of humans and machine learning systems were compared to give insights in the underlying processes. The results indicate that both human and machine learning models are capable to infer professions from facial images with better than chance accuracies. Both humans and machine learning systems perform almost equally well in these tasks, however, performance differences in individual tasks indicate that humans and machine learning algorithms solve the same tasks while relying on different cues in the face images.

Show publication details
Gigilashvili, Davit; Urban, Philipp; Thomas, Jean-Baptiste; Hardeberg, Jon Yngve; Pedersen, Marius

Impact of Shape on Apparent Translucency Differences

2019

27th Color and Imaging Conference

Color Imaging Conference (CIC) <27, 2019, Paris, France>

Translucency is one of the major appearance attributes. Apparent translucency is impacted by various factors including object shape and geometry. Despite general proposals that objectshape and geometry have a significant effect on apparent translucency, no quantification has been made so far. Quantifying andmodeling the impact of geometry, as well as comprehensive understanding of the translucency perception process, are a pointof not only academic, but also industrial interest with 3D printing as an example among many. We hypothesize that a presenceof thin areas in the object facilitates material translucency estimation and changes in material properties have larger impact onapparent translucency of the objects with thin areas. Computergenerated images of objects with various geometry and thicknesshave been used for a psychophysical experiment in order to quantify apparent translucency difference between objects while varying material absorption and scattering properties. Finally, absorption and scattering difference thresholds where the human visual system starts perceiving translucency difference need to beidentified and its consistency needs to be analyzed across different shapes and geometries.

Show publication details

Implementing Secure Applications in Smart City Clouds Using Microservices

2019

Future Generation Computer Systems

Smart Cities make use of ICT technology to address the challenges of modern urban management. The cloud provides an efficient and cost-effective platform on which they can manage, store and process data, as well as build applications performing complex computations and analyses. The quickly changing requirements in a Smart City require flexible software architectures that let these applications scale in a distributed environment such as the cloud. Smart Cities have to deal with huge amounts of data including sensitive information about infrastructure and citizens. In order to leverage the benefits of the cloud, in particular in terms of scalability and cost-effectiveness, this data should be stored in a public cloud. However, in such an environment, sensitive data needs to be encrypted to prevent unauthorized access. In this paper, we present a software architecture design that can be used as a template for the implementation of Smart City applications. The design is based on the microservice architectural style, which provides properties that help make Smart City applications scalable and flexible. In addition, we present a hybrid approach to securing sensitive data in the cloud. Our architecture design combines a public cloud with a trusted private environment. To store data in a cost-effective manner in the public cloud, we encrypt metadata items with CP-ABE (Ciphertext-Policy Attribute-Based Encryption) and actual Smart City data with symmetric encryption. This approach allows data to be shared across multiple administrations and makes efficient use of cloud resources. We show the applicability of our design by implementing a web-based application for urban risk management. We evaluate our architecture based on qualitative criteria, benchmark the performance of our security approach, and discuss it regarding honest-but-curious cloud providers as well as attackers trying to access user data through eavesdropping. Our findings indicate that the microservice architectural style fits the requirements of scalable Smart City applications while the proposed security approach helps prevent unauthorized access.

Show publication details
Stoll, Christian; Kuijper, Arjan [Prüfer]; Fu, Biying [Betreuerin]

Indoor Localization using Particle Filter approach for Single and Multiperson Application in Smart Home

2019

Darmstadt, TU, Bachelor Thesis, 2019

The variation of electric potential, caused by movements, can be used to detect activities in an indoor environment. To detect those variations a sensor system is required. In this thesis, we perform indoor localization, by using a smart floor system, which is able to detect and measure the change in electric potential. We develop a system, which uses three different particle filter approach to process the data from the smart floor system, in order to be able to perform indoor localization for single- and multi-person applications. This system is able to perform all steps necessary to solve this task. It processes the data, sent by the smart floor system, by updating the particles, clusters those particles and it finally assigns the centers of those clusters as positions to people, which are located on the smart floor. We will describe a theoretical approach to particle filter algorithms, and the clustering algorithm DBSCAN. Afterwards, we will describe and discuss our design descisions, as well as our implementation in detail and then evaluate the three proposed algorithms by examining their applicability in practical scenarios in a model apartment and their precision in a laboratory environment. Both those examinations and evaluations will be split into two parts, according to the split task of single- and multi-person applications. Finally, we will discuss those results and point out some properties which are specific for the approaches, we take at our particle filter algorithms and describe future adaptions, we consider necessary for a potential practical application, for the system to be more flexible in interaction and localizing different numbers of people.

Show publication details
Kelling, Jan; Kuijper, Arjan [1. Prüfer]; Voß, Gerrit [2. Prüfer]

Integrating physically based rendering (PBR) techniques into a multi-object-draw rendering framework

2019

Darmstadt, TU, Bachelor Thesis, 2019

To combine modern, textured, physically based rendering (PBR) techniques with optimized culling algorithms that split the scene into batches not by material but spatially, new approaches to resource management - especially texture management - are needed. PBR material descriptions usually contain multiple sets of textures for different properties, color, and normal information but the number of distinct textures that can be used simultaneously when rendering is traditionally limited. While dealing with the modernization of a non-textured and non-PBR pipeline, we describe texture atlas approaches (to handle the resulting texture space management problem), extend them to account for sampling techniques such as anisotropic filtering and evaluate them in comparison to optimized - but not always available - alternatives like bindless textures. Bindless textures lift the strict requirements on the number of textures that can be used at a time but require hardware support and are therefore not available on all devices. Our results show that texture atlas-based approaches can solve the texturing problem for spatially partitioned scene rendering without significant sacrifice of visual quality and that they can also be optimized to have a sufficiently small overhead in most cases. Nonetheless, we found bindless textures a solution worth using when available since it consistently outperformed our atlas-based solutions. In the worst cases, using texture atlases brought significant memory overhead compared to bindless textures.

Show publication details

Integrating Server-based Simulations into Web-based Geo-applications

2019

Eurographics 2019. Short Papers

Annual Conference of the European Association for Computer Graphics (Eurographics) <40, 2019, Genoa, Italy>

In this work, we present a novel approach for combining fluid simulations running on a GPU server with terrain rendered by a web-based 3D GIS system. We introduce a hybrid rendering approach, combining server-side and client-side rendering, to interactively display the results of a shallow water simulation on client devices using web technology. To display water and terrain in unison, we utilize image merging based on depth values.We extend it to deal with numerical and compression artifacts as well as Level-of-detail rendering and use Depth Image Based Rendering to counteract network latency.

Show publication details
Matthiesen, Moritz; Rojtberg, Pavel [1. Gutachten]; Kuijper, Arjan [2. Gutachten]

Interpolation von Kalibrierdaten für Zoom und Autofokus Kameras

2019

Darmstadt, TU, Bachelor Thesis, 2019

In dieser Arbeit wird das Problem betrachtet, dass für jede neue Kameraeinstellung eine neue Kalibrierung vorgenommen werden muss.Ziel dabei ist Kalibrierdaten an bestimmten Kameraeinstellungen zu erstellen, um mithilfe vondiesen die Kalibrierdaten von anderen Kameraeinstellungen herzuleiten. Dabei werden die Kalibrierdaten betrachtet und es wird versucht Beziehungen zwischen den einzelnen Parametern der Kalibrierung herzuleiten. Um diese zu ermitteln wird zwischen verschiedenen Parametern der Kalibrierung interpoliert.

Show publication details
Distergoft, Alexander; Kuijper, Arjan [1. Gutachten]; Mukhopadhyay, Anirban [2. Gutachten]

Interpreting Adversarial Examples in Medical Imaging

2019

Darmstadt, TU, Master Thesis, 2019

Deep neural networks (DNNs) have been achieving high accuracy on many important tasks like image classification, detection or segmentation. Yet, recent discoveries have shown a high degree of susceptibility for these deep-learning algorithms under attack. DNNs seem to be vulnerable to small amounts of non-random noise, created by perturbing the input to output mapping of the network. These perturbations can severely affect the performance of DNNs and thus endanger systems where such models are employed. The purpose of this thesis is to examine adversarial examples in clinical settings, be it digitally created or physical ones. For this reason we studied the performance of DNNs under the following three attack scenarios: 1. We hypothesize that adversarial examples might occur from incorrect mapping of the image space to the lower dimensional generation manifold. The hypothesis is tested by creating a proxy task of a pose estimation of surgical tools in its simplest form. For this we define a clear decision boundary. We use exhaustive search on a synthetic toy dataset to localize possible resions of successful one-pixel-attacks in image space. 2. We design a small scale prospective evaluation on how Deep-learning (DL) dermoscopy systems perform under physical world attacks. The publically available Physical Attacks on Dermoscopy Dataset (PADv1) is used for this evaluation. The introduced susceptibility and robustness values reveal that such attacks lead to accuracy loss across popular state-of-the-art DL-architectures. 3. As a pilot study to understand the vulnerabilities of DNNs that perform under regression tasks we design a set of auxiliary tasks that are used to create adversarial examples for non-classification-models. We train auxiliary networks on augmented datasets to satisfy the defined auxiliary tasks and create adversarial examples that might influence the decision of a regression model without knowing about the underlying system or hyperparameters.

Show publication details
Youssef, Salahedine; Kuijper, Arjan [1. Review]; Boutros, Fadi [2. Review]

Investigate the use of ultrasonic sensor for human pose estimation in smart environments

2019

Darmstadt, TU, Bachelor Thesis, 2019

Human monitoring is a major research direction in computer vision, with application in smart living assistants, human-computer interaction, surveillance, health monitoring, etc. This variety of applications has led to the design of many human monitoring systems in order to extract information about environment inhabitances based on different technologies. In computer vision, this task can be achieved by generating a 2D skeleton representing the human body. However, users do not favor constant camera monitoring. This thesis investigates the use of ultrasonic sensors for human pose estimation, which have a very low cost and require only minimalistic infrastructure. We do this by establishing a framework for data collection of human poses, using an ultrasound sensor and a camera for labeling the data. We collected data from 25 people, performing 4 different activities and evaluated the collected data on 4 different artificial neural network architectures by training them and comparing their performance against each other, showing that an LSTM architecture achieved results up to 67% accuracy. The use of a non-visual input stream for pose estimation is also motivated by the less privacy intrusive nature of ultrasound data, compared with videos of homes and people inside them with the application in smart living environments.

Show publication details
Braun, Andreas; Zander-Walz, Sebastian; Majewski, Martin; Kuijper, Arjan

Investigating Large Curved Interaction Devices

2019

Personal and Ubiquitous Computing

Large interactive surfaces enable novel forms of interaction for their users, particularly in terms of collaborative interaction. During longer interactions, the ergonomic factors of interaction systems have to be taken into consideration. Using the full interaction space may require considerable motion of the arms and upper body over a prolonged period of time, potentially causing fatigue. In this work, we present Curved, a large-surface interaction device, whose shape is designed based on the natural movement of an outstretched arm. It is able to track one or two hands above or on its surface by using 32 capacitive proximity sensors. Supporting both touch and mid-air interaction can enable more versatile modes of use. We use image processing methods for tracking the user's hands and classify gestures based on their motion. Virtual reality is a potential use case for such interaction systems and was chosen for our demonstration application. We conducted a study with ten users to test the gesture tracking performance, as well as user experience and user preference for the adjustable system parameters.

Show publication details
Boller, Viola; Pesavento, Marius [1. Gutachten]; Damer, Naser [2. Gutachten]

Investigating the Use of Deeply Calculated Flows and Dynamic Routed Network Capsules in Face Presentation Attack Detection

2019

Darmstadt, TU, Master Thesis, 2019

Identifying an individual based on their facial characteristics is convenient and is gaining more and more importance. Only a normal camera is needed for data capturing, the individual characteristic can’t be stolen or forgot like a password, and it’s a non-intrusive capturing method, which enables identification from distances. With increasing usage of face recognition systems, also the amount of sophisticated attacks increase. Therefore vulnerabilities of such a recognition system are used. Especially the attack directly at the sensor, by presenting fake biometrics at the sensor, can be performed with low efforts and costs, and without detailed knowledge about the biometric system. That’s why this kind of attacks have caused wide attention in the biometric community. The attack is called presentation attack and there are many detection algorithms developed to solve this issue. Most existing presentation attack detection algorithms work great, when deployed in the same environment as they are trained in. But different environmental conditions are still challenging to solve. This thesis deals with the presentation attack issue by developing an approach, which combines deep optical flows and a capsule neural network. Optical flows have already proven to be a good preprocessing step, because only motion information is kept and environmental conditions won’t have a big impact on it. Deep optical flows are produced by end-to-end trained convolutional neural networks for the optical flow calculation. Capsule neural networks train neurons in a more general way by forwarding information of found pattern’s poses in the network additionally to it’s probabilities. Forwarding is achieved by a so called routing-by-agreement method, which allows to take relationships between the found patterns into account. For presentation attack detection development not enough data is available to train a deep neural network from scratch. This could be overcome by using a capsule neural network for classification. A deep understanding can be learned, needing less data. The contribution of this thesis is the combination of deep optical flows and a capsule neural network, which hasn’t been tested so far. This is realized by stacking multiple optical flows over the time and downsample this information, thus that the capsule neural network is able to train a classification. This pipeline is structurally optimized by testing different optical flow sizes, data normalization and multiple class training. Also a new downsampling and time stacking method is developed to keep outliers in the reduced data and order a stack according to the contained data information. This has proven to help the network to classify. In the end, the behavior of the network on various attacks is analyzed. In this thesis, performance improvements are achieved by structure optimizations and a novel downsampling and time stacking method. The structural improvements increased the model performance up to 12% and the new downsampling and time stacking method leads to an average performance improvement of 7.56% for two databases. However, the achieved performance does not outperform recent state-of-the-art presentation attack detection methods and the analysis over different attacks reveals that the developed structure adapts weakly to changes in the attack performance or to new attacks. Nevertheless, stable results for different environmental conditions are achieved.

Show publication details
Fellner, Dieter W. [Hrsg.]; Welling, Daniela [Red.]; Ackeren, Janine van [Red.]; Bergstedt, Bettina [Red.]; Krüger, Kathrin [Red.]; Prasche, Svenja [Betreuer]

Jahresbericht 2018

2019

Das Fraunhofer IGD hat seine Forschungsaktivitäten vor Kurzem in vier Leitthemen gebündelt, welche die Basis seiner Arbeit bilden und verschiedene Themen abteilungsübergreifend miteinander verknüpfen. Eines dieser Leitthemen ist "Visual Computing as a Service - Die Plattform für angewandtes Visual Computing". Die Basis dieser universellen Plattform für Visual-Computing Lösungen ist gelegt und wird kontinuierlich erweitert. Dieser technologische Ansatz bildet die Grundlage für die weiteren Leitthemen. In der "Individuellen Gesundheit - Digitale Lösungen für das Gesundheitswesen" werden die Daten betrachtet, die in der personalisierten Medizin anfallen - mithilfe der Visual-Computing-Technologien des Instituts. Im Leitthema "Intelligente Stadt - Innovativ, digital und nachhaltig" ist die Fragestellung, wie man den Lebenszyklus urbaner Prozesse unterstützen kann. Und im Leitthema "Digitalisierte Arbeit - Der Mensch in der Industrie 4.0" geht es erster Linie um die Unterstützung des Menschen in der durch die Digitalisierung veränderten Produktion.

Show publication details

Joint Schedule and Layout Autotuning for Sparse Matrices with Compound Entries on GPUs

2019

Vision, Modeling, and Visualization

Vision, Modeling, and Visualization (VMV) <24, 2019, Rostock, Germany>

Large sparse matrices with compound entries, i.e., complex and quaternionic matrices as well as matrices with dense blocks, are a core component of many algorithms in geometry processing, physically based animation, and other areas of computer graphics. We generalize several matrix layouts and apply joint schedule and layout autotuning to improve the performance of the sparse matrix-vector product on massively parallel graphics processing units. Compared to schedule tuning without layout tuning, we achieve speedups of up to 5:5x. In comparison to cuSPARSE, we achieve speedups of up to 4:7x.

Show publication details
Antony, Niklas; Urban, Bodo [Gutachter]; Bieber, Gerald [Betreuer]

Konzeption und Entwicklung einer Kommunikationsstruktur für autonom-mobile Assistenzsysteme

2019

Rostock, Univ., Bachelor Thesis, 2019

In Deutschland herrscht ein Mangel an Arbeitskräften in Krankenhäusern und Pflegeheimen, welcher durch die steigende Lebenserwartung stetig zunimmt. Mithilfe von autonom-mobilen Robotern lassen sich Pflegekräfte bei ihrer Arbeit entlasten. Lokalisierung und Zielfindung sind die wesentlichen Funktionalitäten dieser Roboter. In der vorliegenden Arbeit wird ein Konzept für ein autonom-mobiles Assistenzsystem mit darauf aufbauendem Prototypen vorgestellt, das QR-Codes als Orientierungspunkte verwendet. Die Einsatzumgebung beschränkt sich auf Innenräume. Hierbei ist ein Smartphone über Bluetooth mit dem mobilen Roboter verbunden, das mit der Rück-Kamera an die Raumdecke gerichtet ist und die dort befestigten QR-Codes ausliest. Die QR-Codes speichern ausreichend Informationen über ihre Umgebung, sodass es dem Roboter möglich ist, ausschließlich mithilfe dieser Informationen und ohne Suchalgorithmus einen zugewiesenen Zielpunkt zu erreichen. Um Kollisionen zu vermeiden, ist der Roboter zusätzlich mit einem Ultraschallsensor ausgestattet. Die Ergebnisse aus Versuchen mit dem Prototypen werden vorgestellt sowie Möglichkeiten der Weiterentwicklung erwähnt.

Show publication details
Zielinski, Michael; Urban, Bodo [Gutachter]; Nonnemann, Lars [Betreuer]

Koordiniertes Workflow Management mit Hilfe von Brushing & Linking Mechanismen

2019

Rostock, Univ., Bachelor Thesis, 2019

Die Art und Weise wie Datenmengen untersucht werden, erlebte in den letzten Jahren einige Veränderungen. Da die Menge der Daten mit der Zeit immer weiter anwächst, müssen verschiedene Analysewerkzeuge zur Untersuchung der Daten verwendet werden. Durch die gleichzeitige Nutzung mehrerer Werkzeuge entsteht jedoch die Problematik, dass unterschiedliche Aspekte wie beispielsweise die Position oder die Skalierung der Werkzeugansichten koordiniert werden müssen. Die Koordination dieser Werkzeuge reicht dabei vom sequentiellen Ausführen der Werkzeuge bis zur Nutzung von anwendungsspezifischen Analysesystemen. Weiterhin haben sich oftmals Workflows zur Analyse der Datenmengen etabliert. Im Kontext eines Workflows werden die Analysewerkzeuge in vielen Fällen jedoch nicht ausreichend miteinander koordiniert, sodass die Datenübergabe oder das Starten und Beenden von Analysewerkzeugen manuell durchgeführt werden muss. Dies führt zu erhöhtem Arbeitsaufwand auf Seiten der Datenanalytiker. Ziel dieser Arbeit ist es, basierend auf dem aktuellen Stand der Technik Konzepte zur workflowbasierten Datenanalyse zu entwickeln, um die Koordination unterschiedlicher Werkzeuge in analytischen Prozessen zu unterstützen. Dabei wird die Aufteilung des Informationsraumes auf Daten- und Visualisierungsebene untersucht. Weiterhin nutzt der entwickelte Ansatz Brushing & Linking zur Verknüpfung mehrerer Ansichten durch anwendungsübergreifende Interaktion. Die konzeptionell entwickelten Ansätze werden prototypisch in das vom Fraunhofer IGD entwickelte Health@Hand-System eingebunden. Dieses System ist ein virtueller Leitstand zur Visualisierung von Vitaldaten, damit beispielsweise Anomalien festgestellt werden können, sodass bei Störfällen frühzeitig interveniert wird. Um repetitive Arbeitsprozesse automatisieren zu können, wird der workflow-basierte Ansatz dieser Arbeit in der Implementierung in Health@Hand verwendet. Dabei wird ein gegebenes Szenario zur Detektion von kardiovaskulären Symptomen und Anomalien umgesetzt. Diese Arbeit gliedert sich in sieben Kapitel. Im nachfolgenden zweiten Kapitel werden grundlegende Techniken zum Aufteilen und Navigieren von Ansichten im Darstellungs- sowie Informationsraum vorgestellt. Darauf folgt in Kapitel 3 die Beschreibung von aktuellen Systemen, welche jeweils unterschiedliche Probleme der Ansichts- und Werkzeugkoordination lösen. Anschließend wird in Kapitel 4 eine Problem- und Anforderungsanalyse zur workflow-basierten Datenanalyse durchgeführt. Auf Basis dessen wird in den Kapiteln 5 und 6 das entwickelte Konzept eines Systems zur workflow-basierten Datenanalyse beschrieben und die prototypische Implementierung dessen in Health@Hand vorgestellt. Die Arbeit schließt schlussendlich mit einem Fazit der entwickelten Ansätze sowie einem Ausblick der noch auszuführenden Arbeiten ab.

Show publication details
Kaymak, Yasin; Kuijper, Arjan [1. Gutachten]; Ben Hmida, Helmi [2. Gutachten]

Künstliche Intelligenz Widget für ClickDigital

2019

Darmstadt, TU, Bachelor Thesis, 2019

Die in den letzten Jahren vorangetriebene Umsetzung der Vision Internet der Dinge(IoT) hat auch zu einem massiven Zuwachs von IoT-Geräten gesorgt. Dadurch hat im IoT-Umfeld der Aspekt Sicherheit immer mehr an Bedeutung gewonnen. Das Gewährleisten einer reibungslosen und sicheren Nutzung der IoT-Geräte würde auch das Fortsetzten der Vision IoT unterstützen. Eine Gegenmaßnahme Sicherheit zu gewährleisten wären Anomalie-Erkennungsalgorithmen. Diese Algorithmen sind in der Lage, die von den IoT-Geräten erzeugten Unmengen von Daten (Big Data) nach Fehlverhalten zu analysieren, um überhaupt die Möglichkeit anzubieten, Gegenmaßnahmen zu treffen. In dieser Arbeit wird ein Algorithmus vorgestellt, der die von IoT-Geräten erzeugten Daten nach Fehlverhalten untersuchen kann. Zudem wird dieser Algorithmus Teil der Entwicklungsumgebung ClickDigital.

Show publication details
Uecker, Marc; Kügler, David [1. Prüfer]; Kuijper, Arjan [2. Prüfer]

LEMBAS: Latent Embedding Based Architecture Search

2019

Darmstadt, TU, Bachelor Thesis, 2019

In the field of computer vision, deep learning is an essential element of many applications. Its success has long extended to various industries, far beyond machine learning research. Many state-of-the-art approaches succeed by developing custom neural network architectures for specific tasks. However, most practical applications borrow pre-defined architectures from standard image classification benchmarks. While this approach is useful if the benchmark task closely resembles the target task, it does not guarantee good performance in other circumstances. Therefore, designing custom architectures would likely improve the performance of many applications. Neural network design requires experience, statistical knowledge, and experimentation. Many developers lack expertise, computational resources, or time for this process. This is especially true for researchers outside the machine learning field. One solution to this problem is presented by neural architecture search. It is an automated approach to neural architecture design. Existing methods deliver good performance, but often require datacenter-scale computation power. In this thesis, the process of automatic neural architecture search is investigated in a portable, low- resource setting. The presented methods are an adaptable, low-cost solution for searching neural architectures on custom tasks. This work also provides an extensible codebase, which natively scales to various hardware configurations.

Show publication details
Schulz, Hans-Jörg; Röhlig, Martin; Nonnemann, Lars; Aehnelt, Mario; Diener, Holger; Urban, Bodo; Schumann, Heidrun

Lightweight Coordination of Multiple Independent Visual Analytics Tools

2019

IVAPP 2019. Proceedings

International Conference on Information Visualization Theory and Applications (IVAPP) <10, 2019, Prague, Czech Republic>

With the advancement of Visual Analytics (VA) and its spread into various application fields comes along a specialization of methods and tools. This adds complexity and requires extra effort when devising domain-dependent VA solutions, as for every new domain question a new specialized tool or framework must be developed. In this paper, we investigate the possibility of using and re-using existing tools – domain-dependent and general-purpose – by loosely coupling them into specialized VA tool ensembles as needed. We call such coupling among independent tools lightweight coordination, as it is minimally-invasive, pair-wise, and opportunistic in utilizing whichever interface a VA tool offers. We propose the use of lightweight coordination for managing the workflow, the data flow, and the control flow among VA tools, and we show how it can be supported with suitable setups of the multiple tool UIs involved. This concept of lightweight coordination is exemplified with a health care scenario, where an ensemble of independent VA tools is used in a concerted way to pursue the visual analysis of a patient’s troublesome vital data.

  • 978-989-758-354-4
Show publication details
Montefiori, Erica; Modenese, Luca; Di Marco, Roberto; Magni-Manzoni, Silvia; Malattia, Clara; Petrarca, Maurizio; Ronchetti, Anna; Tanturri De Horatio, Laura; Dijkhuizen, Pieter van; Wang, Anqi; Wesarg, Stefan; Viceconti, Marco; Mazzà, Claudia

Linking Joint Impairment and Gait Biomechanics in Patients with Juvenile Idiopathic Arthritis

2019

Annals of Biomedical Engineering

Juvenile Idiopathic Arthritis (JIA) is a paediatric musculoskeletal disease of unknown aetiology, leading to walking alterations when the lower-limb joints are involved. Diagnosis of JIA is mostly clinical. Imaging can quantify impairments associated to inflammation and joint damage. However, treatment planning could be better supported using dynamic information, such as joint contact forces (JCFs). To this purpose, we used a musculoskeletal model to predict JCFs and investigate how JCFs varied as a result of joint impairment in eighteen children with JIA. Gait analysis data and magnetic resonance images (MRI) were used to develop patient-specific lower-limb musculoskeletal models, which were evaluated for operator-dependent variability (< 3.6°, 0.05 N kg21 and 0.5 BW for joint angles, moments, and JCFs, respectively). Gait alterations and JCF patterns showed high between-subjects variability reflecting the pathology heterogeneity in the cohort. Higher joint impairment, assessed with MRI-based evaluation, was weakly associated to overall joint overloading. A stronger correlation was observed between impairment of one limb and overload of the contralateral limb, suggesting risky compensatory strategies being adopted, especially at the knee level. This suggests that knee overloading during gait might be a good predictor of disease progression and gait biomechanics should be used to inform treatment planning.

Show publication details
Pandikow, Lars; Kuijper, Arjan [1. Gutachten]; Gutbell, Ralf [2. Gutachten]

Localization and Mapping of Monocular Cameras in Large Urban Environments using Virtual City Models

2019

Darmstadt, TU, Master Thesis, 2019

In recent years there has been a lot of progress on the task of simultaneous localization and mapping (SLAM) of image sequences from monocular cameras. Latest methods utilize advancements in machine learning to improve the quality of both the camera tracking as well as the reconstruction of the environment. One of these methods called CNN-SLAM uses a neural network to estimate depth maps for keyframes and fuses them with stereo observations from neighboring image frames. To make use of the images within a geo-referenced context they first have to be localized globally. The use of additional sensors to determine the position and orientation of the camera is not only more expensive, but also sensitive to errors. This thesis proposes a real-time system that combines the methods of CNN-SLAM with image based localization within a simple city model. The SLAM algorithm tracks the camera movement while the use of a depth estimation network enables the recovery of the scale of the scene. A genetic algorithm is implemented to quickly refine estimated camera poses by aligning synthetic views of the city model with semantic segmentations of the images. This does not only localize the camera trajectories but also helps to compensate tracking errors caused by the SLAM algorithm. The evaluation showed the systems ability to compute scaled trajectories correctly, to compensate tracking drift and densly reconstruct the scene in the vicinity of the camera. It also revealed the unreliability of image localization without a constraint search space, tracking drift during rotational movement and inaccurate semantic segmentations.

Show publication details
Bülow, Maximilian von; Guthe, Stefan; Ritz, Martin; Santos, Pedro; Fellner, Dieter W.

Lossless Compression of Multi-View Cultural Heritage Image Data

2019

GCH 2019

Eurographics Workshop on Graphics and Cultural Heritage (GCH) <17, 2019, Sarajevo, Bosnia and Herzegovina>

Photometric multi-view 3D geometry reconstruction and material capture are important techniques for cultural heritage digitalization. Capturing images of artifacts with high resolution and high dynamic range and the possibility to store them losslessly enables future proof application of this data. As the images tend to consume immense amounts of storage, compression is essential for long time archiving. In this paper, we present a lossless image compression approach for multi-view and material reconstruction datasets with a strong focus on data created from cultural heritage digitalization. Our approach achieves compression rates of 2:1 compared against an uncompressed representation and 1.24:1 when compared against Gzip.

Show publication details
Iffland, Thomas; Senner, Ivo [Referent]; Dominik, Andreas [Koreferent]

Microservice-Testansätze und Frameworkevaluation

2019

Giessen, Technische Hochschule Mittelhessen, Master Thesis, 2019

Show publication details
Cui, Jian; Sourin, Alexei [Supervisor]; Fellner, Dieter W. [Co-supervisor]; Kuijper, Arjan [Co-supervisor]

Mid-air hand interaction with optical tracking for 3D modelling

2019

Singapore, Nanyang Technological Univ., Diss., 2019

Compared to common 2D interaction done with mouse and other 2D-tracking devices, 3D hand tracking with low-cost optical cameras can provide more degrees of freedom, as well as natural gestures when shape modeling and assembling are done in virtual spaces. However, though quite precise, the optical tracking devices cannot avoid problems intrinsic to hand interaction, such as hand tremor and jump release, and they also introduce an additional problem of occlusion. This thesis investigates whether interaction during 3D modeling can be improved by using optical sensors so that 3D tasks can be performed in a way similar to interaction in real life and as efficient as when using common 2D-tracking based interaction while still minimizing the intrinsic problems of precise hand manipulations and optical problems. After surveying the relevant works and analyzing technical capabilities of the commonly available optical sensors, two approaches are thoroughly investigated for the natural mid-air hand interaction in precise 3D modeling – they are collision-based and gesture-based interaction. For collision-based methods, a set of virtual interaction techniques is proposed to realistically simulate real-life manipulation and deformation with one and two hands. For gesture-based interaction, a core set of interaction techniques is also devised which allows natural real-life interaction ways to be used. In addition, algorithms are proposed for both collision-based and gesture-based interaction to enhance the precision while minimizing the problems of hand tremor and jump release. However, the results show that virtual interaction designed with collision-based methods is always slower than real-life interaction due to missing force feedback. Although common gesture-based interaction is less affected by its problem, it still cannot completely get rid of the problems of occlusion and jump release. Eventually, a new method of gesture-based interaction is proposed to use hands in a way similar to how it is done when playing the Theremin – an electronic musical instrument controlled without physical contact by hands of the performer. It is suggested that the dominant hand controls manipulation and deformation of objects while the non-dominant hand controls grasping, releasing and variable precision of interaction. Based on this method, a generic set of reliable and precise gesture-based interaction techniques is designed and implemented for various manipulation and deformation tasks. It is then proved with the user studies that for the tasks involving 3D manipulations and deformations, the proposed way of hand interaction is easy to learn, not affected by the common problems of hand tracking, as well as more convenient and faster than common 2D interaction done with mouse for some 3D tasks.

Show publication details

Minutiae-Based Gender Estimation for Full and Partial Fingerprints of Arbitrary Size and Shape

2019

Computer Vision - ACCV 2018

Asian Conference on Computer Vision (ACCV) <14, 2018, Perth, Australia>

Lecture Notes in Computer Science (LNCS)
11361

Since fingerprints are one of the most widely deployed biometrics, accurate fingerprint gender estimation can positively affect several applications. For example, in criminal investigations, gender classification may significantly minimize the list of potential subjects. Previous work mainly offered solutions for the task of gender classification based on complete fingerprints. However, partial fingerprint captures are frequently occurring in many applications, including forensics and the fast growing field of consumer electronics. Due to its huge variability in size and shape, gender estimation on partial fingerprints is a challenging problem. Therefore, in this work we propose a flexible gender estimation scheme by building a gender classifier based on an ensemble of minutiae. The outputs of the single minutia gender predictions are combined by a novel adjusted score fusion approach to obtain an enhanced gender decision. Unlike classical solutions this allows to deal with unconstrained fingerprint parts of arbitrary size and shape. We performed investigations on a publicly available database and our proposed solution proved to significantly outperform state-of-the-art approaches on both full and partial fingerprints. The experiments indicate a reduction in the gender estimation error by 19.34% on full fingerprints and 28.33% on partial captures in comparison to previous work.

  • 978-3-030-20886-8
Show publication details
Tran, Mai Ly; Kuijper, Arjan [1. Gutachten]; Terhörst, Philipp [2. Gutachten]

Mitigating Ethnic Bias in Face Recognition Models through Fair Template Comparison

2019

Darmstadt, TU, Master Thesis, 2019

Face recognition systems find many uses in daily life. For example, they can be used to unlock your phone or automatically tag a person in a photo but they are also used in other application fields such as in security environments or surveillance. However, there is a significant problem with these systems: they are often biased. These systems make much more mistakes on women and darker-skinned people than on men and light-skinned people. This bias comes from data which is heavily skewed towards light-skinned men and the systems learn from these data, reflecting this bias. As face recognition systems become more prevalent, solving this problem increasingly gains importance, especially when these mistakes can have a large impact, such as when they are used for identifying criminals but entire groups of persons are discriminated. The important question is: How can the bias be reduced as much as possible so that the systems get fairer while maintaining a sufficient recognition performance? There are several ways to tackle bias. Previous approaches tried to introduce balanced datasets or remove features which may lead to a bias. However, often they have to deal with the challenge of providing enough data for a balanced dataset or with performance drops. This is especially true for minority groups, as it is intrinsically hard to collect more data for them. Therefore, there exists an even stronger bias against minority groups. In this thesis, the focus is on reducing the ethnic bias of facial recognition systems through a fair template comparison method: We propose applying two different fairness concepts during the training of template comparison models by adding them as penalization terms to the loss function. The first concept, group fairness, aims at equalizing groups while the second concept, individual fairness, aims at equal treatment for similar individuals. Our approach is evaluated on two different datasets. The template comparison is realized as logistic regression and neural network models. The experiments show not only the influence of the fairness terms but also that it is possible to achieve a fairer system without a significant face recognition performance drop.

Show publication details
Rana, Ubaid; Kuijper, Arjan [1. Prüfer]; Burkhardt, Dirk [2. Prüfer]

Named-Entity Recognition on Publications and Raw-Text for Meticulous Insight at Visual Trend Analytics

2019

Darmstadt, TU, Master Thesis, 2019

In the modern data-driven era, a massive amount of research documents are available from publicly accessible digital libraries in the form of academic papers, journals and publications. This plethora of data does not lead to new insights or knowledge. Therefore, suitable analysis techniques and graphical tools are needed to derive knowledge in order to get insight of this big data. To address this issue, researchers have developed visual analytical systems along with machine learning methods, e.g text mining with interactive data visualization, which leads to gain new insights of current and upcoming technology trends. These trends are significant for researchers, business analysts, and decision-makers for innovation, technology management and to make strategic decisions. Nearly every existing search portal uses the traditional meta-information e.g only about the author and title to find the documents that match a search request and overlook the opportunity of extracting content-related information. It limits the possibility of discovering most relevant publications, moreover it lacks the knowledge required for trend analysis. To collect this very concrete information, named entity recognition must be used to be able to better identify the results and trends. The state-of-the-art systems use static approach for named entity recognition which means that upcoming technologies remain undetected. Modern techniques like distant supervision methods leverage big existing community-maintained data sources, such as Wikipedia, to extract entities dynamically. Nonetheless, these methods are still unstable and have never been tried on complex scenarios such as trend analysis before. The aim of this thesis is to enable entity recognition on both static tables and dynamic com- munity updated data sources like Wikipedia & DBpedia for trend analysis. To accomplish this goal, a model is suggested which will enable entity extraction on DBpedia and translate the extracted entities into interactive visualizations. The analysts can use these visualizations to gain trend insights, evaluate research trends or to analyze prevailing market moods and indus- try trends.

Show publication details

Nocturnal Respiration Pattern of healthy people as a hint for sleep state detection

2019

Proceedings of the 12th ACM International Conference on PErvasive Technologies Related to Assistive Environments

ACM International Conference on PErvasive Technologies Related to Assistive Environments (PETRA) <12, 2019, Rhodes, Greece>

ACM International Conference Proceedings Series (ICPS)
01608

Sleep state detection is important to distinguish between a healthy sleep and sleep disorders. Common sleep state analysis methods consist of identifying signals of EEG, EOG, or EMG etc. that can only be assessed in sleep laboratories. The respiration rate and pattern are also affected by the sleep states but are not included in the sleep state analysis method. Since sleep is very important for the recreation of humans, we assume that sleep is mirroring the strain of the day and the general health condition. In our research, we identified a certain respiration rate pattern during sleep in 5 out of 17 healthy persons that might be an identifier for sleep states or for interactions of daytime activity and sleep. Therefore, we introduce this new respiration pattern as “pumping breathing” and compare it with other known respiration patterns.

Show publication details
Debiasi, Luca; Damer, Naser; Moseguí Saladié, Alexandra; Rathgeb, Christian; Scherhag, Ulrich; Busch, Christoph; Kirchbuchner, Florian; Uhl, Andreas

On the Detection of GAN-Based Face Morphs Using Established Morph Detectors

2019

Image Analysis and Processing - ICIAP 2019

International Conference of Image Analysis and Processing (ICIAP) <20, 2019, Trento, Italy>

Lecture Notes in Computer Science (LNCS)
11752

Face recognition systems (FRS) have been found to be highly vulnerable to face morphing attacks. Due to this severe security risk, morph detection systems do not only need to be robust against classical landmark-based face morphing approach (LMA), but also future attacks such as neural network based morph generation techniques. The focus of this paper lies on an experimental evaluation of the morph detection capabilities of various state-of-the-art morph detectors with respect to a recently presented novel face morphing approach, MorGAN, which is based on Generative Adversarial Networks (GANs). In this work, existing detection algorithms are confronted with different attack scenarios: known and unknown attacks comprising different morph types (LMA and MorGAN). The detectors’ performance results are highly dependent on the features used by the detection algorithms. In addition, the image quality of the morphed face images produced with the MorGAN approach is assessed using well-established no-reference image quality metrics and compared to LMA morphs. The results indicate that the image quality of MorGAN morphs is more similar to bona fide images compared to classical LMA morphs.

  • 978-3-030-30644-1
Show publication details

Ontology in Holonic Cooperative Manufacturing: A Solution to Share and Exchange the Knowledge

2019

Knowledge Discovery, Knowledge Engineering and Knowledge Management

International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management (IC3K) <9, 2017, Funchal, Madeira, Portugal>

Communications in Computer and Information Science (CCIS)
976

Cooperative manufacturing is a new trend in industry, which depends on the existence of a collaborative robot. A collaborative robot is usually a light-weight robot which is capable of operating safely with a human co-worker in a shared work environment. During this cooperation, a vast amount of information is exchanged between the collaborative robot and the worker. This information constructs the cooperative manufacturing knowledge, which describes the production components and environment. In this research, we propose a holonic control solution, which uses the ontology concept to represent the cooperative manufacturing knowledge. The holonic control solution is implemented as an autonomous multi-agent system that exchanges the manufacturing knowledge based on an ontology model. Ultimately, the research illustrates and implements the proposed solution over a cooperative assembly scenario, which involves two workers and one collaborative robot, whom cooperate together to assemble a customized product.

  • 978-3-030-15639-8
Show publication details
Wientapper, Folker; Kuijper, Arjan [Referent]; Fellner, Dieter W. [Referent]; Stricker, Didier [Betreuer]

Optimal Spatial Registration of SLAM for Augmented Reality

2019

Darmstadt, TU., Diss., 2019

Augmented reality (AR) is a paradigm that aims at fusing the perceived real environment of a human with digital information located in 3D space. Typically, virtual 3D graphics are overlayed into the captured images of a moving camera or directly into the user's field-of-view by means of optical see-through displays (OST). For a correct perspective and view-dependent alignment of the visualization, it is required to solve various static and dynamic geometric registration problems in order to create the impression that the virtual and the real world are seamlessly interconnected.The advances during the last decade in the field of simultaneous localization and mapping (SLAM) represent an important contribution to this general problem. It is now possible to reconstruct the real environment and to simultaneously capture the dynamic movements of a camera from the images without having to instrument the environment in advance. However, SLAM in general can only partly solve the entire registration problem, because the retrieved 3D scene geometry and the calculated motion path are spatially related only with regard to an arbitrarily selected coordinate system. Without a proper reconciliation of coordinate systems (spatial registration), the real world of the human observer still remains decoupled from the virtual world. Existing approaches for solving this problem either require the availability of a virtual 3D model that represents a real object with sufficient accuracy (model-based tracking), or they rely on use-case specific assumptions and additional sensor data (such as GPS signals or the Manhattan-world assumption). Therefore, these approaches are bound to these additional prerequisites, which limit the general applicability. The circumstance that automated registration is desirable but not always possible, creates the need for techniques that allow a user to specify connections between the real and the virtual world when setting up AR applications, so that it becomes possible to support and control the process of registration. These techniques must be complemented with numerical algorithms that optimally exploit the provided information to obtain precise registration results.Within this context, the present thesis provides the following contributions.* We propose a novel, closed-form (non-iterative) algorithm for calculating a Euclidean or a similarity transformation. The presented algorithm is a generalization of recent state-of-the-art solvers for computing the camera pose based on 2D measurement points in the image (perspective-n-point problem) - a fundamental problem in computer vision that has attracted research for many decades. The generalization consists in extending and unifying these algorithms, so that they can handle other types of input correspondences than originally designed for. With this algorithm, it becomes possible to perform a rigid registration of SLAM systems to a target coordinate system based on heterogeneous and partially indeterminate input data.* We address the global refinement of structure and motion parameters by means of iterative sparse minimization (bundle adjustment or BA), which has become a standard technique inside SLAM systems. We propose a variant of BA in which information about the virtual domain is integrated as constraints by means of an optimization-on-manifold approach. This serves for compensating low-frequency deformations (non-rigid registration) of the estimated camera path and the reconstructed scene geometry caused by measurement error accumulation and the ill-conditionedness of the BA problem.* We present two approaches in which a user can contribute with his knowledge for registering a SLAM system. In a first variant, the user can place markers in the real environment with predefined connections to the virtual coordinate system. Precise positioning of the markers is not required, rather they can be placed arbitrarily on surfaces or along edges, which notably facilitates the preparative effort. During run-time, the dispersed information is collected and registration is accomplished automatically. In a second variant, the user is given the possibility to mark salient points in an image sequence during a preparative preprocessing step and to assign corresponding points in the virtual 3D space via a simple point-and-click metaphor. The result of this preparative phase is a precisely registered and ready-to-use reference model for camera tracking at run-time.* Finally, we propose an approach for geometric calibration of optical see-trough displays. We present a parametric model, which allows to dynamically adapt the rendering of virtual 3D content to the current viewpoint of the human observer, including a pre-correction of image aberrations caused by the optics or irregularly curved combiners. In order to retrieve its parameters, we propose a camera-based approach, in which elements of the real and the virtual domain are simultaneously observed. The calibration procedure was developed for a head-up display in a vehicle. A prototypical extension to head-mounted displays is also presented.

Show publication details
Kraus, Maurice; Kuijper, Arjan [1. Gutachten]; Stein, Christian [2. Gutachten]

Out-of-Core Speicherverwaltung für die Visualisierung großer Modelle in Spiele Engines

2019

Darmstadt, TU, Bachelor Thesis, 2019

Game engines are becoming an integral part of industrial visualization, especially they allow for the rapid development of applications that would otherwise have to be created by a dedicated expert in a specialized graphics program. Furthermore, it is sometimes necessary to display 3D data from different sources (e.g., CAD programs) on several devices with differing limitations. Unfortunately, game engines are historically used to create static content. Static in the sense that all contained models, textures, lighting, etc. are known in advance. This results in various problems. In the context of industrial data, 3D models are often too large to be loaded at once, which means that they need to be loaded into the engine subsequently. This, in turn, requires optimized memory management. However, here the topological and structural addressability often decreases. While one difficulty is the sheer size of the data, the other challenge is that the data to be visualized is not known in advance. Consequently, it is necessary to import the data into the engine at runtime. This thesis aims to contribute to a framework that allows the visualization of massive 3D models in game engines. For this purpose, the integration and implementation of approaches for runtime asset loading are evaluated. Furthermore, algorithms will be proposed to optimize memory management, which in turn will allow visualizing large models on limited memory. Finally, the applicability of the algorithms for different memory boundaries will be investigated within a prototypical application.

Show publication details
Getto, Roman; Fellner, Dieter W. [Betreuer]; Schreck, Tobias [Betreuer]

Parametric Procedural Models for 3D Object Retrieval, Classification and Parameterization

2019

Darmstadt, TU., Diss., 2019

The amount of 3D objects has grown over the last decades, but we can expect that it will grow much further in the future. 3D objects are also becoming more and more accessible to non-expert users. The growing amount of available 3D data is welcome for everyone working with this type of data, as the creation and acquisition of many 3D objects is still costly. However, the vast majority of available 3D objects are only present as pure polygon meshes. We arguably can not assume to get meta-data and additional semantics delivered together with 3D objects stemming from non-expert or 3D scans of real objects from automatic systems. For this reason content-based retrieval and classification techniques for 3D objects has been developed.Many systems are based on the completely unsupervised case. However, previous work has shown that there are strong possibilities of highly increasing the performance of these tasks by using any type of previous knowledge. In this thesis I use procedural models as previous knowledge. Procedural models describe the construction process of a 3D object instead of explicitly describing the components of the surface. These models can include parameters into the construction process to generate variations of the resulting 3D object. Procedural representations are present in many domains, as these implicit representations are vastly superior to any explicit representation in terms of content generation, flexibility and reusability. Therefore, using a procedural representation always has the potential of outclassing other approaches in many aspects. The usage of procedural models in 3D object retrieval and classification is not highly researched as this powerful representation can be arbitrary complex to create and handle. In the 3D object domain, procedural models are mostly used for highly regularized structures like buildings and trees.However, Procedural models can deeply improve 3D object retrieval and classification, as this representation is able to offer a persistent and reusable full description of a type of object. This description can be used for queries and class definitions without any additional data. Furthermore, the initial classification can be improved further by using a procedural model: A procedural model allows to completely parameterize an unknown object and further identify characteristics of different class members. The only drawback is that the manual design and creation of specialized procedural models itself is very costly. In this thesis I concentrate on the generalization and automation of procedural models for the application in 3D object retrieval and 3D object classification.For the generalization and automation of procedural models I propose to offer different levels of interaction for a user to fulfill the possible needs of control and automation. This thesis presents new approaches for different levels of automation: the automatic generation of procedural models from a single exemplary 3D object. The semi-automatic creation of a procedural model with a sketch-based modeling tool. And the manual definition a procedural model with restricted variation space. The second important step is the insertion of parameters into the procedural model, to define the variations of the resulting 3D object. For this step I also propose several possibilities for the optimal level of control and automation: An automatic parameter detection technique. A semi-automatic deformation based insertion. And an interface for manually inserting parameters by choosing one of the offered insertion principles. It is also possible to manually insert parameters into the procedures if the user needs the full control on the lowest level.To enable the usage of procedural models directly for 3D object retrieval and classification techniques I propose descriptor-based and deep learning based approaches. Descriptors measure the difference of 3D objects. By using descriptors as comparison algorithm, we can define the distance between procedural models and other objects and order these by similarity. The procedural models are sampled and compared to retrieve an optimal object retrieval list. We can also directly use procedural models as data basis for a retraining of a convolutional neural network. By deep learning a set of procedural models we can directly classify new unknown objects without any further large learning database. Additionally, I propose a new multi-layered parameter estimation approach using three different comparison measures to parameterize an unknown object. Hence, an unknown object is not only classified with a procedural model but the approach is also able to gather new information about the characteristics of the object by using the procedural model for the parameterization of the unknown object.As a result, the combination of procedural models with the tasks of 3D object retrieval and classification lead to a meta concept of a holistically seamless system of defining, generating, comparing, identifying, retrieving, recombining, editing and reusing 3D objects.

Show publication details

Performing Indoor Localization with Electric Potential Sensing

2019

Journal of Ambient Intelligence and Humanized Computing

Location-based services or smart home applications all depend on an accurate indoor positioning system. Basically one divides these systems into token-based and token-free localization systems. In this work, we focus on the token-free system based on smart floor technology. Smart floors can typically be built using pressure sensors or capacitive sensors. However, these set-ups are often hard to deploy as mechanical or electrical features are required below the surface and even harder to replace when detected a sensor malfunctioning. Therefore we present a novel indoor positioning system using an uncommon form of passive electric field sensing (EPS), which detects the electric potential variation caused by body movement. The EPS-based smart floor set-up is easy to install by deploying a grid of passive electrode wires underneath any non-conductive surfaces. Easy maintenance is also ensured by the fact that the sensors are not placed underneath the surface, but on the side. Due to the passive measuring nature, low power consumption is achieved as opposed to active capacitive measurement. Since we do not collect image data as in visual-based systems and all sensor data is processed locally, we preserve the user’s privacy. The proposed architecture achieves a high position accuracy and an excellent spatial resolution. Based on our evaluation conducted in our living lab, we measure a mean positioning error of only 12.7 cm.

Show publication details
Pflanzer, Yannick; Kuijper, Arjan [1. Gutachten]; Ritz, Martin [2. Gutachten]

Phenomenological Acquisition and Rendering of Optical Material Behavior for Entire 3D Objects

2019

Darmstadt, TU, Bachelor Thesis, 2019

In the last few years, major improvements in 3D scanning and rendering technology have been accomplished. Especially the acquisition of surface appearance information has seen innovation thanks to phenomenological approaches for capturing lighting behavior. In this work, the current Bi-directional Texturing Function (BTF) and Approximate- BTF (ABTF) approaches were extended to allow for a greater depth of effects to be captured as well as the ability to reproduce entire 3D objects from different viewing angles. The proposed Spherical Harmonic BTF (SHBTF) is able to model the captured surface appearance of objects by encoding all measured light samples into spherical harmonic coefficients, allowing for calculation of the surface appearance for any given light direction. In contrast to the ABTF, an SHBTF can capture multiple views of the same object which enables it to efficiently reproduce anisotropic material properties and subsurface scattering in addition to the spatially varying effects captured by an ABTF. The CultArc3D capturing setup used for all measurements is versatile enough to deliver view and light samples from a full hemisphere around an arbitrary object. It is now possible to capture entire 3D objects as opposed to many other BTF acquisition techniques. Challenges for the SH based lighting solution are ringing artifacts, growing stronger with rising SH bands. Another challenge for a full 3D experience was the re-projection of camera images onto a 3D model, depending heavily on the camera hardware calibration. The SH based approach has the potential to produce compelling results given further optimizations of the SH and re-projection accuracy.

Show publication details
Jabhe, Ali Khesrau; Kuijper, Arjan [1. Prüfer]; Kügler, David [2. Prüfer]

Physical World Attacks in Medical Imaging

2019

Darmstadt, TU, Bachelor Thesis, 2019

During this decade Deep Learning (DL) has become a game changer in the area of MedicalImaging. More and more tasks whether classification, detection, segmentation, etc. are beingsolved by DL algorithms with significantly high performance gains compared to traditionalmethods. However, Deep Neural Network (DNN)-based classification models have been foundto be vulnerable to adversarial attacks - maliciously and digitally manipulated image data -resulting in misclassification of the model.While only observed in Medical Imaging, adversarial examples define an established securitythreat to computer vision tasks. Furthermore, recent research has demonstrated theexistence of adversarial examples in the physical world - perturbations by physical artifactscausing misclassification instead of digitally perturbed data fed into the model. Since the existenceof adversarial examples in the physical world is not yet explored in Medical Imagingand trends are towards DL methods starting to get approved for clinical practice, this problemneeds more attention.Using the example of skin lesion classification, as it is likely to be the next aspirant for afully automated medical product, this work throws light on the potential threat adversarialexamples in the physical world pose to patients, physicians, and the healthcare system by generatinga new dataset "Physical Attacks in Dermoscopy" (PADv1) and evaluate susceptibilityand robustness of five state-of-the-art DNN-based skin lesion classifiers under physical attack.All architectures were trained on the HAM10000 dataset, a comprehensive challenge-verifiedbenchmark dataset for Machine Learning. Results show that on average the accuracy of thesearchitectures drops by 30.8%, and robustness decreases by 50% which means that every seconddiagnosis is fooled. Furthermore, a visualization of Gradient-weighted class activationmaps is provided to help understand or even interpret a models decision-making on clean andattacked image data.

Show publication details
Rajkarnikar, Angeelina; Kuijper, Arjan [1. Prüfer]; Mukhopadhyay, Anirban [2. Prüfer]

Practical Defense against Medical Adversarial Examples

2019

Darmstadt, TU, Master Thesis, 2019

This thesis aims to provide practical solutions for two different problems using Machine Learning and Deep Learning (DL) techniques in Healthcare applications: a) A practical defense against adversarial attacks so that DL can be used without vulnerabilities. b) A practical framework for evaluating Machine Learning (ML) algorithms on spectroscopic data. With increasing benefits of using deep learning in clinical applications, the vulnerabilities of Deep Neural Networks to adversarial examples are also surfacing. Adversarial examples are images that are injected with imperceptible perturbations that results in it’s misclassification in image classification problems. Healthcare, being a safety critical field is vulnerable to adversarial attacks, yet there are limited research on practical defenses on making the network robust against these attacks. We propose a defense strategy against adversarial attacks on deep learning classification models used in the field of medical diagnosis by adding random noise at inference time. Empirically, we evaluate the effectiveness of this simple defense strategy against single-step and iterative adversarial attacks. We demonstrate our technique through experiments, by introducing randomness, the accuracy of attacked images are increased. This techniques shows an significant increase in accuracy from 2.2% to 98.5% for MNIST, 8% to 88.08% for CIFAR-10 and 2.4% to 76.81% for HAM10000. This approach makes no assumptions about the model architecture. Hence it is simple to implement, yet effective in developing more robust deep learning models. Infrared (IR) spectroscopy is a non-invasive analytical method successfully used for multicomponent analysis of bio-fluid samples. We develop a ML framework with various multivariate calibration algorithms to predict the concentrations of components in the dialysate using IR spectroscopy. We analytically show that the obtained results of correlation between predicted and clinical reference methodology readings for 5 components are reasonable.

Show publication details

Presenting a Data Imputation Concept to Support the Continuous Assessment of Human Vital Data and Activities

2019

Proceedings of the 12th ACM International Conference on PErvasive Technologies Related to Assistive Environments

ACM International Conference on PErvasive Technologies Related to Assistive Environments (PETRA) <12, 2019, Rhodes, Greece>

ACM International Conference Proceedings Series (ICPS)
01608

Data acquisition of mobile tracking devices often suffers from invalid and non-continuous input data streams. This issue especially occurs with current wearables tracking the user’s activity and vital data. Typical reasons include the short battery life and the fact that the body-worn tracking device may be doffed. Other reasons, such as technical issues, can corrupt the data and render it unusable. In this paper, we introduce a data imputation concept which complements and thus fixes incomplete datasets by using a new merging approach that is particularly suitable for assessing activities and vital data. Our technique enables the dataset to become coherent and comprehensive so that it is ready for further analysis. In contrast to previous approaches, our technique enables the controlled creation of continuous data sets that also contain information on the level of uncertainty for possible reconversions, approximations, or later analysis.

Show publication details
Lenhart, Malte; Kuijper, Arjan [Betreuer]; Wilmsdorff, Julian von [Betreuer]

Prototyping Platform for Capacitive and Passive Electrical Field Sensing

2019

Darmstadt, TU, Bachelor Thesis, Jahr

In this thesis the Linoc prototyping toolkit is presented. It is built around two capac- itive and two Electric Potential Sensing (EPS) groups providing unobtrusive proximity detection in the field of Human Computer Interface (HCI). The toolkits focus lies on its usability in order to be adapted in future research and novel use cases. Its strength is the possibility to change its configuration at run time. A common obstacle in the beginning of a project is the time required to familiarize with present tools and systems, before the actual project can be attended to. This can be made worse by dependencies on previous work, often not fully documented and without training given from the original designer. Good toolkits can help to overcome this problem by providing a layer of abstraction and allowing to work on a higher level. If the toolkit however requires too much time to familiarize or behaves too restrictive, its goal has been missed and no benefits are generated. To access the quality of this thesis’ work, the Linoc toolkit was evaluated in terms of three different aspects: demonstration, usage and technical performance. A usage study found good reception, a steep learning curve and an interest to use the toolkit in the future. Technical benchmarks show a detectable range equal to its predecessors and in a demonstration it was shown that the toolkit can actually be used in projects.

Show publication details
Horn, Angelina; Stork, André [1. Gutachten]; Luu, Thu Huong [2. Gutachten]

Prozedurale Erzeugung von Füllstrukturen orientiert an der medialen Achse

2019

Darmstadt, TU, Bachelor Thesis, 2019

In dieser Arbeit wird ein neuer Ansatz von Fullstrukturen zur additiven Fertigung, welcher sich an der medialen Achse orientiert, vorgestellt und implementiert. Die Grundlagen die zum Verstandnis dieses Ansatzes notwendig sind werden ebenfalls erortert. Diese bestehen aus der additiven Fertigung, der prozeduralen Generierung sowie den geometrischen Beschaffenheiten der medialen Achse und des Voronoi Diagramms. Im Zusammenhang damit werden die Vor- und Nachteile der medialen Achse und ihre Wichtigkeit im Bezug auf den vorgestellten Ansatz erlautert. Es wird eine Auswahl an aktuellen Arbeiten vorgestellt, welche direkt mit den aufgezeigten Grundlagen und dem medialen Ansatz in Verbindung stehen. Des Weiteren wird eine Auswahl an vorhandenen Fullmustern analysiert, so wie ihre Starken und Schwachen aufgezeigt. Daraufhin wird eine neue Fullstruktur vorgestellt, welche eine grose Schwache der meisten vorhandenen Fullstrukturen nicht aufweist. Diese besteht darin, dass sich die meisten Fullmuster an der globalen x- und y-Achse orientieren. Sie ist nicht an das globale Koordinatensystem gebunden und soll so zur Stabilitat der Fullstruktur beitragen. Es wird erlautert, dass die Programme Voxel Core und Erosion Thickness zur Gewinnung der medialen Achse eines Modells fur die Implementierung des medialen Ansatzes verwendet werden und ihre Funktionalitat erklart. Des Weiteren wird die Konvertierung der medialen Achse im Polygon File Format dar gelegt. Die fur die Implementierung notwendige Erzeugung der Zellen und die Generierung des Fullmusters wird daraufhin konkretisiert. Es konnte gezeigt werden, dass die Berechnung des aus dem medialen Ansatzes entstehende Fullstruktur deutlich schneller ist als die zweier anderer Fullstrukturen. Es wird die Annahme getroffen, dass ebenfalls der Materialverbrauch geringer ist. Dies und die Stabilitat der generierten Fullstrukturen bleibt allerdings noch zu prufen. Abschliesend werden die Erkenntnisse dieser Arbeit in drei grundlegende Punkte zusammengefasst, welche bei der zukunftigen Implementierung des medialen Ansatzes zu berucksichtigen sind. Diese enthalten die Beschaffenheit der medialen Achse, die Anpassbarkeit der Fullstruktur und die Notwendigkeit der Analyse der medialen Achse.

Show publication details
Bulduk, Botan; Stork, André [1. Gutachten]; Luu, Thu Huong [2. Gutachten]

Prozedurale innere Strukturen für den 3D-Druck

2019

Darmstadt, TU, Bachelor Thesis, 2019

In dieser Arbeit wird eine neue Methode zur Erstellung von inneren Strukturen für den 3D-Druck vorgestellt. Diese basiert auf Skeletten mit denen sich Volumen generieren lassen können.

Show publication details
Mukhopadhyay, Anirban; Kügler, David; Bucher, Andreas; Fellner, Dieter W.; Vogl, Thomas J.

Putting Trust First in the Translation of AI for Healthcare

2019

Ercim News

From screening diseases to personalised precision treatments, AI is showing promise in healthcare. But how comfortable should we feel about giving black box algorithms the power to heal or kill us?

Show publication details
Bors, Christian; Bernard, Jürgen; Bögl, Markus; Gschwandtner, Theresia; Kohlhammer, Jörn; Miksch, Silvia

Quantifying Uncertainty in Multivariate Time Series Pre-Processing

2019

EuroVA 2019

International EuroVis Workshop on Visual Analytics (EuroVA) <10, 2019, Porto, Portugal>

In multivariate time series analysis, pre-processing is integral for enabling analysis, but inevitably introduces uncertainty intothe data. Enabling the assessment of the uncertainty and allowing uncertainty-aware analysis, the uncertainty needs to bequantified initially. We address this challenge by formalizing the quantification of uncertainty for multivariate time series preprocessing. To tackle the large design space, we elaborate key considerations for quantifying and aggregating uncertainty. Weprovide an example how the quantified uncertainty is used in a multivariate time series pre-processing application to assess theeffectiveness of pre-processing steps and adjust the pipeline to minimize the introduction of uncertainty.

Show publication details

Real-time texturing for 6D object instance detection from RGB Images

2019

Adjunct Proceedings of the 2019 IEEE International Symposium on Mixed and Augmented Reality

IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct) <18, 2019, Beijing, China>

For objected detection, the availability of color cues strongly influences detection rates and is even a prerequisite for many methods. However, when training on synthetic CAD data, this information is not available. We therefore present a method for generating a texture-map from image sequences in real-time. The method relies on 6 degree-of-freedom poses and a 3D-model being available. In contrast to previous works this allows interleaving detection and texturing for upgrading the detector on-the-fly. Our evaluation shows that the acquired texture-map significantly improves detection rates using the LINEMOD [5] detector on RGB images only. Additionally, we use the texture-map to differentiate instances of the same object by surface color.

Show publication details
Boutros, Fadi; Kuijper, Arjan [1. Gutachten]; Damer, Naser [2. Gutachten]

Reducing Ethnic Bias of Face Recognition by Ethnic Augmentation

2019

Darmstadt, TU, Master Thesis, 2019

Automated face recognition has gained wider deployment ground after the recent accuracy gains achieved by deep learning techniques. Despite the rapid performance gains, face recognition still suffers from very critical issues. One of the recently uncovered, and very sensitive, challenges is the ethnicity bias in face recognition decision. This is the case, unfortunately, even in the latest commercial and academic face recognition system. In 2018, the National Institute of Standards and Technology (NIST) published the latest report regarding the evaluation result of commercial face recognition solutions from several major biometric vendors. The report specifically evaluated and demonstrated the variance of the error rates of the evaluated solutions based on demographic variations. This thesis is one of the first research efforts in addressing the decision bias challenge in biometrics. It builds its hypothesis on the strong assumption that ethnicity bias is caused by the relative under-representation of certain ethnicities in training databases. This work introduces a novel ethnicity-driven data augmentation approach to reduce biometric bias. The proposed approach successfully utilize a generative image model to generate new face images that preserve the identity of their source images while partially transforming their ethnicity to the targeted ethnicity group. A large-scale ethnicity-labeled face images database is created in order to develop and evaluate the proposed approach. To achieve that, part of this thesis focused on creating an ethnicity classifier to annotate face images, achieving accuracy in the state-of-the-art range. The proposed ethnicity-driven face generative model is developed based on the ethnicity labeled images to generate realistically and high-resolution face images, depending on a limit amount of training data. More importantly, the thesis proves that the proposed augmentation approach strongly preserves the identity of the input images and partially transforms the ethnicity. The augmented images are used as part of the training data of a face recognition model. The achieved verification results prove that the proposed ethnicity augmentation methods significantly and consistently reduced the ethnicity bias of the face recognition model. For examples, the ERR was reduced from 0.159 to 0.130 when verifying inter-ethnicity samples of Black individuals on a model trained on Asian individual images, respectively before and after applying the proposed training data augmentation. Moreover, the overall performance of the face recognition model was improved. However, this improvement was more significant, as intended, in the targeted ethnicity groups.

Show publication details
Fuchs, Moritz; Kuijper, Arjan [1. Gutachten]; Noll, Matthias [2. Gutachten]

Registrierung eines Biopsieroboters zu bildbasierten Planungsdaten

2019

Darmstadt, TU, Bachelor Thesis, 2019

Das Setzen eines perkutanen Zugangs durch einen Roboter ist erstrebenswert, um die Genauigkeit von Eingriffen, wie Biopsien, thermale und non-thermale Ablation, zu verbessern. Für das dafür verwendete Robotersystem ist es notwendig, die Transformation des Roboters innerhalb eines CT zu bestimmen. Der Roboter hat hierfür ein Registrierungswerkzeug, dass auch die Nadelführung beinhaltet, an seinem Endeffektor angebracht. Das Registrierungswerkzeug beinhaltet vier Kugeln, die als Marker für die Ermittlung der Transformation dienen. In dieser Arbeit wurde sich damit beschäftigt die Kugeln zu lokalisieren. Es wird hier zu ein zweistufiges Verfahren verwendet, dass zunächst die eine Region of Interest(ROI) lokalisiert um anschließend die genau Transformation zu ermitteln. Für die Ermittlung der genauen Transformation wurden Optimierungsverfahren getestet, als auch Merkmal-Extraaktionstechniken. Als Optimierungsverfahren werden der Amoeba Algorithmus und der Gradientenabstieg, in Verbindung mit verschieden Metriken und Modellen getestet. Jedoch sind die Ergebnisse der Optimierer abhängig von der Initialisierung. Eine fehlerhafte Initialisierung konnte von den Optimierern nicht ausgeglichen werden. Bei der Merkmal-Extraaktionstechnik werden die Mittelpunkte der vier Kugeln bestimmt und zu einem Modell registriert. Zur Bestimmung der Kugelmittelpunkte wurde die Hough-Transformation verwendet. Es lässt damit eine zuverlässige Genauigkeit von DICE = 0.95879 auf den alten Registrierungswerkzeugen erreichen. Durch die bessere Fertigung und Bildqualität ist auf neueren Registrierungswerkzeugen ein DICE = 0.98515 möglich. Die Translationsabweichung ist dabei auf Sub-Voxel-Spacing genau.

Show publication details
Frank, Sebastian; Kuijper, Arjan

Robust Driver Foot Tracking and Foot Gesture Recognition Using Capacitive Proximity Sensing

2019

Journal of Ambient Intelligence and Smart Environments

Nowadays, there is an increasing trend towards automated driving. This is supported by both driver assistance systems getting more and more available and powerful, and research for car manufacturing industries. As a consequence, driver hands and feet are less involved in vehicle control. Increasing automation will even let them become idle. Recent gesture recognition mainly focuses on hand interaction. This work focuses on possibilities for feet gesture interaction. Many gesture recognition systems rely on computing intensive, privacy concerns causing video systems. Furthermore, these systems require a line of sight and therefore visible interior design integration. The proposed system shows that invisibly integrated capacitive proximity sensors can do the job, too. They do not cause privacy issues and they can be integrated under non-conductive materials. Therefore, there is no visible interior design impact. The proposed solution distinguishes between four feet gestures. There is no limitation to feet movement. Further, an evaluation including six participants and a vehicle legroom mockup shows the system function. This work contributes to the basis of driver foot gesture recognition pointing to further applications and more comprehensive investigations.

Show publication details
Bieber, Gerald; Chodan, Wencke; Bader, Rainer; Hölle, Bernd; Herrmann, Peter; Dreher, Ingo

RoRo – A New Robotic Rollator Concept to Assist the Elderly and Caregivers

2019

Proceedings of the 12th ACM International Conference on PErvasive Technologies Related to Assistive Environments

ACM International Conference on PErvasive Technologies Related to Assistive Environments (PETRA) <12, 2019, Rhodes, Greece>

ACM International Conference Proceedings Series (ICPS)
01608

People who suffer from difficulties in ambulating can be supported by using wheeled walking frames, also called rollators. Mechanical rollators are very helpful and provide physical stability but their functionality is limited. Electro-powered rollators can support the user whenever motor power is needed, e.g., when walking uphill or crossing the curbside of a sidewalk. The full potential of electric and smart rollators is not yet used. In this paper, we describe a new Robotic Rollator (RoRo) concept. The aim of RoRo is to guide elderly people autonomously through clinics and rehabilitation homes, e.g., to lead them to the radiology department or to the physiotherapist’s office. Furthermore, RoRo trains the elderly and examines their mobility, stability, and strength, as well as their visual-spatial and cognitive abilities. For this purpose, RoRo is equipped with additional sensors to monitor vital data of the user and to relate them to the physical load. The autonomous rollator RoRo interacts in the closed controlled indoor environments with infrared markers (that cannot be seen by humans) to allow spatial positioning. In addition to the technological aims of RoRo, another focus of the ongoing project lies on a balanced interaction between RoRo and the patient to motivate therapeutic exercises, physical activity (like going for a walk), and simple entertainment. In the future, the autonomous rollator may become a social robot that trains and accompanies the user like a personal acquaintance.

Show publication details

Seamless and Non-repetitive 4D Texture Variation Synthesis and Real-time Rendering for Measured Optical Material Behavior

2019

Computational Visual Media

We show how to overcome the single weakness of an existing fully automatic system for acquisition of spatially varying optical material behavior of real object surfaces. While the expression of spatially varying material behavior with spherical dependence on incoming light as a 4D texture (an ABTF material model) allows flexible mapping onto arbitrary 3D geometry, with photo-realistic rendering and interaction in real time, this very method of texture-like representation exposes it to common problems of texturing, striking in two disadvantages. Firstly, non-seamless textures create visible artifacts at boundaries. Secondly, even a perfectly seamless texture causes repetition artifacts due to their organised placement in large numbers over a 3D surface. We have solved both problems through our novel texture synthesis method that generates a set of seamless texture variations randomly distributed over the surface at shading time. When compared to regular 2D textures, the inter-dimensional coherence of the 4D ABTF material model poses entirely new challenges to texture synthesis, which includes maintaining the consistency of material behavior throughout the 4D space spanned by the spatial image domain and the angular illumination hemisphere. In addition, we tackle the increased memory consumption caused by the numerous variations through a fitting scheme specifically designed to reconstruct the most prominent effects captured in the material model.

Show publication details
Ma, Jingting; Lin, Feng [Supervisor]; Erdt, Marius [Supervisor]; Fellner, Dieter W. [Co-Supervisor]; Wesarg, Stefan [Co-Supervisor]

Self-learning Shape Recognition in Medical Images

2019

Singapore, Nanyang Technological University, Master Thesis, 2019

A massive amount of medical image data, e.g. from Computed Tomography (CT) and Magnetic Resonance Imaging (MRI), is generated from hospitals every day. Biological structure segmentation is very useful to support surgery planning and treatments, as an ideal delineation of the outline of the target object can offer a precise location and quantitative analysis for further clinical diagnoses such as identification of tumorous tissues. However, the large dimension and complex patterns in medical image data make manual annotation extremely time-consuming and problematic. Accordingly, automatic biomedical image segmentation becomes a crucial pre-requisite in practice and has been a critical research issue over tens of years. However, major challenges exist in medical image segmentation such as the low intensity contrast to surrounding tissues and complex geometry of shape. Moreover, limited amounts of labeled training data give rise to difficulties as well. Numerous approaches have been proposed to mitigate these challenges, from low-level image processing to supervised machine learning techniques. It is worth mentioning that statistical shape models (SSMs) based segmentation approaches have achieved remarkable success in a widespread of applications. SSMs are trained mostly using self-learning approaches to parameterize the significant variabilities of biological shapes, subsequently, the learned shape prior is adopted in image adaption to guide the shape fitting. Despite the success, SSMs-based segmentation approaches suffer from the limitation that the power of SSMs rises and falls with the quality of training data and geometrical complexity of the target shape. Furthermore, the existing image adaption may not be efficient in cases where the target object has a small and distorted structure. Therefore, this thesis aims to de- rive SSMs that are robust to training data corruption and are able to represent complex patterns, and address the problem of the poor image adaption to realize the challenging object segmentation. As training data is often corrupted by many factors like inherent noise/artifacts and non-ideal delineations in this thesis, many efforts have been devoted to developing SSMs that are robust to data corruption. First, early attempts proposing an imputation method and weighted Robust Principal Component Analysis (WRPCA) have been made to ad- dress arbitrary corruptions under the assumption of linear distribution. Nevertheless, deriving a quality model is still demanding as the shape variance of biological structures may not simply follow Gaussian distribution. To combat this, a kernelized RPCA is proposed to cope with outliers in a nonlinear distribution. The idea is performing the low-rank modeling on the kernel matrix to achieve nonlinear dimensionality reduction, and outlier recovery thereof. To increase the generality and feasibility, this thesis, furthermore, presents a general nonlinear data compression technique, the Robust Kernel PCA (RKPCA), with the aim of constructing a low-rank nonlinear subspace free of outliers. In terms of evaluation, the proposed RKPCA delivers high performance on not only creating SSMs but also on outlier recovery. Experiments are conducted using two representative datasets, a set of 30 public CT kidneys and a set of 49 internal MRI ankle bones. Embedded into an existing segmentation framework, experimental results show that SSM built with the proposed RKPCA outperforms the state-of-the-art modeling techniques in terms of model quality and segmentation accuracy. Since SSMs fail to adopt in cases where the target structure occupies a relatively small or distorted area, deep neural networks that remedy this shortcoming are considered thereof. However, redundant background contents in 3D volume may significantly influence the accuracy of deep deep neural networks. Aiming at challenging structures that occupy relatively small areas and have large variances, a novel unified segmentation framework is proposed that incorporates SSM on the top of deep neural network for de- tailed refinement. The motivation is aggregating both spatial and intensity based features from a limited amount of data. Globally optimized via Bayesian inference, the segmentation is driven by a dynamic weighted Gaussian Mixture Model integrating the probability scores from the deep neural network and the shape prior from the SSM. Under a public NIH dataset of CT pancreas, the proposed segmentation framework achieves the best average Dice Similarity Coefficient compared to the-state-of-the-art approaches. The majority of this work is based on public tools: the Medical Imaging Inter- action Toolkit (MITK) for SSMs investigation and analysis and the public library Keras for deep neural networks development. All medical image datasets used in this thesis have been validated by clinical experts.

Show publication details

Self-Service Data Preprocessing and Cohort Analysis for Medical Researchers

2019

2019 IEEE Workshop on Visual Analytics in Healthcare

IEEE Workshop on Visual Analytics in Healthcare (VAHC) <10, 2019, Vancouver, BC, Canada>

Medical researchers are increasingly interested in data-driven approaches to support informed decisions in many medical areas. They collect data about the patients they treat, often creating their own specialized data tables with more characteristics than what is defined in their clinical information system (CIS). Usually, these data tables or sEHR (small electronical health records) are rather small, maybe containing the data of only hundreds of patients. Medical researchers are struggling to find an easy way to first clean and transform these sEHR, and then create cohorts and perform confirmative or exploratory analysis. This paper introduces a methodology and identifies requirements for building systems for self-service data preprocessing and cohort analysis for medical researchers. We also describe a system based on this methodology and the requirements that shows the benefits of our approach. We further highlight these benefits with an example scenario from our projects with clinicians specialized on head&neck cancer treatment.

Show publication details
Wang, Anqi; Franke, Andreas; Wesarg, Stefan

Semi-automatic Segmentation of JIA-induced Inflammation in MRI Images of Ankle Joints

2019

Medical Imaging 2019: Image Processing

SPIE Medical Imaging Symposium <2019, San Diego, CA, USA>

Proceedings of SPIE
10949

The autoimmune disease Juvenile Idiopathic Arthritis (JIA) affects children of under 16 years and leads to the symptom of inflamed synovial membranes in affected joints. In clinical practice, characteristics of these inflamed membranes are used to stage the disease progression and to predict erosive bone damage. Manual outlining of inflammatory regions in each slide of a MRI dataset is still the gold standard for detection and quantification, however, this process is very tiresome and time-consuming. In addition, the inter- and intra-observer variability is a known problem of human annotators. We have developed the first method to detect inflamed regions in and around major joints in the human ankle. First, we use an adapted coupled shape model framework to segment the ankle bones in a MRI dataset. Based on these segmentations, joints are defined as locations where two bones are particularly close to each other. A number of potential inflammation candidates are generated using multi-level thresholding. Since it is known that inflamed synovial membranes occur in the proximity of joints, we filter out structures with similar intensities such as vessels and tendons sheaths using not only a vesselness filter, but also their distance to the joints and their size. The method has been evaluated on a set of 10 manually annotated clinical MRI datasets and achieved the following results: Precision 0.6785 ± 0.1584, Recall 0.5388 ± 0.1213, DICE 0.5696 ± 0.0976.

Show publication details
Ramesh, Supradha; Ben Hmida, Helmi [1. Gutachten]; Kuijper, Arjan [2. Gutachten]

Smart Recommendation for Anomaly Detection in IoT

2019

Darmstadt, TU, Master Thesis, 2019

Anomaly detection is the task of finding instances in a dataset that are different from the normal data. Today, anomaly detection is a core part of many IoT applications and finding abnormal instances is crucial in many applications. For example in network intrusion detection, identification of failures in mechanical systems and in smart sensors and abnormal usage of resources and diagnosis in the medical domain. In all of these applications, the amount of stored data has increased dramatically in the last decade, resulting in a strong demand for algorithms suitable for these crucial challenges. Existing research on anomaly detection has been fragmented across different application domains. Without a good understanding of how different techniques are related to each other and what the strengths and weaknesses of the techniques are, a large number of algorithms are created for the problem of anomaly detection in the field of IoT for specific use cases. This causes scalability issues in reusing the algorithm. In many solutions, the foundation of data are not taken into account, which leads to poor performance of the anomaly detection. The main goal of this thesis is to bridge this gap and to provide an efficient recommendation on anomaly detection algorithm, based on the good understanding of algorithms and characteristics of data. A broad spectrum of anomaly detection has been proposed mainly for semi-supervised and unsupervised anomaly detection. The assumptions, advantages, limitations, and variations are highlighted for algorithms and addressed for local, global and collective anomaly detection problems. The analysis of the characteristics of data is measured extensively, to resonance map the algorithms based on the characteristics and structure of the data. Hence this thesis proposes a formal solution, validated with the working prototype for recommending an anomaly detection algorithms. This solution allows the dynamic inspection of arbitrary IoT data in addition to an interactive environment to acquire domain knowledge when needed.

Show publication details
Alte, Daniel; Kuijper, Arjan [Betreuer]; Mukhopadhyay, Anirban [Betreuer]

Spatial interpretation of DNN-Classifiers in medical imaging

2019

Darmstadt, TU, Master Thesis, 2019

A shortage of radiologists in both industrialized countries and developing countries leads to delayed or absent treatments of diseases. Modern medical artificial intelligence systems can potentially assist radiologists to achieve more efficient detection and diagnosis of diseases. However, such systems rely on deep neural networks, which are mostly black boxes. Especially the clinical sector requires interpretable systems. Recent work introduces tractable uncertainty estimates using dropout (Monte Carlo Dropout). For classification tasks, this could indicate a radiologist, how confident a network is in its prediction. Uncertainty estimates answer the question of if a network "knows", whether it is predicting falsely. A different way of interpreting a Convolutional Neural Network are Class Activation Maps, a visualization technique showing discriminative spatial features for a given classification. This thesis investigates, whether combining Class Activation Maps and Monte Carlo Dropout leads to obtaining spatial uncertainty estimates. A correlation analysis shows that the predictive entropy can separate spatial uncertainties in two clusters. The proposed method is evaluated qualitatively. It is observed that redundant features can cause high spatial uncertainties. Furthermore, we show qualitatively that the proposed method finds spatial uncertainties. Additionally, it has to be noted that our approach treats each pixel of the uncertainty map independently from all other pixels. Future work can investigate the maps with dependent pixels.

Show publication details

Suppressing Gender and Age in Face Templates Using Incremental Variable Elimination

2019

The 12th IAPR International Conference On Biometrics

IAPR International Conference on Biometrics (ICB) <12, 2019, Crete, Greece>

Recent research on soft-biometrics showed that more information than just the person’s identity can be deduced from biometric data. Using face templates only, information about gender, age, ethnicity, health state of the person, and even the sexual orientation can be automatically obtained. Since for most applications these templates are expected to be used for recognition purposes only, this raises major privacy issues. Previous work addressed this problem purely on image level regarding function creep attackers without knowledge about the systems privacy mechanism. In this work, we propose a soft-biometric privacy enhancing approach that reduces a given biometric template by eliminating its most important variables for predicting soft-biometric attributes. Training a decision tree ensemble allows deriving a variable importance measure that is used to incrementally eliminate variables that allow predicting sensitive attributes. Unlike previous work, we consider a scenario of function creep attackers with explicit knowledge about the privacy mechanism and evaluated our approach on a publicly available database. The experiments were conducted to eight baseline solutions. The results showed that in many cases IVE is able to suppress gender and age to a high degree with a negligible loss of the templates recognition ability. Contrary to previous work, which is limited to the suppression of binary (gender) attributes, IVE is able, by design, to suppress binary, categorical, and continuous attributes.

Show publication details
Ströter, Daniel; Stork, André [1. Gutachten]; Mueller-Roemer, Johannes [2. Gutachten]

Tetrahedral Mesh Processing and Data Structures for Adaptive Volumetric Mesh Booleans on GPUs

2019

Darmstadt, TU, Master Thesis, 2019

The virtual prototyping process is time consuming and laborious. Especially the necessity of returning to CAD after CAE analysis imposes development times. In order to allow for immediate editing of FEM meshes, this thesis attempts to devise an efficient concept for adaptive boolean operations on tetrahedral FEM meshes. Due to the impressive aggregated processing power of GPUs, the proposed concept utilizes the GPU. This thesis presents algorithms and data structures amenable to tetrahedral mesh processing on the GPU. It presents an efficient adaptive subdivision refinement algorithm. This thesis also introduces a new spatial data structure for efficient traversal and construction on the GPU. Another contribution is a GPU efficient mesh facet classification procedure, which in conjunction with a GPU parallel mesh composition procedure enables rapid construction of result meshes. Additionally, this thesis presents a parallel tetrahedral mesh optimization procedure for GPUs. The proposed concept allows for basic adaptive boolean operations on tetrahedral meshes accelerated by the GPU. With the contributed tetrahedral mesh processing functionality, development times in the virtual prototyping process reduce by at least a factor of 50x. This is a compelling result for tetrahedral mesh processing on the GPU.

Show publication details
Micallef, Luana; Schulz, Hans-Jörg; Angelini, Marco; Aupetit, M.; Chang, Remco; Kohlhammer, Jörn; Perer, Adam; Santucci, Giuseppe

The Human User in Progressive Visual Analytics

2019

EuroVis 2019. Eurographics / IEEE VGTC Conference on Visualization 2019: Short Papers

Eurographics / IEEE VGTC Conference on Visualization (EuroVis) <21, 2019, Porto, Portugal>

The amount of generated and analyzed data is ever increasing, and processing such large data sets can take too long in situations where time-to-decision or fluid data exploration are critical. Progressive visual analytics (PVA) has recently emerged as a potential solution that allows users to analyze intermediary results during the computation without waiting for the computation to complete. However, there has been limited consideration on how these techniques impact the user. Based on discussions from a Dagstuhl seminar held in October 2018, this paper characterizes PVA users by their common roles, their main tasks, and their distinct focus of analysis. It further discusses cognitive biases that play a particular role in PVA. This work will help PVA visualization designers in devising systems that are tailored for their specific target users and their characteristics.

Show publication details
Chodan, Wencke; Krause, Silvio; Meza-Cuevas, Mario A.; Kadner, Martin; Rockstroh, Jan; König, Carsten; Aehnelt, Mario; Urban, Bodo; Bieber, Gerald

The SEBA System. A novel approach for assessing psychological stress continuously at the workplace

2019

iWOAR 2019

International Workshop on Sensor-based Activity Recognition (iWOAR) <6, 2019, Rostock, Germany>

ACM International Conference Proceedings Series

Stress at work is a major cause of health problems for the employees and of costs for companies and the healthcare system. To prevent stress-related disorders, first both the stress level and the exposition to possible stressors must be known. The SEBA system assesses both and produces live data streams that are constantly and automatically evaluated. The system is a head-worn portable device. Multiple sensors assess biosignals of the users that are known to be sensitive towards the feeling of stress (e.g., pulse, eye blink rate, breathing rate, brain activity) and ambient conditions that could influence the feeling of stress (e.g., air quality, flickering lights, temperature, draft). SEBA classifies the individual stress level using a neural network and sends the processed data to a mobile application for visualization purposes. In this paper, we introduce the concept of the SEBA system, including its hardware, sensors, firmware, and software. The SEBA system is currently being development; the paper outlines the current state of development and possible obstacles.

Show publication details
Bartuzi, Ewelina; Damer, Naser

Thermal and Cross-spectral Palm Image Matching in the Visual Domain by Robust Image Transformation

2019

The 12th IAPR International Conference On Biometrics

IAPR International Conference on Biometrics (ICB) <12, 2019, Crete, Greece>

Synthesizing visual-like images from those captured in the thermal spectrum allows for direct cross-domain comparisons. Moreover, it enables thermal-to-thermal comparisons that take advantage of feature extraction methodologies developed for the visual domain. Hand based biometrics are socially accepted and can operate in a touchless mode. However, certain deployment scenarios requires captures in non-visual spectrums due to impractical illumination requirements. Generating visual-like palm images from thermal ones faces challenges related to the nature of hand biometrics. Such challenges are the dynamic nature of the hand and the difficulties in accurately aligning hand’s scale and rotation, especially in the understudied thermal domain. Building such a synthetic solution is also challenged by the lack of large-scale databases that contain images collected in both spectra, as well as generating images of appropriate resolutions. Driven by these challenges, this paper presents a novel solution to transfer thermal palm images into high-quality visual-like images, regardless of the limited training data, or scale and rotational variations. We proved quality similarity and high correlation of the generated images to the original visual images. We used the synthesized images within verification approaches based on CNN and hand crafted-features. This allowed significantly improved the cross-spectral and thermal-to-thermal verification performances, reducing the EER from 37.12% to 16.25% and from 3.04% to 1.65%, respectively in both cases when using CNN-based features.

Show publication details
Damer, Naser; Saladie, Alexandra Moseguí; Zienert, Steffen; Wainakh, Yaza; Kirchbuchner, Florian; Kuijper, Arjan; Terhörst, Philipp

To Detect or not to Detect: The Right Faces to Morph

2019

The 12th IAPR International Conference On Biometrics

IAPR International Conference on Biometrics (ICB) <12, 2019, Crete, Greece>

Recent works have studied the face morphing attack detection performance generalization over variations in morphing approaches, image re-digitization, and image source variations. However, these works assumed a constant approach for selecting the images to be morphed (pairing) across their training and testing data. A realistic variation in the pairing protocol in the training data can result in challenges and opportunities for a stable attack detector. This work extensively study this issue by building a novel database with three different pairing protocols and two different morphing approaches. We study the detection generalization over these variations for single image and differential attack detection, along with handcrafted and CNNbased features. Our observations included that training an attack detection solution on attacks created from dissimilar face images, in contrary to the common practice, can result in an overall more generalized detection performance. Moreover, we found that differential attack detection is very sensitive to variations in morphing and pairing protocols.

Show publication details
Kubon, Philipp; Kuijper, Arjan [1. Gutachten]; Boutros, Fadi [2. Gutachten]

Ubiquitous Person Detection and Identification in Smart Living Environments

2019

Darmstadt, TU, Bachelor Thesis, 2019

The recent advances in ubiquitous computing and the Internet of Things induce the awareness of smart environments and enhance the interaction between the system and the users. This enables energy savings, improvements in human comfort and assistance, and many other convenience services. However, it requires abilities to detect, count and identify current occupying invidiuals within the smart environment. Person detection and identification with devices like cameras is a well-addressed topic in literature. However, this vision-based sensing is not socially acceptable in a home setting. Person detection based on contact sensors, such as wearable devices, relies too much on correct behavior of its users, and can be regarded as inconvenient especially for older adults, as it requires a constant contact with the users. This works aims at using ambient sensors that can be installed in existing indoor environments to detect and identify individuals in smart environments. Ambient sensors can mitigate disadvantages of other sensing methods: (a) ambient sensors can be seamlessly integrated into homes, (b) they can sense without direct interaction from their users, (c) they are more socially acceptable than video surveillance. These benefits make it realistic to capture ambient sensor information constantly, which could make it possible to detect and identify people with context-aware environments. In order to achieve person detection and identification, three different tasks are investigated: Single Human Occupancy Detection, Multiple Human Occupancy Detection, and Human Identification. This thesis investigates the use of different machine learning methods for aforementioned tasks, including neural networks, SVM, kNN, Discriminant Analysis and CART, trains and evaluates on three different databases of ambient sensor measurements, and compares with the current methods proposed in literature. A bidirectional recurrent model that uses GRU cells is proposed to extract patterns from time series data. On a data set specifically intended for occupancy detection, the state-of-the-art is outperformed with neural network models, achieving up to 99,44% accuracy. Another utilized database is a composition of sensor data collected from 30 different apartments, annotated with daily life activities. These activity annotations are useful enough to gain knowledge for all three detection/identification tasks. While not enough information is provided to fully explore multiple person detection and identification, it is shown that (a) a system can predict whether the environment is not occupied, or that one person, or multiple people are present, and (b) ambient sensor measurement patterns are sufficient to distinguish two similar apartments that have one resident each, so indirectly two persons can be identified.

Show publication details

Underwater Color Restoration Using U-Net Denoising Autoencoder

2019

Proceedings of the 11th International Symposium Image and Signal Processing and Analysis

International Symposium on Image and Signal Processing and Analysis (ISPA) <11, 2019, Dubrovnik, Croatia>

Visual inspection of underwater structures by vehicles, e.g. remotely operated vehicles (ROVs), plays an important role in scientific, military, and commercial sectors. However, the automatic extraction of information using software tools is hindered by the characteristics of water which degrade the quality of captured videos. As a contribution for restoring the color of underwater images, Underwater Denoising Autoencoder (UDAE) model is developed using a denoising autoencoder with U-Net architecture. The proposed network takes into consideration the accuracy and the computation cost to enable realtime implementation on underwater visual tasks using end-toend autoencoder network. Underwater vehicles perception is improved by reconstructing captured frames; hence obtaining better performance in underwater tasks. Related learning methods use generative adversarial networks (GANs) to generate color corrected underwater images, and to our knowledge this paper is the first to deal with a single autoencoder capable of producing same or better results. Moreover, image pairs are constructed for training the proposed network, where it is hard to obtain such dataset from underwater scenery. At the end, the proposed model is compared to a state-of-the-art method.

Show publication details
Hashisho, Yousif; Lukas, Uwe von [Gutachter]; Staadt, Oliver [Gutachter]; Albadawi, Mohamad [Supervising Advisor]; Krause, Tom [Supervising Advisor]

Underwater Image Enhancement Using Autoencoders

2019

Rostock, Univ., Master Thesis, 2019

Visual inspection of underwater structures by vehicles, e.g. remotely operated vehicles (ROVs), plays an important role in scientific, military, and commercial aspects. However, the automatic extraction of information using software tools is hindered by the optical characteristics of water which degrade the quality of the videos. As a contribution for enhancing underwater images, we develop an algorithm using a single denoising autoencoder to restore the color and remove the disturbances such as marine snow from underwater images. Marine snow in some images is only partially removed using the proposed network; however, we show the reason behind this failure. Related learning methods use generative adversarial networks (GANs) for generating color corrected underwater images, and to our knowledge this thesis is the first to deal with a single autoencoder capable of producing same or better results. Moreover, underwater aligned image pairs are established for the training of the proposed network, where it is hard to obtain such dataset from underwater scenery. The objective is to increase the accuracy and reliability on automatic underwater operations that rely on robotic perception without human interference by improving the quality of the captured frames. At the end, the proposed network is evaluated using Mean Squared Error (MSE), Peak Signal-to- Noise Ratio (PSNR), and Structural Similarity Index Metric (SSIM) quality metrics. Additionally, we compare our experiment with a related method. The proposed network takes into consideration the computation cost and the accuracy to have real-time implementation on visual-driven tasks using a single autoencoder.

Show publication details
Bieber, Gerald; Haescher, Marian; Antony, Niklas; Hoepfner, Florian; Krause, Silvio

Unobtrusive Vital Data Recognition by Robots to Enhance Natural Human–Robot Communication

2019

Social Robots: Technological, Societal and Ethical Aspects of Human-Robot Interaction

The ongoing technical improvement of robotic assistants, such as robot vacuum cleaners, telepresence robots, or shopping assistance robots, requires a powerful but unobtrusive form of communication between humans and robots. The capabilities of robots are expanding, which entails a need to improve and increase the perception of all possible communication channels. Therefore, the modalities of text- or speech-based communication have to be extended by body language and direct feedback such as non-verbal communication. In order to identify the feelings or bodily reactions of their interlocutor, we suggest that robots should use unobtrusive vital data assessment to recognize the emotional state of the human. Therefore, we present the concept of vital data recognition through the robot touching and scanning body parts. Thereby, the robot measures tiny movements of the skin, muscles, or veins caused by the pulse and heartbeat. Furthermore, we introduce a camera-based, non-body contact optical heart rate recognition method that can be used in robots in order to identify humans’ reactions during robot-human communication or interaction. For the purpose of heart rate and heart rate variability detection, we have used standard cameras (webcams) that are located inside the robot’s eye. Although camera-based vital sign identification has been discussed in previous research, we noticed that certain limitations with regard to real-world applications still exist. We identified artificial light sources as one of the main influencing factors. Therefore, we propose strategies that aim to improve natural communication between social robots and humans.

Show publication details

Unsupervised Privacy-enhancement of Face Representations Using Similarity-sensitive Noise Transformations

2019

Applied Intelligence

Face images processed by a biometric system are expected to be used for recognition purposes only. However, recent work presented possibilities for automatically deducing additional information about an individual from their face data. By using soft-biometric estimators, information about gender, age, ethnicity, sexual orientation or the health state of a person can be obtained. This raises a major privacy issue. Previous works presented supervised solutions that require large amount of private data in order to suppress a single attribute. In this work, we propose a privacy-preserving solution that does not require these sensitive information and thus, works in an unsupervised manner. Further, our approach offers privacy protection that is not limited to a single known binary attribute or classifier. We do that by proposing similarity-sensitive noise transformations and investigate their effect and the effect of dimensionality reduction methods on the task of privacy preservation. Experiments are done on a publicly available database and contain analyses of the recognition performance, as well as investigations of the estimation performance of the binary attribute of gender and the continuous attribute of age. We further investigated the estimation performance of these attributes when the prior knowledge about the used privacy mechanism is explicitly utilized. The results show that using this information leads to significantly enhancement of the estimation quality. Finally, we proposed a metric to evaluate the trade-off between the privacy gain and the recognition loss for privacy-preservation techniques. Our experiments showed that the proposed cosine-sensitive noise transformation was successful in reducing the possibility of estimating the soft private information in the data, while having significantly smaller effect on the intended recognition performance.

Show publication details
Niemeyer, Frank; Dolereit, Tim; Neumann, Matthias; Albiez, Jan; Vahl, Matthias; Geist, Michael

Untersuchungen von optischen Scansystemen zur geometrischen Erfassung von Unterwasserstrukturen

2019

Hydrographische Nachrichten

Dieser Beitrag beschäftigt sich mit optischen Erfassungs- und Scansystemen für den Bereich unter Wasser. Dabei wurde eine Einteilung in photogrammetrische, trigonometrische und impulsbasierte Systeme vorgenommen. Es wurden Messungen mit verschiedenen Systemen im Schleppkanal des Lehrstuhls für Strömungstechnik der Universität Rostock durchgeführt. Stellvertretend für photogrammetrische Erfassungssysteme kam das Stereokamerasystem vom Fraunhofer-Institut für Graphische Datenverarbeitung zum Einsatz. Das ULS-200-Scansystem von 2GRobotics und das SeaVision- Lasersystem von Kraken Robotik standen stellvertretend für trigonometrische Scansysteme zur Verfügung. Ein impulsbasiertes Scansystem stand für die Untersuchungen nicht bereit. Zusätzlich wurde das SeaVision-Scansystem in der Ostsee nahe des künstlichen Riffs bei Nienhagen unter realistischen Bedingungen getestet. Die Messungen spiegeln das derzeitige Potenzial optischer Messsysteme für den Bereich unter Wasser wider. Vor- und Nachteile der Systeme werden diskutiert.

Show publication details

User Guidance for Interactive Camera Calibration

2019

Virtual, Augmented and Mixed Reality: Multimodal Interaction

International Conference Virtual Augmented and Mixed Reality (VAMR) <11, 2019, Orlando, FL, USA>

Lecture Notes in Computer Science (LNCS)
11574

For building a Augmented Reality (AR) pipeline, the most crucial step is the camera calibration as overall quality heavily depends on it. In turn camera calibration itself is influenced most by the choice of camera-to-pattern poses – yet currently there is only little research on guiding the user to a specific pose. We build upon our novel camera calibration framework that is capable to generate calibration poses in real-time and present a user study evaluating different visualization methods to guide the user to a target pose. Using the presented method even novel users are capable to perform a precise camera calibration in about 2 min.

  • 978-3-030-21606-1
Show publication details
Schader, Philipp; Kuijper, Arjan [1. Prüfer]; Lücke-Tieke, Hendrik [2. Prüfer]

User Guidance in Workflow Modeling Environments

2019

Darmstadt, TU, Master Thesis, 2019

Workflow systems are present in many applications and are used by many different user groups. With ever growing toolboxes available in the workflow editors, the process of se- lecting the right tool for the job becomes more and more difficult. At the same time more specialized workflows can be expressed and the workflow system grows in power. These two sides of the same coin form a dilemma, which this work tries to mitigate by providing recommendations derived from previously created workflows. However, if preexisting work- flows are rare for the system at hand, this leads to a bootstrapping problem. To provide the user with recommendations in case of data sparsity in the user’s system, the recommender may leverage the knowledge from available workflows originated in other systems. With this domain transfer step the bootstrap problem could be mitigated. In this thesis an approach is presented to transfer structural and type information from work- flows between workflow systems using abstract types. A recommender system is described leveraging this abstract type information to generate recommendations in the target work- flow system. The transfer capabilities are evaluated using two major workflow systems for scientific workflows, Taverna and Galaxy. An implementation of the approach is presented integrating it into an existing workflow system to provide recommendations to the user while editing the workflow.

Show publication details
Bernard, Jürgen; Sessler, David; Kohlhammer, Jörn; Ruddle, Roy A.

Using Dashboard Networks to Visualize Multiple Patient Histories: A Design Study on Post-operative Prostate Cancer

2019

IEEE Transactions on Visualization and Computer Graphics

In this design study, we present a visualization technique that segments patients' histories instead of treating them as raw event sequences, aggregates the segments using criteria such as the whole history or treatment combinations, and then visualizes the aggregated segments as static dashboards that are arranged in a dashboard network to show longitudinal changes. The static dashboards were developed in nine iterations, to show 15 important attributes from the patients' histories. The final design was evaluated with five non-experts, five visualization experts and four medical experts, who successfully used it to gain an overview of a 2,000 patient dataset, and to make observations about longitudinal changes and differences between two cohorts. The research represents a step-change in the detail of large-scale data that may be successfully visualized using dashboards, and provides guidance about how the approach may be generalized.

Show publication details

Virtual Reality in Media and Technology

2019

Digital Transformation
Fraunhofer-Forschungsfokus

Virtual and Augmented Reality technologies have by now become established in numerous engineering areas of application. Also in the cultural and media fields interactive three-dimensional content is being increasingly made available for information purposes, and used in scientific research. On the one hand, this development is accelerated by current advances in smartphones, tablets and head-mounted displays. These support complex 3D applications in mobile application scenarios, and enable us to capture our real physical environment using multimodal sensors in order to correlate it with the digital 3D world. On the other hand, new automated digitization technologies such as CultLab3D of the Fraunhofer Institute for Computer Graphics Research IGD allow the production of the necessary digital replicas of real objects, quickly, economically and of high quality.

  • 978-3-662-58133-9
Show publication details
Schulz, Hans-Jörg [Ed.] [et al.]

Vision, Modeling, and Visualization

2019

Vision, Modeling, and Visualization (VMV) <24, 2019, Rostock, Germany>

  • 978-3-03868-098-7
Show publication details
Riaz, Muhammad Ali; Kuijper, Arjan [1. Prüfer]; Burkhardt, Dirk [2. Prüfer]

Visual Trend Analysis on Condensed Expert Data beside Research Library Data for Enhanced Insights

2019

Darmstadt, TU, Master Thesis, 2019

In the present age of information, we live amidst seas of digital text documents including academic publications, white papers, news articles, patents, newspapers. To tackle the issue of the ever-increasing amount of text documents, researchers from the field of text mining and information visualization have developed tools and techniques to facilitate text analysis. In the context of visual trend analysis on text data, the use of well-structured patent data and public digital libraries are quite established. However, both sources of information have their limitations. For instance, the registration process for patents takes at least one year, which makes the extracted insights not suitable to research on present scenarios. In contrast to patent data, the digital libraries are up-to-date but provide high-level insights, only limited to broader research domains, and the data usage is almost restricted on meta information, such as title, author names and abstract, and they do not provide full text. For a certain type of detailed analysis such as competitor analysis or portfolio analysis, data from digital libraries is not enough, it would also make sense to analyze the full-text. Even more, it can be beneficial to analyze only a limited dataset that is filtered by an expert towards a very specific field, such as additive printing or smart wearables for medical observations. Sometimes also a mixture of both digital library data and manually collected documents is relevant to be able to validate a certain trend, where one gives a big picture and other gives a very condensed overview of the present scenario. The thesis aims, therefore, to focus on such manually collected documents by experts that can be defined as condensed data. So, the major goal of this thesis is to conceptualize and implement a solution that enables the creation and analysis of such a condensed data set and compensate therewith the limitations of digital library data analysis. As a result, a visual trend analysis system for analyzing text documents is presented, it utilizes the best of both state-of-the-art text analytics and information visualization techniques. In a nutshell, the presented trend analysis system does two things. Firstly, it is capable of extracting raw data from text documents in the form of unstructured text and meta-data, convert it into structured and analyzable formats, extract trends from it and present it with appropriate visualizations. Secondly, the system is also capable of performing gap-analysis tasks between two data sources, which in this case is digital library data and data from manually collected text documents (Condensed Expert Data). The proposed visual trend analysis system can be used by researchers for analyzing the research trends, organizations to identify current market buzz and industry trends, and many other use-cases where text data is the primary source of valuable information.

Show publication details
Lehmann, Markus; Kuijper, Arjan [1. Gutachten]; Bernard, Jürgen [2. Gutachten]

Visual-Interactive Combination of Selection Strategies to Improve Data Labeling Processes

2019

Darmstadt, TU, Master Thesis, 2019

Labeling training data is an important task in Machine Learning for building effective and efficient classifiers. There are different approaches to gather labeled instances for a particular data set. The two most important fields of strategies are Active Learning and Visual-Interactive Labeling. In previous work, these strategies were examined and compared, resulting in a set of atomic labeling strategies. Additionally, a quasi-optimal strategy was analyzed in order to infer knowledge from its behavior. This investigation resulted in two main insights. First, the labeling process consists of different phases. Second, the performance of a strategy depends on the data set and its characteristics.In this work, we propose a toolkit which enables users to create novel labeling strategies. First, we present multiple visual interfaces users can employ to examine the space of existing algorithms. Then, we introduce a definition of ensembles users can build upon in order to combine existing strategies to novel strategies. Multiple methods to measure the quality of labeling strategies are provided to users, enabling them to examine the gap between their strategies and existing strategies. The different phases of the labeling process are included in the toolkit in order to allow users to always apply the most appropriate strategy in each phase. During the entire process, users are supported by automated guidance in the improvement of their strategies.We evaluate our concept from different perspectives in order to assess its quality. Overall, we observe that our approach enables users to build ensemble strategies which outperform existing strategies. The insights from this work can be applied to develop novel concepts towards ensemble building as well as to improve the generalization of strategies to other data sets.

Show publication details
Reinemuth, Heiko; Kuijper, Arjan [1. Prüfer]; Bernard, Jürgen [2. Prüfer]

Visual-Interactive Labeling of Multivariate Time Series to Support Semi-Supervised Machine Learning

2019

Darmstadt, TU, Master Thesis, 2019

The labeling of multivariate time series is an essential requirement of data-centric decisionmaking processes in many time-oriented application domains. The basic idea of labeling is to assign (semantic) meaning to specific sections or time steps of the time series and to the time series as a whole, accordingly. Hence, weather phenomena can be characterized, EEG signals can be studied, or movement patterns can be marked in sensor data. In the context of this work a visual-interactive labeling tool was developed that allows nonexpert users to assign semantic meaning to any multivariate time series in an effective and efficient way. Enabling experts as well as non-experts to label multivariate time series in a visual-interactive way has never been proposed in the information visualization and visual analytics research communities before. This thesis combines active learning methods, a visual analytics approach, and novel visual-interactive interfaces to achieve an intuitive data exploration and labeling process for users. Visual guidance based on data analysis and model-based predictions empowers users to select and label meaningful instances from the time series. This user-side selection and labeling task can be taken over by an automated model or data-based process. Visual representations of labeling quality and novel interfaces allow for additional user-side refinement.

Show publication details
Bernard, Jürgen; Hutter, Marco; Reinemuth, Heiko; Pfeifer, Hendrik; Bors, Christian; Kohlhammer, Jörn

Visual-Interactive Preprocessing of Multivariate Time Series Data

2019

Computer Graphics Forum

Eurographics Conference on Visualization (EuroVis) <21, 2019, Porto, Portugal>

Pre-processing is a prerequisite to conduct effective and efficient downstream data analysis. Pre-processing pipelines often require multiple routines to address data quality challenges and to bring the data into a usable form. For both the construction and the refinement of pre-processing pipelines, human-in-the-loop approaches are highly beneficial. This particularly applies to multivariate time series, a complex data type with multiple values developing over time. Due to the high specificity of this domain, it has not been subject to in-depth research in visual analytics. We present a visual-interactive approach for preprocessing multivariate time series data with the following aspects. Our approach supports analysts to carry out six core analysis tasks related to pre-processing of multivariate time series. To support these tasks, we identify requirements to baseline toolkits that may help practitioners in their choice. We characterize the space of visualization designs for uncertainty-aware pre-processing and justify our decisions. Two usage scenarios demonstrate applicability of our approach, design choices, and uncertainty visualizations for the six analysis tasks. This work is one step towards strengthening the visual analytics support for data pre-processing in general and for uncertainty-aware pre-processing of multivariate time series in particular.

Show publication details
Reynolds, Steven Lamarr; Kuijper, Arjan [1. Gutachten]; Schufrin, Marija [2. Gutachten]

Visualization Interface to Improve the Transparency of Collected Personal Data on the Internet

2019

Darmstadt, TU, Master Thesis, 2019

In den letzten Jahrzehnten hat das Internet ein enormes Wachstum erlebt. Die Nutzung von Online- Diensten im alltäglichen Leben für allerlei Arten von Aktivitäten steigt. In den meisten Fällen sammeln Unternehmen dabei Daten zur Bereitstellung und Verbesserung ihrer Online-Dienste. Viele Nutzer sind sich jedoch nicht über das Ausmaß und die Art der bei der Nutzung erhobenen und gespeicherten Daten bewusst. Diese Arbeit ist der Fragestellung gewidmet, wie der durchschnittliche Internetnutzer darin unterstützt werden kann, einen Einblick in die von ihm gesammelten Daten zu erhalten und damit einen bewussteren Umgang zu entwickeln. Ziel der dabei entwickelten Anwendung ist es, die Transparenz der gesammelten Daten zu verbessern. Dazu wurde ein Überblick darüber gewonnen, wie Daten aktuell im Internet gesammelt werden. Basierend auf diesem Überblick wurde für diese Arbeit als Datengrundlage die personenbezogenen Daten von Online-Diensten ausgewählt, die Nutzer mit dem Auskunftsrecht der europäischen Datenschutz-Grundverordnung (DSGVO) anfordern können. Darauf aufbauend wurden Konzepte entwickelt, wie diese Daten mit Hilfe von Informationsvisualisierung für den durchschnittlichen Internetnutzer visualisiert werden können. Das am besten geeignete Konzept wurde ausgewählt und prototypisch als Webinterface implementiert. Nutzer können von mehreren ausgewählten Online- Plattformen ihre Daten einfügen und visuell explorieren. Durch die ausgewählte Zusammensetzung von interaktiven Visualisierungsansätzen soll bei der Einschätzung der Menge, der Typen und beim Erkunden von Mustern oder Trends unterstützt werden. Eine zusätzliche Funktion erlaubt es dem Nutzer, die einzelnen Datenelemente nach der wahrgenommenen Sensibilität zu bewerten. Hierdurch soll der Nutzer bewusst dazu angeregt werden, sich mit seinen Daten auseinanderzusetzen. Die implementierte Anwendung wurde im Rahmen dieser Arbeit mit 37 echten Nutzern und deren persönlichen Daten evaluiert. Aus den Ergebnissen lässt sich ableiten, dass die Nutzung der Anwendung einen Einfluss auf die Haltung zur Privatsphäre in Online-Diensten der Teilnehmer hatte und somit einen möglichen Weg in Richtung besseres Bewusstsein für die eigene Privatsphäre im Internet darstellt.

Show publication details

Visualizing Time Series Consistency for Feature Selection

2019

Journal of WSCG Vol. 27, No.1-2, 2019. Proceedings

International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision (WSCG) <27, 2019, Plzen, Czech Republic>

Feature selection is an effective technique to reduce dimensionality, for example when the condition of a system is to be understood from multivariate observations. The selection of variables often involves a priori assumptions about underlying phenomena. To avoid the associated uncertainty, we aim at a selection criterion that only considers the observations. For nominal data, consistency criteria meet this requirement: a variable subset is consistent, if no observations with equal values on the subset have different output values. Such a model-agnostic criterion is also desirable for forecasting. However, consistency has not yet been applied to multivariate time series. In this work, we propose a visual consistency-based technique for analyzing a time series subset’s discriminating ability w.r.t. characteristics of an output variable. An overview visualization conveys the consistency of output progressions associated with comparable observations. Interaction concepts and detail visualizations provide a steering mechanism towards inconsistencies. We demonstrate the technique’s applicability based on two real-world scenarios. The results indicate that the technique is open to any forecasting task that involves multivariate time series, because analysts could assess the combined discriminating ability without any knowledge about underlying phenomena.