Liste der Fachpublikationen

Ergebnisse 1 - 52 von 52
Show publication details
Hegde, Chaitra; Kuijper, Arjan [1. Gutachter]; Dennstädt, Marco [2. Gutachter]

A Distributed Task Scheduler for Cuttlefish::Web to Optimize the Cost and Runtime

2019

Darmstadt, TU, Master Thesis, 2019

The ability to mass produce customized products by additively layering materials has placed 3D printing in the spotlight of the manufacturing industry. Cuttlefish is a 3D printer driver which generates printable files from a 3D mesh. When executed at scale, the driver consumes a considerable amount of computing resources. This highlights the need for a distributed system that is capable of efficiently scaling up or down depending on the type of input while operating under cost and time constraints. Through this master thesis, an intelligent task scheduler which runs print jobs on suitable computers and optimizes cost and runtime based on the user’s preference is implemented. Several Machine Learning algorithms are evaluated to build the system classification and running time prediction models, and the best performing model is deployed as a service. The realized architecture highlights methods to develop an Intelligent task scheduler. They also form a baseline for Cuttlefish::Web to be used on Cloud Infrastructure.

Show publication details
Zhou, Tong-xue; Zeng, Dong-dong; Kuijper, Arjan

A Template Consensus Method for Visual Tracking

2019

Optoelectronics Letters

Visual tracking is a challenging problem in computer vision. Recently, correlation filter-based trackers have shown to provide excellent tracking performance. Inspired by a sample consensus approach proposed for foreground detection, which classifies a given pixel as foreground or background based on its similarity to recently observed samples, we present a template consensus tracker based on the kernelized correlation filter (KCF). Instead of keeping only one target appearance model in the KCF, we make a feature pool to keep several target appearance models in our method and predict the new target position by searching for the location of the maximal value of the response maps. Both quantitative and qualitative evaluations are performed on the CVPR2013 tracking benchmark dataset. The results show that our proposed method improves the original KCF tracker by 8.17% in the success plot and 8.11% in the precision plot.

Show publication details
Han, Xiyu; Lv, Tao; Song, Xiangyu; Nie, Ting; Liang, Huaidan; He, Bin; Kuijper, Arjan

An Adaptive Two-scale Image Fusion of Visible and Infrared Images

2019

IEEE Access

In this paper, we proposed an adaptive two-scale image fusion method using latent low-rank representation (LatLRR). Firstly, both IR and VI images are decomposed into a two-scale representation using LatLRR to generate low-rank parts (the global structure) and saliency parts (the local structure). The algorithm denoises at the same time. Then, the guided filter is used in the saliency parts to make full use of the spatial consistency, which reduces artifacts effectively. With respect to the fusion rule of the low-rank parts, we construct adaptive weights by adopting fusion global-local-topology particle swarm optimization (FGLT-PSO) to obtain more useful information from the source images. Finally, the resulting image is reconstructed by adding the fused low-rank part and the fused saliency part. Experimental results validate that the proposed method outperforms several representative image fusion algorithms on publicly available datasets for infrared and visible image fusion in terms of subjective visual effect and objective assessment.

Show publication details

An Experimental Overview on Electric Field Sensing

2019

Journal of Ambient Intelligence and Humanized Computing

Electric fields exist everywhere. They are influenced by living beings, conductive materials, and other charged entities. Electric field sensing is a passive capacitive measurement technique that detects changes in electric fields and has a very low power consumption. We explore potential applications of this technology and compare it to other measurement approaches, such as active capacitive sensing. Five prototypes have been created that give an overview of the potential use cases and how they compare to other technologies. Our results reveal that electric field sensing can be used for indoor applications as well as outdoor applications. Even a mobile usage is possible due to the low energy consumption of this technology.

Show publication details
Montefiori, Erica; Modenese, Luca; Di Marco, Roberto; Magni-Manzoni, Silvia; Malattia, Clara; Petrarca, Maurizio; Ronchetti, Anna; Tanturri De Horatio, Laura; Dijkhuizen, Pieter van; Wang, Anqi; Wesarg, Stefan; Viceconti, Marco; Mazzà, Claudia

An Image-based Kinematic Model of the Tibiotalar and Subtalar Joints and its Application to Gait Analysis in Children with Juvenile Idiopathic Arthritis

2019

Journal of Biomechanics

In vivo estimates of tibiotalar and the subtalar joint kinematics can unveil unique information about gait biomechanics, especially in the presence of musculoskeletal disorders affecting the foot and ankle complex. Previous literature investigated the ankle kinematics on ex vivo data sets, but little has been reported for natural walking, and even less for pathological and juvenile populations. This paper proposes an MRI-based morphological fitting methodology for the personalised definition of the tibiotalar and the subtalar joint axes during gait, and investigated its application to characterise the ankle kinematics in twenty patients affected by Juvenile Idiopathic Arthritis (JIA). The estimated joint axes were in line with in vivo and ex vivo literature data and joint kinematics variation subsequent to inter-operator variability was in the order of 1°. The model allowed to investigate, for the first time in patients with JIA, the functional response to joint impairment. The joint kinematics highlighted changes over time that were consistent with changes in the patient’s clinical pattern and notably varied from patient to patient. The heterogeneous and patient-specific nature of the effects of JIA was confirmed by the absence of a correlation between a semi-quantitative MRI-based impairment score and a variety of investigated joint kinematics indexes. In conclusion, this study showed the feasibility of using MRI and morphological fitting to identify the tibiotalar and subtalar joint axes in a non-invasive patient-specific manner. The proposed methodology represents an innovative and reliable approach to the analysis of the ankle joint kinematics in pathological juvenile populations.

Show publication details
Lian, Runze; Kuijper, Arjan [Gutacher]; Fu, Biying [Betreuer]

Anomaly Detection and probable path prediction for Single and Multiperson-Application in Smart Homes

2019

Darmstadt, TU, Master Thesis, 2019

The most popular outdoor positioning system, global positioning system (GPS), does not perform well in indoor environment. Because this system primarily depends on the signal propagation in the air and complex architecture of buildings will interfere with signal propagation, i.e., its indoor positioning performance will be limited by the line-of-sight nature. While the drawbacks of GPS, other indoor positioning techniques (such as Wi-Fi based, RFID based) can provide Location-based-service (LBS) for various applications, which make our life comfortable and smart. As one kind of these sensing and positioning techniques, the passive Electric Field Sensing has numerous advantages compared to the others, e.g., lower power consumption and no personal information and specific positioning tokens required. So it is applied in our Smart Floor system to position and track movement for users, which mainly aims at the elderly care. On the other hand, a passive EFS-based positioning system might be susceptible to disturbance, due to the aliasing effect and noises from environment. To address this problem, I studied and investigated the Anomaly Detection issue in the Machine Learning domain, which aims at discovering proper ML algorithms to improve the positioning and movement prediction performance of our Smart Floor system. In this thesis, I proposed a novel ML algorithm for this goal, namely the Dictionary-based Anomaly Detection Algorithm. Compared with other existing algorithms, this dictionary algorithm exploits not only the normal data but also coupling outliers to obtain our desired results, i.e. indoor positions of users. Furthermore, combining with a customized positioning scheme relying on anchor points, the Dictionary-based indoor Positioning and Movement Prediction approach preformed well in our living laboratory. Moreover as discussion and expansion, the Dictionary-based Anomaly Detection Algorithm is especially practicable in application scenarios where a large amount of outliers and normal data are always at the same time observed.

Show publication details

Automatic Detection of the Nasal Cavities and Paranasal Sinuses Using Deep Neural Networks

2019

2019 IEEE International Symposium on Biomedical Imaging

IEEE International Symposium on Biomedical Imaging (ISBI) <16, 2019, Venice, Italy>

The nasal cavity and paranasal sinuses present large interpatient variabilities. Additional circumstances like for example, concha bullosa or nasal septum deviations complicate their segmentation. As in other areas of the body a previous multistructure detection could facilitate the segmentation task. In this paper an approach is proposed to individually detect all sinuses and the nasal cavity. For a better delimitation of their borders the use of an irregular polyhedron is proposed. For an accurate prediction the Darknet-19 deep neural network is used which combined with the You Only Look Once method has shown very promising results in other fields of computer vision. 57 CT scans were available of which 85% were used for training and the remaining 15% for validation.

  • 978-1-5386-3640-4
Show publication details
Sachs, Moritz Karl; Unbescheiden, Matthias [Betreuer]; Buxmann, Peter [Betreuer]

Automatisierte Geschäftsmodellanalysen mit Deep Neural Networks

2019

Darmstadt, TU, Master Thesis, 2019

Ein Hauptkriterium für das Investment eines Venture Capital Fonds in ein Start-up ist dessen Geschäftsmodell. Dieses ist im Businessplan enthalten. Das Screening, sowie die Analyse der eingereichten Businesspläne, erfolgt bei den meisten Venture Capital Fonds überwiegend durch Menschen. Mit der vorliegenden Arbeit wird untersucht, inwieweit die Analyse der in den Businessplänen enthaltenen Geschäftsmodelle mit Hilfe von Deep Neural Networks automatisiert werden kann. Ziel war die Entwicklung eines Prototypen, der die in den Businessplänen enthaltenen Geschäftsmodelle automatisch extrahiert und in das Metamodell Startup Navigator überführt. Dem Knowledge Discovery in Databases Prozess folgend wurden hierfür die Businesspläne eines Venture Capital Fonds aufbereitet und damit ein tiefes Convolutional Neural Network, der Multilabel k-Nearest Neighbour Algorithmus, sowie eine Support Vector Machine mit Naive Bayes Features trainiert. Die Ergebnisse des entwickelten Prototypen zeigen, dass die in den Businessplänen enthaltenen Geschäftsmodelle automatisch extrahiert und in das Metamodell Startup Navigator überführt werden können. Es erscheint plausibel, dass mit mehr Trainingsdaten und einer intensiveren Hyperparameteroptimierung die Korrektklassifizierungsrate verbessert werden kann, sodass der Prototyp zum Aufbau eines Geschäftsmodellkorpus genutzt werden könnte.

Show publication details
Abuladze, David; Kuijper, Arjan [1. Gutachten]; Ben Hmida, Helmi [2. Gutachten]

ClickDigital IDE Enrichment with Internet of Things Rule Widgets

2019

Darmstadt, TU, Bachelor Thesis, 2019

In the recent years, the Internet of Things (IoT) has become an important subject for different services and for the society in general, where different companies use IoT Systems in different ways to improve everyday life quality. Examples of use can be smart adjustment of heating systems, logistics, smart homes, device management for IoT System gateway, device management for connected agricultures, connected cars, and many others. Simplification of the communication process between the end user and devices of the IoT Systems has been a point of concern in many computer science studies. One of them is the master thesis elaborated by Zafar [1]. In his research titled “Enhancing User Experience in the Internet of Things systems regarding Smart Rule Management”, Zafar [1] based his work on several friendly Rule Management interfaces and their arguments pro and contra. After having analyzed and compared relevant studies that have been published at that time, Zafar [1] has suggested a number of user-friendly Rules based widgets within ClickDigital toolkit. ClickDigital toolkit is a dashboard that combines and integrates different IoT System Service interfaces. These widgets aim to manage rules, created by the end users, without any technical background. In my Bachelor Present I will extend the ClickDigital Integrated Development Environment (IDE), which is a web toolkit developed by the Fraunhofer-Institut für Graphische Datenverarbeitung. The present Bachelor Thesis focuses on the ClickDigital IDE enrichment with an Internet of Things Rule widgets. Enrichment of the ClickDigital IDE consists of Rule widgets implementation and validation. The main target includes the extension and the enrichment of the existing ClickDigital widgets repository with a set of Rule Management widgets. The analysis of the previous relevant study by Zafar [1] and other sources has been performed, and several improvements have been implemented. It allows resulting in a better and more efficient communication process between end users and devices within ClickDigital toolkit.

Show publication details
Zeng, Dong-dong; Zhu, Ming; Kuijper, Arjan

Combining Background Subtraction Algorithms with Convolutional Neural Network

2019

Journal of Electronic Imaging

Accurate and fast extraction of foreground objects is a key prerequisite for a wide range of computer vision applications, such as object tracking and recognition. Thus, many background subtraction (BGS) methods for foreground object detection have been proposed in recent decades. However, this is still regarded as a tough problem due to a variety of challenges, such as illumination variations, camera jitter, dynamic backgrounds, and shadows. Currently, there is no single method that can handle all the challenges in a robust way. We try to solve this problem from a perspective of combining different state-of-the-art BGS algorithms to create a more robust and more advanced foreground detection algorithm. More specifically, an encoder–decoder fully convolutional neural network architecture is adapted and trained to automatically learn how to leverage the characteristics of different algorithms to fuse the results produced by different BGS algorithms and produce a more precise result. Comprehensive experiments evaluated on the CDnet 2014 dataset demonstrate that the proposed method outperforms all the considered single BGS algorithms. We show that our solution is more efficient than other BGS combination strategies.

Show publication details
Chaudry, Zaki Ullah; Kuijper, Arjan [Betreuer]; Rus, Silvia [Prüfer]

Designing a General Purpose Smart Textile for a Healthy Lifestyle

2019

Darmstadt, TU, Master Thesis, 2019

We live in a world where wearable technology is fast becoming a part of our daily lives. These wearables vary from a wide range of gadgets such as watches, goggles, shoes, textile etc. One thing all of these wearables have in common is health and fitness monitoring in order to make human life better, which is being achieved by the help of monitoring the activities of the users with the help of various sensors incorporated within the wearable technology. In this thesis the main focus is on creating a smart textile which typically could be placed on different forms of furniture being used in daily life such as bed, chair, couch or table, which than monitors the activities and ergonomics of the person using the furniture. Based on the embedded sensor matrix and advanced algorithms the textile would automatically detect the type of furniture it is placed on. The textile will also constitute of capacitive sensors which, separately or in combination with the accelerometer sensors, detect the user activity (by detecting the breathing rhythm, movement or user interaction). When used in combination with a bed this textile with the help of the sensors will be able to detect the movement, breathing rhythm and user interaction for the user. The same prototype can be used by a person sitting on a chair or couch to detect the breathing, movement and interaction. In case of a table the prototype will be able to measure movement and interaction only. Some smart furniture is already available in the market, but the prices are often to high for a average user to be able to afford them and also the use is limited to just one product. With this textile a user can convert existing dumb furniture into smart furniture, hence providing a wide range of usage with the help of just one textile. In result creating a place-able smart textile technology for a better and healthy lifestyle at an affordable price. This textile minimizes the distance between humans and smart living and fills in the gaps which are beyond the capacity of current wearable technology, while staying cost effective and flexible. Therefore being a great addition to the ecosystem of smart living.

Show publication details

2019

Proceedings of the 12th ACM International Conference on PErvasive Technologies Related to Assistive Environments

ACM International Conference on PErvasive Technologies Related to Assistive Environments (PETRA) <12, 2019, Rhodes, Greece>

Smart textiles and garments promise intriguing new possibilities for the wearer. Integrated interaction can create new experiences and sensors can detect relevant information about the wearer. However, this poses an additional challenge for the designer of smart garments, about how to integrate these technologies. In this work, we want to investigate how human intuition and technical knowledge feed into the design of smart garments. Using a jacket that tracks its whereabouts as a use case, we have collected a dataset from 18 test subjects with varying technical knowledge, on what sensor patterns they would create on the garment. Using a specifically created simulation framework, we have evaluated the performance of the created sensor patterns. We observed that many participants intuitively create well-working patterns, while technical knowledge does not play a significant role in the resulting performance.

  • 978-1-4503-6232-0
Show publication details
Mettel, Matthias Ruben; Alekseew, Michael; Stocklöw, Carsten; Braun, Andreas

Designing and Evaluating Safety Services Using Depth Cameras

2019

Journal of Ambient Intelligence and Humanized Computing

Not receiving help in the case of an emergency is one of the most common fears of older adults that live independently at home. Falls are a particularly frequent occurrence and often the cause of serious injuries. In the last years, various ICT solutions for supporting older adults at home have been developed. Based on sensors and services in a smart environment they provide a wide range of services. In this work we have designed and evaluated safety-related services, based on a single Microsoft Kinect that is installed in a user’s home. We created two services to investigate the benefits and limitations of these solutions. The first is a fall detection service that registers falls in real-time, using a novel combination of static and dynamic skeleton tracking. The second is a fall prevention service that detects potentially dangerous objects in the walking path, based on scene analysis in a depth image. We conducted technical and user evaluations for both services, in order to get feedback on the feasibility, limitations, and potential future improvements.

Show publication details
Helfmann, Stefan; Kuijper, Arjan [1. Gutachten]; Rus, Silvia [2. Gutachten]

Designing Smart Home Controls for Elderly

2019

Darmstadt, TU, Master Thesis, 2019

As technology advances, the idea of a "smart home" gets implemented more and more, and offers multiple ways to help people managing their homes. This offers a big opportunity to help elderly people with their daily lives. However, most feel overwhelmed by modern technology and opt out of using it at all. This thesis provides an alternative take on a Smart Home remote control. After collecting, discussing and evaluating several design principles and guidelines we propose two possible concept remotes and corresponding mock-ups, that are subjected to an evaluation by elderly people to choose the one they prefer. This preferred model will be along with proposed improvements made into a functional prototype and, again, be evaluated by the elderly. The final prototype was received positively and additional improvements are proposed.

Show publication details
Damer, Naser; Boller, Viola; Wainakh, Yaza; Boutros, Fadi; Terhörst, Philipp; Braun, Andreas; Kuijper, Arjan

Detecting Face Morphing Attacks by Analyzing the Directed Distances of Facial Landmarks Shifts

2019

Pattern Recognition

German Conference on Pattern Recognition (GCPR) <40, 2018, Stuttgart, Germany>

Lecture Notes in Computer Science (LNCS)
11269

Face morphing attacks create face images that are verifiable to multiple identities. Associating such images to identity documents lead to building faulty identity links, causing attacks on operations like border crossing. Most of previously proposed morphing attack detection approaches directly classified features extracted from the investigated image. We discuss the operational opportunity of having a live face probe to support the morphing detection decision and propose a detection approach that take advantage of that. Our proposed solution considers the facial landmarks shifting patterns between reference and probe images. This is represented by the directed distances to avoid confusion with shifts caused by other variations. We validated our approach using a publicly available database, built on 549 identities. Our proposed detection concept is tested with three landmark detectors and proved to outperform the baseline concept based on handcrafted and transferable CNN features.

  • 978-3-030-12938-5
Show publication details
Hiemenz, Benedikt; Krämer, Michel

Dynamic Searchable Symmetric Encryption for Storing Geospatial Data in the Cloud

2019

International Journal of Information Security

We present a dynamic searchable symmetric encryption scheme allowing users to securely store geospatial data in the cloud. Geospatial data sets often contain sensitive information, for example, about urban infrastructures. Since clouds are usually provided by third parties, these data need to be protected. Our approach allows users to encrypt their data in the cloud and make them searchable at the same time. It does not require an initialization phase, which enables users to dynamically add new data and remove existing records. We design multiple protocols differing in their level of security and performance, respectively. All of them support queries containing boolean expressions, as well as geospatial queries based on bounding boxes, for example. Our findings indicate that although the search in encrypted data requires more runtime than in unencrypted data, our approach is still suitable for real-world applications.We focus on geospatial data storage, but our approach can also be applied to applications from other areas dealing with keyword-based searches in encrypted data. We conclude the paper with a discussion on the benefits and drawbacks of our approach.

Show publication details

E-Textile Capacitive Electrodes: Fabric or Thread

2019

Proceedings of the 12th ACM International Conference on PErvasive Technologies Related to Assistive Environments

ACM International Conference on PErvasive Technologies Related to Assistive Environments (PETRA) <12, 2019, Rhodes, Greece>

Back pain is one of the most common illnesses in Western civilizations. Office work and lack of motion can lead to deterioration over time. Many people already use seat cushions to improve their posture during work or leisure. In this work, we present an E-Textile cushion. This seat cushion is equipped with capacitive proximity sensors that track the proximity and motion of the sitting user and distinguish up to 7 postures. Giving a user immediate feedback on the posture can facilitate more healthy behavior. We evaluated a number of different electrode setups, materials, and classification methods, leading to a maximum accuracy of 97.1%.

  • 978-1-4503-6232-0
Show publication details
Abrams, Jesse F.; Vashishtha, Anand; Wong, Seth T.; Nguyen, An; Mohamed, Azlan; Wieser, Sebastian; Kuijper, Arjan; Wilting, Andreas; Mukhopadhyay, Anirban

Habitat-Net: Segmentation of Habitat Images Using Deep Learning

2019

Ecological Informatics

Understanding environmental factors that influence forest health, as well as the occurrence and abundance of wildlife, is a central topic in forestry and ecology. However, the manual processing of field habitat data is time-consuming and months are often needed to progress from data collection to data interpretation. To shorten the time to process the data we propose here Habitat-Net: a novel deep learning application based on Convolutional Neural Networks (CNN) to segment habitat images of tropical rainforests. Habitat-Net takes color images as input and after multiple layers of convolution and deconvolution, produces a binary segmentation of the input image. We worked on two different types of habitat datasets that are widely used in ecological studies to characterize the forest conditions: canopy closure and understory vegetation. We trained the model with 800 canopy images and 700 understory images separately and then used 149 canopy and 172 understory images to test the performance of Habitat-Net. We compared the performance of Habitat-Net to the performance of a simple threshold based method, manual processing by a second researcher and a CNN approach called U-Net, upon which Habitat-Net is based. Habitat-Net, U-Net and simple thresholding reduced total processing time to milliseconds per image, compared to 45 s per image for manual processing. However, the higher mean Dice coefficient of Habitat-Net (0.94 for canopy and 0.95 for understory) indicates that accuracy of Habitat-Net is higher than that of both the simple thresholding (0.64, 0.83) and U-Net (0.89, 0.94). Habitat-Net will be of great relevance for ecologists and foresters, who need to monitor changes in their forest structures. The automated workflow not only reduces the time, it also standardizes the analytical pipeline and, thus, reduces the degree of uncertainty that would be introduced by manual processing of images by different people (either over time or between study sites).

Show publication details
Zhou, Wei; Ma, Calwen; Yao, Tong; Chang, Peng; Zhang, Qi; Kuijper, Arjan

Histograms of Gaussian Normal Distribution for 3D Feature Matching in Cluttered Scenes

2019

The Visual Computer

3D feature descriptors provide essential information to find given models in captured scenes. In practical applications, these scenes often contain clutter. This imposes severe challenges on the 3D object recognition leading to feature mismatches between scenes and models. As such errors are not fully addressed by the existing methods, 3D feature matching still remains a largely unsolved problem. We therefore propose our Histograms of Gaussian Normal Distribution (HGND) for capturing salient feature information on a local reference frame (LRF) that enables us to solve this problem. We define a LRF on each local surface patch by using the eigenvectors of the scatter matrix. Different from the traditional local LRF-based methods, our HGND descriptor is based on the combination of geometrical and spatial information without calculating the distribution of every point and its geometrical information in a local domain. This makes it both simple and efficient. We encode the HGND descriptors in a histogram by the geometrical projected distribution of the normal vectors. These vectors are based on the spatial distribution of the points.We use three public benchmarks, the Bologna, the UWA and the Ca’ Foscari Venezia dataset, to evaluate the speed, robustness, and descriptiveness of our approach. Our experiments demonstrate that the HGND is fast and obtains a more reliable matching rate than state-of-the-art approaches in cluttered situations.

Show publication details
Subasioglu, Meltem; Damer, Naser [Supervisor]; Kaschube, Matthias [Supervisor]

Humans vs Machines: A Comparison of Human and Machine Learning Performance in Inferring Professions from Facial Images

2019

Frankfurt am Main, Univ., Master Thesis, 2019

Convolutional neural networks have seen a rise in the computer vision community in the recent years, even surpassing human accuracies in face recognition tasks. However, research regarding different face classification and face clustering experiments, which do not focus on the discrimination of face images according to unique individuals in the dataset, are still very limited. Many psychological studies indicate that humans are capable in inferring leaders, status and competence from facial images. This work focused on the question whether humans are indeed capable in telling profession – given by the branch of the profession and the career status – from face images and whether current state-of-the-art machine learning approaches are able to do the same. Furthermore, the performance of humans and machine learning systems were compared to give insights in the underlying processes. The results indicate that both human and machine learning models are capable to infer professions from facial images with better than chance accuracies. Both humans and machine learning systems perform almost equally well in these tasks, however, performance differences in individual tasks indicate that humans and machine learning algorithms solve the same tasks while relying on different cues in the face images.

Show publication details

Implementing Secure Applications in Smart City Clouds Using Microservices

2019

Future Generation Computer Systems

Smart Cities make use of ICT technology to address the challenges of modern urban management. The cloud provides an efficient and cost-effective platform on which they can manage, store and process data, as well as build applications performing complex computations and analyses. The quickly changing requirements in a Smart City require flexible software architectures that let these applications scale in a distributed environment such as the cloud. Smart Cities have to deal with huge amounts of data including sensitive information about infrastructure and citizens. In order to leverage the benefits of the cloud, in particular in terms of scalability and cost-effectiveness, this data should be stored in a public cloud. However, in such an environment, sensitive data needs to be encrypted to prevent unauthorized access. In this paper, we present a software architecture design that can be used as a template for the implementation of Smart City applications. The design is based on the microservice architectural style, which provides properties that help make Smart City applications scalable and flexible. In addition, we present a hybrid approach to securing sensitive data in the cloud. Our architecture design combines a public cloud with a trusted private environment. To store data in a cost-effective manner in the public cloud, we encrypt metadata items with CP-ABE (Ciphertext-Policy Attribute-Based Encryption) and actual Smart City data with symmetric encryption. This approach allows data to be shared across multiple administrations and makes efficient use of cloud resources. We show the applicability of our design by implementing a web-based application for urban risk management. We evaluate our architecture based on qualitative criteria, benchmark the performance of our security approach, and discuss it regarding honest-but-curious cloud providers as well as attackers trying to access user data through eavesdropping. Our findings indicate that the microservice architectural style fits the requirements of scalable Smart City applications while the proposed security approach helps prevent unauthorized access.

Show publication details

Integrating Server-based Simulations into Web-based Geo-applications

2019

Eurographics 2019. Short Papers

Annual Conference of the European Association for Computer Graphics (Eurographics) <40, 2019, Genoa, Italy>

In this work, we present a novel approach for combining fluid simulations running on a GPU server with terrain rendered by a web-based 3D GIS system. We introduce a hybrid rendering approach, combining server-side and client-side rendering, to interactively display the results of a shallow water simulation on client devices using web technology. To display water and terrain in unison, we utilize image merging based on depth values.We extend it to deal with numerical and compression artifacts as well as Level-of-detail rendering and use Depth Image Based Rendering to counteract network latency.

Show publication details
Matthiesen, Moritz; Rojtberg, Pavel [1. Gutachten]; Kuijper, Arjan [2. Gutachten]

Interpolation von Kalibrier daten für Zoom und Autofokus Kameras

2019

Darmstadt, TU, Bachelor Thesis, 2019

In dieser Arbeit wird das Problem betrachtet, dass für jede neue Kameraeinstellung eine neue Kalibrierung vorgenommen werden muss.Ziel dabei ist Kalibrierdaten an bestimmten Kameraeinstellungen zu erstellen, um mithilfe vondiesen die Kalibrierdaten von anderen Kameraeinstellungen herzuleiten. Dabei werden die Kalibrierdaten betrachtet und es wird versucht Beziehungen zwischen den einzelnen Parametern der Kalibrierung herzuleiten. Um diese zu ermitteln wird zwischen verschiedenen Parametern der Kalibrierung interpoliert.

Show publication details
Distergoft, Alexander; Kuijper, Arjan [1. Gutachten]; Mukhopadhyay, Anirban [2. Gutachten]

Interpreting Adversarial Examples in Medical Imaging

2019

Darmstadt, TU, Master Thesis, 2019

Deep neural networks (DNNs) have been achieving high accuracy on many important tasks like image classification, detection or segmentation. Yet, recent discoveries have shown a high degree of susceptibility for these deep-learning algorithms under attack. DNNs seem to be vulnerable to small amounts of non-random noise, created by perturbing the input to output mapping of the network. These perturbations can severely affect the performance of DNNs and thus endanger systems where such models are employed. The purpose of this thesis is to examine adversarial examples in clinical settings, be it digitally created or physical ones. For this reason we studied the performance of DNNs under the following three attack scenarios: 1. We hypothesize that adversarial examples might occur from incorrect mapping of the image space to the lower dimensional generation manifold. The hypothesis is tested by creating a proxy task of a pose estimation of surgical tools in its simplest form. For this we define a clear decision boundary. We use exhaustive search on a synthetic toy dataset to localize possible resions of successful one-pixel-attacks in image space. 2. We design a small scale prospective evaluation on how Deep-learning (DL) dermoscopy systems perform under physical world attacks. The publically available Physical Attacks on Dermoscopy Dataset (PADv1) is used for this evaluation. The introduced susceptibility and robustness values reveal that such attacks lead to accuracy loss across popular state-of-the-art DL-architectures. 3. As a pilot study to understand the vulnerabilities of DNNs that perform under regression tasks we design a set of auxiliary tasks that are used to create adversarial examples for non-classification-models. We train auxiliary networks on augmented datasets to satisfy the defined auxiliary tasks and create adversarial examples that might influence the decision of a regression model without knowing about the underlying system or hyperparameters.

Show publication details
Boller, Viola; Pesavento, Marius [1. Gutachten]; Damer, Naser [2. Gutachten]

Investigating the Use of Deeply Calculated Flows and Dynamic Routed Network Capsules in Face Presentation Attack Detection

2019

Darmstadt, TU, Master Thesis, 2019

Identifying an individual based on their facial characteristics is convenient and is gaining more and more importance. Only a normal camera is needed for data capturing, the individual characteristic can’t be stolen or forgot like a password, and it’s a non-intrusive capturing method, which enables identification from distances. With increasing usage of face recognition systems, also the amount of sophisticated attacks increase. Therefore vulnerabilities of such a recognition system are used. Especially the attack directly at the sensor, by presenting fake biometrics at the sensor, can be performed with low efforts and costs, and without detailed knowledge about the biometric system. That’s why this kind of attacks have caused wide attention in the biometric community. The attack is called presentation attack and there are many detection algorithms developed to solve this issue. Most existing presentation attack detection algorithms work great, when deployed in the same environment as they are trained in. But different environmental conditions are still challenging to solve. This thesis deals with the presentation attack issue by developing an approach, which combines deep optical flows and a capsule neural network. Optical flows have already proven to be a good preprocessing step, because only motion information is kept and environmental conditions won’t have a big impact on it. Deep optical flows are produced by end-to-end trained convolutional neural networks for the optical flow calculation. Capsule neural networks train neurons in a more general way by forwarding information of found pattern’s poses in the network additionally to it’s probabilities. Forwarding is achieved by a so called routing-by-agreement method, which allows to take relationships between the found patterns into account. For presentation attack detection development not enough data is available to train a deep neural network from scratch. This could be overcome by using a capsule neural network for classification. A deep understanding can be learned, needing less data. The contribution of this thesis is the combination of deep optical flows and a capsule neural network, which hasn’t been tested so far. This is realized by stacking multiple optical flows over the time and downsample this information, thus that the capsule neural network is able to train a classification. This pipeline is structurally optimized by testing different optical flow sizes, data normalization and multiple class training. Also a new downsampling and time stacking method is developed to keep outliers in the reduced data and order a stack according to the contained data information. This has proven to help the network to classify. In the end, the behavior of the network on various attacks is analyzed. In this thesis, performance improvements are achieved by structure optimizations and a novel downsampling and time stacking method. The structural improvements increased the model performance up to 12% and the new downsampling and time stacking method leads to an average performance improvement of 7.56% for two databases. However, the achieved performance does not outperform recent state-of-the-art presentation attack detection methods and the analysis over different attacks reveals that the developed structure adapts weakly to changes in the attack performance or to new attacks. Nevertheless, stable results for different environmental conditions are achieved.

Show publication details
Fellner, Dieter W. [Hrsg.]; Welling, Daniela [Red.]; Ackeren, Janine van [Red.]; Bergstedt, Bettina [Red.]; Krüger, Kathrin [Red.]; Prasche, Svenja [Betreuer]

Jahresbericht 2018

2019

Das Fraunhofer IGD hat seine Forschungsaktivitäten vor Kurzem in vier Leitthemen gebündelt, welche die Basis seiner Arbeit bilden und verschiedene Themen abteilungsübergreifend miteinander verknüpfen. Eines dieser Leitthemen ist "Visual Computing as a Service - Die Plattform für angewandtes Visual Computing". Die Basis dieser universellen Plattform für Visual-Computing Lösungen ist gelegt und wird kontinuierlich erweitert. Dieser technologische Ansatz bildet die Grundlage für die weiteren Leitthemen. In der "Individuellen Gesundheit - Digitale Lösungen für das Gesundheitswesen" werden die Daten betrachtet, die in der personalisierten Medizin anfallen - mithilfe der Visual-Computing-Technologien des Instituts. Im Leitthema "Intelligente Stadt - Innovativ, digital und nachhaltig" ist die Fragestellung, wie man den Lebenszyklus urbaner Prozesse unterstützen kann. Und im Leitthema "Digitalisierte Arbeit - Der Mensch in der Industrie 4.0" geht es erster Linie um die Unterstützung des Menschen in der durch die Digitalisierung veränderten Produktion.

Show publication details
Antony, Niklas; Urban, Bodo [Gutachter]; Bieber, Gerald [Betreuer]

Konzeption und Entwicklung einer Kommunikationsstruktur für autonom-mobile Assistenzsysteme

2019

Rostock, Univ., Bachelor Thesis, 2019

In Deutschland herrscht ein Mangel an Arbeitskräften in Krankenhäusern und Pflegeheimen, welcher durch die steigende Lebenserwartung stetig zunimmt. Mithilfe von autonom-mobilen Robotern lassen sich Pflegekräfte bei ihrer Arbeit entlasten. Lokalisierung und Zielfindung sind die wesentlichen Funktionalitäten dieser Roboter. In der vorliegenden Arbeit wird ein Konzept für ein autonom-mobiles Assistenzsystem mit darauf aufbauendem Prototypen vorgestellt, das QR-Codes als Orientierungspunkte verwendet. Die Einsatzumgebung beschränkt sich auf Innenräume. Hierbei ist ein Smartphone über Bluetooth mit dem mobilen Roboter verbunden, das mit der Rück-Kamera an die Raumdecke gerichtet ist und die dort befestigten QR-Codes ausliest. Die QR-Codes speichern ausreichend Informationen über ihre Umgebung, sodass es dem Roboter möglich ist, ausschließlich mithilfe dieser Informationen und ohne Suchalgorithmus einen zugewiesenen Zielpunkt zu erreichen. Um Kollisionen zu vermeiden, ist der Roboter zusätzlich mit einem Ultraschallsensor ausgestattet. Die Ergebnisse aus Versuchen mit dem Prototypen werden vorgestellt sowie Möglichkeiten der Weiterentwicklung erwähnt.

Show publication details
Zielinski, Michael; Urban, Bodo [Gutachter]; Nonnemann, Lars [Betreuer]

Koordiniertes Workflow Management mit Hilfe von Brushing & Linking Mechanismen

2019

Rostock, Univ., Bachelor Thesis, 2019

Die Art und Weise wie Datenmengen untersucht werden, erlebte in den letzten Jahren einige Veränderungen. Da die Menge der Daten mit der Zeit immer weiter anwächst, müssen verschiedene Analysewerkzeuge zur Untersuchung der Daten verwendet werden. Durch die gleichzeitige Nutzung mehrerer Werkzeuge entsteht jedoch die Problematik, dass unterschiedliche Aspekte wie beispielsweise die Position oder die Skalierung der Werkzeugansichten koordiniert werden müssen. Die Koordination dieser Werkzeuge reicht dabei vom sequentiellen Ausführen der Werkzeuge bis zur Nutzung von anwendungsspezifischen Analysesystemen. Weiterhin haben sich oftmals Workflows zur Analyse der Datenmengen etabliert. Im Kontext eines Workflows werden die Analysewerkzeuge in vielen Fällen jedoch nicht ausreichend miteinander koordiniert, sodass die Datenübergabe oder das Starten und Beenden von Analysewerkzeugen manuell durchgeführt werden muss. Dies führt zu erhöhtem Arbeitsaufwand auf Seiten der Datenanalytiker. Ziel dieser Arbeit ist es, basierend auf dem aktuellen Stand der Technik Konzepte zur workflowbasierten Datenanalyse zu entwickeln, um die Koordination unterschiedlicher Werkzeuge in analytischen Prozessen zu unterstützen. Dabei wird die Aufteilung des Informationsraumes auf Daten- und Visualisierungsebene untersucht. Weiterhin nutzt der entwickelte Ansatz Brushing & Linking zur Verknüpfung mehrerer Ansichten durch anwendungsübergreifende Interaktion. Die konzeptionell entwickelten Ansätze werden prototypisch in das vom Fraunhofer IGD entwickelte Health@Hand-System eingebunden. Dieses System ist ein virtueller Leitstand zur Visualisierung von Vitaldaten, damit beispielsweise Anomalien festgestellt werden können, sodass bei Störfällen frühzeitig interveniert wird. Um repetitive Arbeitsprozesse automatisieren zu können, wird der workflow-basierte Ansatz dieser Arbeit in der Implementierung in Health@Hand verwendet. Dabei wird ein gegebenes Szenario zur Detektion von kardiovaskulären Symptomen und Anomalien umgesetzt. Diese Arbeit gliedert sich in sieben Kapitel. Im nachfolgenden zweiten Kapitel werden grundlegende Techniken zum Aufteilen und Navigieren von Ansichten im Darstellungs- sowie Informationsraum vorgestellt. Darauf folgt in Kapitel 3 die Beschreibung von aktuellen Systemen, welche jeweils unterschiedliche Probleme der Ansichts- und Werkzeugkoordination lösen. Anschließend wird in Kapitel 4 eine Problem- und Anforderungsanalyse zur workflow-basierten Datenanalyse durchgeführt. Auf Basis dessen wird in den Kapiteln 5 und 6 das entwickelte Konzept eines Systems zur workflow-basierten Datenanalyse beschrieben und die prototypische Implementierung dessen in Health@Hand vorgestellt. Die Arbeit schließt schlussendlich mit einem Fazit der entwickelten Ansätze sowie einem Ausblick der noch auszuführenden Arbeiten ab.

Show publication details
Kaymak, Yasin; Kuijper, Arjan [Betreuer]; Ben Hmida, Helmi [Betreuer]

Künstliche Intelligenz Widget für ClickDigital

2019

Darmstadt, TU, Bachelor Thesis, 2019

Die in den letzten Jahren vorangetriebene Umsetzung der Vision Internet der Dinge(IoT) hat auch zu einem massiven Zuwachs von IoT-Geräten gesorgt. Dadurch hat im IoT-Umfeld der Aspekt Sicherheit immer mehr an Bedeutung gewonnen. Das Gewährleisten einer reibungslosen und sicheren Nutzung der IoT-Geräte würde auch das Fortsetzten der Vision IoT unterstützen. Eine Gegenmaßnahme Sicherheit zu gewährleisten wären Anomalie-Erkennungsalgorithmen. Diese Algorithmen sind in der Lage, die von den IoT-Geräten erzeugten Unmengen von Daten (Big Data) nach Fehlverhalten zu analysieren, um überhaupt die Möglichkeit anzubieten, Gegenmaßnahmen zu treffen. In dieser Arbeit wird ein Algorithmus vorgestellt, der die von IoT-Geräten erzeugten Daten nach Fehlverhalten untersuchen kann. Zudem wird dieser Algorithmus Teil der Entwicklungsumgebung ClickDigital.

Show publication details
Schulz, Hans-Jörg; Röhlig, Martin; Nonnemann, Lars; Aehnelt, Mario; Diener, Holger; Urban, Bodo; Schumann, Heidrun

Lightweight Coordination of Multiple Independent Visual Analytics Tools

2019

IVAPP 2019. Proceedings

International Conference on Information Visualization Theory and Applications (IVAPP) <10, 2019, Prague, Czech Republic>

With the advancement of Visual Analytics (VA) and its spread into various application fields comes along a specialization of methods and tools. This adds complexity and requires extra effort when devising domain-dependent VA solutions, as for every new domain question a new specialized tool or framework must be developed. In this paper, we investigate the possibility of using and re-using existing tools – domain-dependent and general-purpose – by loosely coupling them into specialized VA tool ensembles as needed. We call such coupling among independent tools lightweight coordination, as it is minimally-invasive, pair-wise, and opportunistic in utilizing whichever interface a VA tool offers. We propose the use of lightweight coordination for managing the workflow, the data flow, and the control flow among VA tools, and we show how it can be supported with suitable setups of the multiple tool UIs involved. This concept of lightweight coordination is exemplified with a health care scenario, where an ensemble of independent VA tools is used in a concerted way to pursue the visual analysis of a patient’s troublesome vital data.

  • 978-989-758-354-4
Show publication details
Montefiori, Erica; Modenese, Luca; Di Marco, Roberto; Magni-Manzoni, Silvia; Malattia, Clara; Petrarca, Maurizio; Ronchetti, Anna; Tanturri De Horatio, Laura; Dijkhuizen, Pieter van; Wang, Anqi; Wesarg, Stefan; Viceconti, Marco; Mazzà, Claudia

Linking Joint Impairment and Gait Biomechanics in Patients with Juvenile Idiopathic Arthritis

2019

Annals of Biomedical Engineering

Juvenile Idiopathic Arthritis (JIA) is a paediatric musculoskeletal disease of unknown aetiology, leading to walking alterations when the lower-limb joints are involved. Diagnosis of JIA is mostly clinical. Imaging can quantify impairments associated to inflammation and joint damage. However, treatment planning could be better supported using dynamic information, such as joint contact forces (JCFs). To this purpose, we used a musculoskeletal model to predict JCFs and investigate how JCFs varied as a result of joint impairment in eighteen children with JIA. Gait analysis data and magnetic resonance images (MRI) were used to develop patient-specific lower-limb musculoskeletal models, which were evaluated for operator-dependent variability (< 3.6°, 0.05 N kg21 and 0.5 BW for joint angles, moments, and JCFs, respectively). Gait alterations and JCF patterns showed high between-subjects variability reflecting the pathology heterogeneity in the cohort. Higher joint impairment, assessed with MRI-based evaluation, was weakly associated to overall joint overloading. A stronger correlation was observed between impairment of one limb and overload of the contralateral limb, suggesting risky compensatory strategies being adopted, especially at the knee level. This suggests that knee overloading during gait might be a good predictor of disease progression and gait biomechanics should be used to inform treatment planning.

Show publication details
Pandikow, Lars; Kuijper, Arjan [1. Gutachten]; Gutbell, Ralf [2. Gutachten]

Localization and Mapping of Monocular Cameras in Large Urban Environments using Virtual City Models

2019

Darmstadt, TU, Master Thesis, 2019

In recent years there has been a lot of progress on the task of simultaneous localization and mapping (SLAM) of image sequences from monocular cameras. Latest methods utilize advancements in machine learning to improve the quality of both the camera tracking as well as the reconstruction of the environment. One of these methods called CNN-SLAM uses a neural network to estimate depth maps for keyframes and fuses them with stereo observations from neighboring image frames. To make use of the images within a geo-referenced context they first have to be localized globally. The use of additional sensors to determine the position and orientation of the camera is not only more expensive, but also sensitive to errors. This thesis proposes a real-time system that combines the methods of CNN-SLAM with image based localization within a simple city model. The SLAM algorithm tracks the camera movement while the use of a depth estimation network enables the recovery of the scale of the scene. A genetic algorithm is implemented to quickly refine estimated camera poses by aligning synthetic views of the city model with semantic segmentations of the images. This does not only localize the camera trajectories but also helps to compensate tracking errors caused by the SLAM algorithm. The evaluation showed the systems ability to compute scaled trajectories correctly, to compensate tracking drift and densly reconstruct the scene in the vicinity of the camera. It also revealed the unreliability of image localization without a constraint search space, tracking drift during rotational movement and inaccurate semantic segmentations.

Show publication details

Minutiae-Based Gender Estimation for Full and Partial Fingerprints of Arbitrary Size and Shape

2019

Computer Vision - ACCV 2018

Asian Conference on Computer Vision (ACCV) <14, 2018, Perth, Australia>

Lecture Notes in Computer Science (LNCS)
11361

Since fingerprints are one of the most widely deployed biometrics, accurate fingerprint gender estimation can positively affect several applications. For example, in criminal investigations, gender classification may significantly minimize the list of potential subjects. Previous work mainly offered solutions for the task of gender classification based on complete fingerprints. However, partial fingerprint captures are frequently occurring in many applications, including forensics and the fast growing field of consumer electronics. Due to its huge variability in size and shape, gender estimation on partial fingerprints is a challenging problem. Therefore, in this work we propose a flexible gender estimation scheme by building a gender classifier based on an ensemble of minutiae. The outputs of the single minutia gender predictions are combined by a novel adjusted score fusion approach to obtain an enhanced gender decision. Unlike classical solutions this allows to deal with unconstrained fingerprint parts of arbitrary size and shape. We performed investigations on a publicly available database and our proposed solution proved to significantly outperform state-of-the-art approaches on both full and partial fingerprints. The experiments indicate a reduction in the gender estimation error by 19.34% on full fingerprints and 28.33% on partial captures in comparison to previous work.

  • 978-3-030-20886-8
Show publication details

Ontology in Holonic Cooperative Manufacturing: A Solution to Share and Exchange the Knowledge

2019

Knowledge Discovery, Knowledge Engineering and Knowledge Management

International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management (IC3K) <9, 2017, Funchal, Madeira, Portugal>

Communications in Computer and Information Science (CCIS)
976

Cooperative manufacturing is a new trend in industry, which depends on the existence of a collaborative robot. A collaborative robot is usually a light-weight robot which is capable of operating safely with a human co-worker in a shared work environment. During this cooperation, a vast amount of information is exchanged between the collaborative robot and the worker. This information constructs the cooperative manufacturing knowledge, which describes the production components and environment. In this research, we propose a holonic control solution, which uses the ontology concept to represent the cooperative manufacturing knowledge. The holonic control solution is implemented as an autonomous multi-agent system that exchanges the manufacturing knowledge based on an ontology model. Ultimately, the research illustrates and implements the proposed solution over a cooperative assembly scenario, which involves two workers and one collaborative robot, whom cooperate together to assemble a customized product.

  • 978-3-030-15639-8
Show publication details
Wientapper, Folker; Kuijper, Arjan [Referent]; Fellner, Dieter W. [Referent]; Stricker, Didier [Betreuer]

Optimal Spatial Registration of SLAM for Augmented Reality

2019

Darmstadt, TU., Diss., 2019

Augmented reality (AR) is a paradigm that aims at fusing the perceived real environment of a human with digital information located in 3D space. Typically, virtual 3D graphics are overlayed into the captured images of a moving camera or directly into the user's field-of-view by means of optical see-through displays (OST). For a correct perspective and view-dependent alignment of the visualization, it is required to solve various static and dynamic geometric registration problems in order to create the impression that the virtual and the real world are seamlessly interconnected.The advances during the last decade in the field of simultaneous localization and mapping (SLAM) represent an important contribution to this general problem. It is now possible to reconstruct the real environment and to simultaneously capture the dynamic movements of a camera from the images without having to instrument the environment in advance. However, SLAM in general can only partly solve the entire registration problem, because the retrieved 3D scene geometry and the calculated motion path are spatially related only with regard to an arbitrarily selected coordinate system. Without a proper reconciliation of coordinate systems (spatial registration), the real world of the human observer still remains decoupled from the virtual world. Existing approaches for solving this problem either require the availability of a virtual 3D model that represents a real object with sufficient accuracy (model-based tracking), or they rely on use-case specific assumptions and additional sensor data (such as GPS signals or the Manhattan-world assumption). Therefore, these approaches are bound to these additional prerequisites, which limit the general applicability. The circumstance that automated registration is desirable but not always possible, creates the need for techniques that allow a user to specify connections between the real and the virtual world when setting up AR applications, so that it becomes possible to support and control the process of registration. These techniques must be complemented with numerical algorithms that optimally exploit the provided information to obtain precise registration results.Within this context, the present thesis provides the following contributions.* We propose a novel, closed-form (non-iterative) algorithm for calculating a Euclidean or a similarity transformation. The presented algorithm is a generalization of recent state-of-the-art solvers for computing the camera pose based on 2D measurement points in the image (perspective-n-point problem) - a fundamental problem in computer vision that has attracted research for many decades. The generalization consists in extending and unifying these algorithms, so that they can handle other types of input correspondences than originally designed for. With this algorithm, it becomes possible to perform a rigid registration of SLAM systems to a target coordinate system based on heterogeneous and partially indeterminate input data.* We address the global refinement of structure and motion parameters by means of iterative sparse minimization (bundle adjustment or BA), which has become a standard technique inside SLAM systems. We propose a variant of BA in which information about the virtual domain is integrated as constraints by means of an optimization-on-manifold approach. This serves for compensating low-frequency deformations (non-rigid registration) of the estimated camera path and the reconstructed scene geometry caused by measurement error accumulation and the ill-conditionedness of the BA problem.* We present two approaches in which a user can contribute with his knowledge for registering a SLAM system. In a first variant, the user can place markers in the real environment with predefined connections to the virtual coordinate system. Precise positioning of the markers is not required, rather they can be placed arbitrarily on surfaces or along edges, which notably facilitates the preparative effort. During run-time, the dispersed information is collected and registration is accomplished automatically. In a second variant, the user is given the possibility to mark salient points in an image sequence during a preparative preprocessing step and to assign corresponding points in the virtual 3D space via a simple point-and-click metaphor. The result of this preparative phase is a precisely registered and ready-to-use reference model for camera tracking at run-time.* Finally, we propose an approach for geometric calibration of optical see-trough displays. We present a parametric model, which allows to dynamically adapt the rendering of virtual 3D content to the current viewpoint of the human observer, including a pre-correction of image aberrations caused by the optics or irregularly curved combiners. In order to retrieve its parameters, we propose a camera-based approach, in which elements of the real and the virtual domain are simultaneously observed. The calibration procedure was developed for a head-up display in a vehicle. A prototypical extension to head-mounted displays is also presented.

Show publication details
Getto, Roman; Fellner, Dieter W. [Betreuer]; Schreck, Tobias [Betreuer]

Parametric Procedural Models for 3D Object Retrieval, Classification and Parameterization

2019

Darmstadt, TU., Diss., 2019

The amount of 3D objects has grown over the last decades, but we can expect that it will grow much further in the future. 3D objects are also becoming more and more accessible to non-expert users. The growing amount of available 3D data is welcome for everyone working with this type of data, as the creation and acquisition of many 3D objects is still costly. However, the vast majority of available 3D objects are only present as pure polygon meshes. We arguably can not assume to get meta-data and additional semantics delivered together with 3D objects stemming from non-expert or 3D scans of real objects from automatic systems. For this reason content-based retrieval and classification techniques for 3D objects has been developed.Many systems are based on the completely unsupervised case. However, previous work has shown that there are strong possibilities of highly increasing the performance of these tasks by using any type of previous knowledge. In this thesis I use procedural models as previous knowledge. Procedural models describe the construction process of a 3D object instead of explicitly describing the components of the surface. These models can include parameters into the construction process to generate variations of the resulting 3D object. Procedural representations are present in many domains, as these implicit representations are vastly superior to any explicit representation in terms of content generation, flexibility and reusability. Therefore, using a procedural representation always has the potential of outclassing other approaches in many aspects. The usage of procedural models in 3D object retrieval and classification is not highly researched as this powerful representation can be arbitrary complex to create and handle. In the 3D object domain, procedural models are mostly used for highly regularized structures like buildings and trees.However, Procedural models can deeply improve 3D object retrieval and classification, as this representation is able to offer a persistent and reusable full description of a type of object. This description can be used for queries and class definitions without any additional data. Furthermore, the initial classification can be improved further by using a procedural model: A procedural model allows to completely parameterize an unknown object and further identify characteristics of different class members. The only drawback is that the manual design and creation of specialized procedural models itself is very costly. In this thesis I concentrate on the generalization and automation of procedural models for the application in 3D object retrieval and 3D object classification.For the generalization and automation of procedural models I propose to offer different levels of interaction for a user to fulfill the possible needs of control and automation. This thesis presents new approaches for different levels of automation: the automatic generation of procedural models from a single exemplary 3D object. The semi-automatic creation of a procedural model with a sketch-based modeling tool. And the manual definition a procedural model with restricted variation space. The second important step is the insertion of parameters into the procedural model, to define the variations of the resulting 3D object. For this step I also propose several possibilities for the optimal level of control and automation: An automatic parameter detection technique. A semi-automatic deformation based insertion. And an interface for manually inserting parameters by choosing one of the offered insertion principles. It is also possible to manually insert parameters into the procedures if the user needs the full control on the lowest level.To enable the usage of procedural models directly for 3D object retrieval and classification techniques I propose descriptor-based and deep learning based approaches. Descriptors measure the difference of 3D objects. By using descriptors as comparison algorithm, we can define the distance between procedural models and other objects and order these by similarity. The procedural models are sampled and compared to retrieve an optimal object retrieval list. We can also directly use procedural models as data basis for a retraining of a convolutional neural network. By deep learning a set of procedural models we can directly classify new unknown objects without any further large learning database. Additionally, I propose a new multi-layered parameter estimation approach using three different comparison measures to parameterize an unknown object. Hence, an unknown object is not only classified with a procedural model but the approach is also able to gather new information about the characteristics of the object by using the procedural model for the parameterization of the unknown object.As a result, the combination of procedural models with the tasks of 3D object retrieval and classification lead to a meta concept of a holistically seamless system of defining, generating, comparing, identifying, retrieving, recombining, editing and reusing 3D objects.

Show publication details

Performing Indoor Localization with Electric Potential Sensing

2019

Journal of Ambient Intelligence and Humanized Computing

Location-based services or smart home applications all depend on an accurate indoor positioning system. Basically one divides these systems into token-based and token-free localization systems. In this work, we focus on the token-free system based on smart floor technology. Smart floors can typically be built using pressure sensors or capacitive sensors. However, these set-ups are often hard to deploy as mechanical or electrical features are required below the surface and even harder to replace when detected a sensor malfunctioning. Therefore we present a novel indoor positioning system using an uncommon form of passive electric field sensing (EPS), which detects the electric potential variation caused by body movement. The EPS-based smart floor set-up is easy to install by deploying a grid of passive electrode wires underneath any non-conductive surfaces. Easy maintenance is also ensured by the fact that the sensors are not placed underneath the surface, but on the side. Due to the passive measuring nature, low power consumption is achieved as opposed to active capacitive measurement. Since we do not collect image data as in visual-based systems and all sensor data is processed locally, we preserve the user’s privacy. The proposed architecture achieves a high position accuracy and an excellent spatial resolution. Based on our evaluation conducted in our living lab, we measure a mean positioning error of only 12.7 cm.

Show publication details
Jabhe, Ali Khesrau; Kuijper, Arjan [1. Prüfer]; Kügler, David [2. Prüfer]

Physical World Attacks in Medical Imaging

2019

Darmstadt, TU, Bachelor Thesis, 2019

During this decade Deep Learning (DL) has become a game changer in the area of MedicalImaging. More and more tasks whether classification, detection, segmentation, etc. are beingsolved by DL algorithms with significantly high performance gains compared to traditionalmethods. However, Deep Neural Network (DNN)-based classification models have been foundto be vulnerable to adversarial attacks - maliciously and digitally manipulated image data -resulting in misclassification of the model.While only observed in Medical Imaging, adversarial examples define an established securitythreat to computer vision tasks. Furthermore, recent research has demonstrated theexistence of adversarial examples in the physical world - perturbations by physical artifactscausing misclassification instead of digitally perturbed data fed into the model. Since the existenceof adversarial examples in the physical world is not yet explored in Medical Imagingand trends are towards DL methods starting to get approved for clinical practice, this problemneeds more attention.Using the example of skin lesion classification, as it is likely to be the next aspirant for afully automated medical product, this work throws light on the potential threat adversarialexamples in the physical world pose to patients, physicians, and the healthcare system by generatinga new dataset "Physical Attacks in Dermoscopy" (PADv1) and evaluate susceptibilityand robustness of five state-of-the-art DNN-based skin lesion classifiers under physical attack.All architectures were trained on the HAM10000 dataset, a comprehensive challenge-verifiedbenchmark dataset for Machine Learning. Results show that on average the accuracy of thesearchitectures drops by 30.8%, and robustness decreases by 50% which means that every seconddiagnosis is fooled. Furthermore, a visualization of Gradient-weighted class activationmaps is provided to help understand or even interpret a models decision-making on clean andattacked image data.

Show publication details
Lenhart, Malte; Kuijper, Arjan [Betreuer]; Wilmsdorff, Julian von [Betreuer]

Prototyping Platform for Capacitive and Passive Electrical Field Sensing

2019

Darmstadt, TU, Bachelor Thesis, Jahr

In this thesis the Linoc prototyping toolkit is presented. It is built around two capac- itive and two Electric Potential Sensing (EPS) groups providing unobtrusive proximity detection in the field of Human Computer Interface (HCI). The toolkits focus lies on its usability in order to be adapted in future research and novel use cases. Its strength is the possibility to change its configuration at run time. A common obstacle in the beginning of a project is the time required to familiarize with present tools and systems, before the actual project can be attended to. This can be made worse by dependencies on previous work, often not fully documented and without training given from the original designer. Good toolkits can help to overcome this problem by providing a layer of abstraction and allowing to work on a higher level. If the toolkit however requires too much time to familiarize or behaves too restrictive, its goal has been missed and no benefits are generated. To access the quality of this thesis’ work, the Linoc toolkit was evaluated in terms of three different aspects: demonstration, usage and technical performance. A usage study found good reception, a steep learning curve and an interest to use the toolkit in the future. Technical benchmarks show a detectable range equal to its predecessors and in a demonstration it was shown that the toolkit can actually be used in projects.

Show publication details
Mukhopadhyay, Anirban; Kügler, David; Bucher, Andreas; Fellner, Dieter W.; Vogl, Thomas J.

Putting Trust First in the Translation of AI for Healthcare

2019

Ercim News

From screening diseases to personalised precision treatments, AI is showing promise in healthcare. But how comfortable should we feel about giving black box algorithms the power to heal or kill us?

Show publication details
Boutros, Fadi; Kuijper, Arjan [1. Gutachten]; Damer, Naser [2. Gutachten]

Reducing Ethnic Bias of Face Recognition by Ethnic Augmentation

2019

Darmstadt, TU, Master Thesis, 2019

Automated face recognition has gained wider deployment ground after the recent accuracy gains achieved by deep learning techniques. Despite the rapid performance gains, face recognition still suffers from very critical issues. One of the recently uncovered, and very sensitive, challenges is the ethnicity bias in face recognition decision. This is the case, unfortunately, even in the latest commercial and academic face recognition system. In 2018, the National Institute of Standards and Technology (NIST) published the latest report regarding the evaluation result of commercial face recognition solutions from several major biometric vendors. The report specifically evaluated and demonstrated the variance of the error rates of the evaluated solutions based on demographic variations. This thesis is one of the first research efforts in addressing the decision bias challenge in biometrics. It builds its hypothesis on the strong assumption that ethnicity bias is caused by the relative under-representation of certain ethnicities in training databases. This work introduces a novel ethnicity-driven data augmentation approach to reduce biometric bias. The proposed approach successfully utilize a generative image model to generate new face images that preserve the identity of their source images while partially transforming their ethnicity to the targeted ethnicity group. A large-scale ethnicity-labeled face images database is created in order to develop and evaluate the proposed approach. To achieve that, part of this thesis focused on creating an ethnicity classifier to annotate face images, achieving accuracy in the state-of-the-art range. The proposed ethnicity-driven face generative model is developed based on the ethnicity labeled images to generate realistically and high-resolution face images, depending on a limit amount of training data. More importantly, the thesis proves that the proposed augmentation approach strongly preserves the identity of the input images and partially transforms the ethnicity. The augmented images are used as part of the training data of a face recognition model. The achieved verification results prove that the proposed ethnicity augmentation methods significantly and consistently reduced the ethnicity bias of the face recognition model. For examples, the ERR was reduced from 0.159 to 0.130 when verifying inter-ethnicity samples of Black individuals on a model trained on Asian individual images, respectively before and after applying the proposed training data augmentation. Moreover, the overall performance of the face recognition model was improved. However, this improvement was more significant, as intended, in the targeted ethnicity groups.

Show publication details
Fuchs, Moritz; Kuijper, Arjan [Betreuer]; Noll, Matthias [Betreuer]

Registrierung eines Biopsieroboters zu bildbasierten Planungsdaten

2019

Darmstadt, TU, Bachelor Thesis, 2019

Das Setzen eines perkutanen Zugangs durch einen Roboter ist erstrebenswert, um die Genauigkeit von Eingriffen, wie Biopsien, thermale und non-thermale Ablation, zu verbessern. Für das dafür verwendete Robotersystem ist es notwendig, die Transformation des Roboters innerhalb eines CT zu bestimmen. Der Roboter hat hierfür ein Registrierungswerkzeug, dass auch die Nadelführung beinhaltet, an seinem Endeffektor angebracht. Das Registrierungswerkzeug beinhaltet vier Kugeln, die als Marker für die Ermittlung der Transformation dienen. In dieser Arbeit wurde sich damit beschäftigt die Kugeln zu lokalisieren. Es wird hier zu ein zweistufiges Verfahren verwendet, dass zunächst die eine Region of Interest(ROI) lokalisiert um anschließend die genau Transformation zu ermitteln. Für die Ermittlung der genauen Transformation wurden Optimierungsverfahren getestet, als auch Merkmal-Extraaktionstechniken. Als Optimierungsverfahren werden der Amoeba Algorithmus und der Gradientenabstieg, in Verbindung mit verschieden Metriken und Modellen getestet. Jedoch sind die Ergebnisse der Optimierer abhängig von der Initialisierung. Eine fehlerhafte Initialisierung konnte von den Optimierern nicht ausgeglichen werden. Bei der Merkmal-Extraaktionstechnik werden die Mittelpunkte der vier Kugeln bestimmt und zu einem Modell registriert. Zur Bestimmung der Kugelmittelpunkte wurde die Hough-Transformation verwendet. Es lässt damit eine zuverlässige Genauigkeit von DICE = 0.95879 auf den alten Registrierungswerkzeugen erreichen. Durch die bessere Fertigung und Bildqualität ist auf neueren Registrierungswerkzeugen ein DICE = 0.98515 möglich. Die Translationsabweichung ist dabei auf Sub-Voxel-Spacing genau.

Show publication details
Frank, Sebastian; Kuijper, Arjan

Robust Driver Foot Tracking and Foot Gesture Recognition Using Capacitive Proximity Sensing

2019

Journal of Ambient Intelligence and Smart Environments

Nowadays, there is an increasing trend towards automated driving. This is supported by both driver assistance systems getting more and more available and powerful, and research for car manufacturing industries. As a consequence, driver hands and feet are less involved in vehicle control. Increasing automation will even let them become idle. Recent gesture recognition mainly focuses on hand interaction. This work focuses on possibilities for feet gesture interaction. Many gesture recognition systems rely on computing intensive, privacy concerns causing video systems. Furthermore, these systems require a line of sight and therefore visible interior design integration. The proposed system shows that invisibly integrated capacitive proximity sensors can do the job, too. They do not cause privacy issues and they can be integrated under non-conductive materials. Therefore, there is no visible interior design impact. The proposed solution distinguishes between four feet gestures. There is no limitation to feet movement. Further, an evaluation including six participants and a vehicle legroom mockup shows the system function. This work contributes to the basis of driver foot gesture recognition pointing to further applications and more comprehensive investigations.

Show publication details

Seamless and Non-repetitive 4D Texture Variation Synthesis and Real-time Rendering for Measured Optical Material Behavior

2019

Computational Visual Media

We show how to overcome the single weakness of an existing fully automatic system for acquisition of spatially varying optical material behavior of real object surfaces. While the expression of spatially varying material behavior with spherical dependence on incoming light as a 4D texture (an ABTF material model) allows flexible mapping onto arbitrary 3D geometry, with photo-realistic rendering and interaction in real time, this very method of texture-like representation exposes it to common problems of texturing, striking in two disadvantages. Firstly, non-seamless textures create visible artifacts at boundaries. Secondly, even a perfectly seamless texture causes repetition artifacts due to their organised placement in large numbers over a 3D surface. We have solved both problems through our novel texture synthesis method that generates a set of seamless texture variations randomly distributed over the surface at shading time. When compared to regular 2D textures, the inter-dimensional coherence of the 4D ABTF material model poses entirely new challenges to texture synthesis, which includes maintaining the consistency of material behavior throughout the 4D space spanned by the spatial image domain and the angular illumination hemisphere. In addition, we tackle the increased memory consumption caused by the numerous variations through a fitting scheme specifically designed to reconstruct the most prominent effects captured in the material model.

Show publication details
Hashisho, Yousif; Lukas, Uwe von [Gutachter]; Staadt, Oliver [Gutachter]; Albadawi, Mohamad [Supervising Advisor]; Krause, Tom [Supervising Advisor]

Underwater Image Enhancement Using Autoencoders

2019

Rostock, Univ., Master Thesis, 2019

Visual inspection of underwater structures by vehicles, e.g. remotely operated vehicles (ROVs), plays an important role in scientific, military, and commercial aspects. However, the automatic extraction of information using software tools is hindered by the optical characteristics of water which degrade the quality of the videos. As a contribution for enhancing underwater images, we develop an algorithm using a single denoising autoencoder to restore the color and remove the disturbances such as marine snow from underwater images. Marine snow in some images is only partially removed using the proposed network; however, we show the reason behind this failure. Related learning methods use generative adversarial networks (GANs) for generating color corrected underwater images, and to our knowledge this thesis is the first to deal with a single autoencoder capable of producing same or better results. Moreover, underwater aligned image pairs are established for the training of the proposed network, where it is hard to obtain such dataset from underwater scenery. The objective is to increase the accuracy and reliability on automatic underwater operations that rely on robotic perception without human interference by improving the quality of the captured frames. At the end, the proposed network is evaluated using Mean Squared Error (MSE), Peak Signal-to- Noise Ratio (PSNR), and Structural Similarity Index Metric (SSIM) quality metrics. Additionally, we compare our experiment with a related method. The proposed network takes into consideration the computation cost and the accuracy to have real-time implementation on visual-driven tasks using a single autoencoder.

Show publication details

Unsupervised Privacy-enhancement of Face Representations Using Similarity-sensitive Noise Transformations

2019

Applied Intelligence

Face images processed by a biometric system are expected to be used for recognition purposes only. However, recent work presented possibilities for automatically deducing additional information about an individual from their face data. By using soft-biometric estimators, information about gender, age, ethnicity, sexual orientation or the health state of a person can be obtained. This raises a major privacy issue. Previous works presented supervised solutions that require large amount of private data in order to suppress a single attribute. In this work, we propose a privacy-preserving solution that does not require these sensitive information and thus, works in an unsupervised manner. Further, our approach offers privacy protection that is not limited to a single known binary attribute or classifier. We do that by proposing similarity-sensitive noise transformations and investigate their effect and the effect of dimensionality reduction methods on the task of privacy preservation. Experiments are done on a publicly available database and contain analyses of the recognition performance, as well as investigations of the estimation performance of the binary attribute of gender and the continuous attribute of age. We further investigated the estimation performance of these attributes when the prior knowledge about the used privacy mechanism is explicitly utilized. The results show that using this information leads to significantly enhancement of the estimation quality. Finally, we proposed a metric to evaluate the trade-off between the privacy gain and the recognition loss for privacy-preservation techniques. Our experiments showed that the proposed cosine-sensitive noise transformation was successful in reducing the possibility of estimating the soft private information in the data, while having significantly smaller effect on the intended recognition performance.

Show publication details
Niemeyer, Frank; Dolereit, Tim; Neumann, Matthias; Albiez, Jan; Vahl, Matthias; Geist, Michael

Untersuchungen von optischen Scansystemen zur geometrischen Erfassung von Unterwasserstrukturen

2019

Hydrographische Nachrichten

Dieser Beitrag beschäftigt sich mit optischen Erfassungs- und Scansystemen für den Bereich unter Wasser. Dabei wurde eine Einteilung in photogrammetrische, trigonometrische und impulsbasierte Systeme vorgenommen. Es wurden Messungen mit verschiedenen Systemen im Schleppkanal des Lehrstuhls für Strömungstechnik der Universität Rostock durchgeführt. Stellvertretend für photogrammetrische Erfassungssysteme kam das Stereokamerasystem vom Fraunhofer-Institut für Graphische Datenverarbeitung zum Einsatz. Das ULS-200-Scansystem von 2GRobotics und das SeaVision- Lasersystem von Kraken Robotik standen stellvertretend für trigonometrische Scansysteme zur Verfügung. Ein impulsbasiertes Scansystem stand für die Untersuchungen nicht bereit. Zusätzlich wurde das SeaVision-Scansystem in der Ostsee nahe des künstlichen Riffs bei Nienhagen unter realistischen Bedingungen getestet. Die Messungen spiegeln das derzeitige Potenzial optischer Messsysteme für den Bereich unter Wasser wider. Vor- und Nachteile der Systeme werden diskutiert.

Show publication details
Bernard, Jürgen; Sessler, David; Kohlhammer, Jörn; Ruddle, Roy A.

Using Dashboard Networks to Visualize Multiple Patient Histories: A Design Study on Post-operative Prostate Cancer

2019

IEEE Transactions on Visualization and Computer Graphics

In this design study, we present a visualization technique that segments patients' histories instead of treating them as raw event sequences, aggregates the segments using criteria such as the whole history or treatment combinations, and then visualizes the aggregated segments as static dashboards that are arranged in a dashboard network to show longitudinal changes. The static dashboards were developed in nine iterations, to show 15 important attributes from the patients' histories. The final design was evaluated with five non-experts, five visualization experts and four medical experts, who successfully used it to gain an overview of a 2,000 patient dataset, and to make observations about longitudinal changes and differences between two cohorts. The research represents a step-change in the detail of large-scale data that may be successfully visualized using dashboards, and provides guidance about how the approach may be generalized.

Show publication details

Virtual Reality in Media and Technology

2019

Digital Transformation
Fraunhofer-Forschungsfokus

Virtual and Augmented Reality technologies have by now become established in numerous engineering areas of application. Also in the cultural and media fields interactive three-dimensional content is being increasingly made available for information purposes, and used in scientific research. On the one hand, this development is accelerated by current advances in smartphones, tablets and head-mounted displays. These support complex 3D applications in mobile application scenarios, and enable us to capture our real physical environment using multimodal sensors in order to correlate it with the digital 3D world. On the other hand, new automated digitization technologies such as CultLab3D of the Fraunhofer Institute for Computer Graphics Research IGD allow the production of the necessary digital replicas of real objects, quickly, economically and of high quality.

  • 978-3-662-58133-9
Show publication details
Riaz, Muhammad Ali; Kuijper, Arjan [1. Prüfer]; Burkhardt, Dirk [2. Prüfer]

Visual Trend Analysis on Condensed Expert Data beside Research Library Data for Enhanced Insights

2019

Darmstadt, TU, Master Thesis, 2019

In the present age of information, we live amidst seas of digital text documents including academic publications, white papers, news articles, patents, newspapers. To tackle the issue of the ever-increasing amount of text documents, researchers from the field of text mining and information visualization have developed tools and techniques to facilitate text analysis. In the context of visual trend analysis on text data, the use of well-structured patent data and public digital libraries are quite established. However, both sources of information have their limitations. For instance, the registration process for patents takes at least one year, which makes the extracted insights not suitable to research on present scenarios. In contrast to patent data, the digital libraries are up-to-date but provide high-level insights, only limited to broader research domains, and the data usage is almost restricted on meta information, such as title, author names and abstract, and they do not provide full text. For a certain type of detailed analysis such as competitor analysis or portfolio analysis, data from digital libraries is not enough, it would also make sense to analyze the full-text. Even more, it can be beneficial to analyze only a limited dataset that is filtered by an expert towards a very specific field, such as additive printing or smart wearables for medical observations. Sometimes also a mixture of both digital library data and manually collected documents is relevant to be able to validate a certain trend, where one gives a big picture and other gives a very condensed overview of the present scenario. The thesis aims, therefore, to focus on such manually collected documents by experts that can be defined as condensed data. So, the major goal of this thesis is to conceptualize and implement a solution that enables the creation and analysis of such a condensed data set and compensate therewith the limitations of digital library data analysis. As a result, a visual trend analysis system for analyzing text documents is presented, it utilizes the best of both state-of-the-art text analytics and information visualization techniques. In a nutshell, the presented trend analysis system does two things. Firstly, it is capable of extracting raw data from text documents in the form of unstructured text and meta-data, convert it into structured and analyzable formats, extract trends from it and present it with appropriate visualizations. Secondly, the system is also capable of performing gap-analysis tasks between two data sources, which in this case is digital library data and data from manually collected text documents (Condensed Expert Data). The proposed visual trend analysis system can be used by researchers for analyzing the research trends, organizations to identify current market buzz and industry trends, and many other use-cases where text data is the primary source of valuable information.

Show publication details
Lehmann, Markus; Kuijper, Arjan [1. Gutachten]; Bernard, Jürgen [2. Gutachten]

Visual-Interactive Combination of Selection Strategies to Improve Data Labeling Processes

2019

Darmstadt, TU, Master Thesis, 2019

Labeling training data is an important task in Machine Learning for building effective and efficient classifiers. There are different approaches to gather labeled instances for a particular data set. The two most important fields of strategies are Active Learning and Visual-Interactive Labeling. In previous work, these strategies were examined and compared, resulting in a set of atomic labeling strategies. Additionally, a quasi-optimal strategy was analyzed in order to infer knowledge from its behavior. This investigation resulted in two main insights. First, the labeling process consists of different phases. Second, the performance of a strategy depends on the data set and its characteristics.In this work, we propose a toolkit which enables users to create novel labeling strategies. First, we present multiple visual interfaces users can employ to examine the space of existing algorithms. Then, we introduce a definition of ensembles users can build upon in order to combine existing strategies to novel strategies. Multiple methods to measure the quality of labeling strategies are provided to users, enabling them to examine the gap between their strategies and existing strategies. The different phases of the labeling process are included in the toolkit in order to allow users to always apply the most appropriate strategy in each phase. During the entire process, users are supported by automated guidance in the improvement of their strategies.We evaluate our concept from different perspectives in order to assess its quality. Overall, we observe that our approach enables users to build ensemble strategies which outperform existing strategies. The insights from this work can be applied to develop novel concepts towards ensemble building as well as to improve the generalization of strategies to other data sets.

Show publication details
Reinemuth, Heiko; Kuijper, Arjan [1. Prüfer]; Bernard, Jürgen [2. Prüfer]

Visual-Interactive Labeling of Multivariate Time Series to Support Semi-Supervised Machine Learning

2019

Darmstadt, TU, Master Thesis, 2019

The labeling of multivariate time series is an essential requirement of data-centric decisionmaking processes in many time-oriented application domains. The basic idea of labeling is to assign (semantic) meaning to specific sections or time steps of the time series and to the time series as a whole, accordingly. Hence, weather phenomena can be characterized, EEG signals can be studied, or movement patterns can be marked in sensor data. In the context of this work a visual-interactive labeling tool was developed that allows nonexpert users to assign semantic meaning to any multivariate time series in an effective and efficient way. Enabling experts as well as non-experts to label multivariate time series in a visual-interactive way has never been proposed in the information visualization and visual analytics research communities before. This thesis combines active learning methods, a visual analytics approach, and novel visual-interactive interfaces to achieve an intuitive data exploration and labeling process for users. Visual guidance based on data analysis and model-based predictions empowers users to select and label meaningful instances from the time series. This user-side selection and labeling task can be taken over by an automated model or data-based process. Visual representations of labeling quality and novel interfaces allow for additional user-side refinement.