Liste der Fachpublikationen

Ergebnisse 1 - 79 von 79
Show publication details
Cui, Jian; Lan, Zirui; Liu, Yisi; Li, Ruilin; Sourina, Olga; Müller-Wittig, Wolfgang K.

A Compact and Interpretable Convolutional Neural Network for cross-subject Driver Drowsiness Detection from single-channel EEG

2021

Methods

Driver drowsiness is one of the main factors leading to road fatalities and hazards in the transportation industry. Electroencephalography (EEG) has been considered as one of the best physiological signals to detect drivers’ drowsy states, since it directly measures neurophysiological activities in the brain. However, designing a calibration-free system for driver drowsiness detection with EEG is still a challenging task, as EEG suffers from serious mental and physical drifts across different subjects. In this paper, we propose a compact and interpretable Convolutional Neural Network (CNN) to discover shared EEG features across different subjects for driver drowsiness detection. We incorporate the Global Average Pooling (GAP) layer in the model structure, allowing the Class Activation Map (CAM) method to be used for localizing regions of the input signal that contribute most for classification. Results show that the proposed model can achieve an average accuracy of 73.22% on 11 subjects for 2-class cross-subject EEG signal classification, which is higher than conventional machine learning methods and other state-of-art deep learning methods. It is revealed by the visualization technique that the model has learned biologically explainable features, e.g., Alpha spindles and Theta burst, as evidence for the drowsy state. It is also interesting to see that the model uses artifacts that usually dominate the wakeful EEG, e.g., muscle artifacts and sensor drifts, to recognize the alert state. The proposed model illustrates a potential direction to use CNN models as a powerful tool to discover shared features related to different mental states across different subjects from EEG signals.

Show publication details
Gao, Shan; Ye, Qixiang; Liu, Li; Kuijper, Arjan; Ji, Xiangyang

A Graphical Social Topology Model for RGB-D Multi-Person Tracking

2021

IEEE Transactions on Circuits and Systems for Video Technology (TCSVT)

Tracking multiple persons is a challenging task especially when persons move in groups and occlude one another. Existing research have investigated the problems of group division and segmentation; however, lacking overall person-group topology modeling limits the ability to handle complex person and group dynamics. We propose a Graphical Social Topology (GST) model in the RGB-D data domain, and estimate object group dynamics by jointly modeling the group structure and states of persons using RGB-D topological representation. With our topology representation, moving persons are not only assigned to groups, but also dynamically connected with each other, which enables in-group individuals to be correctively associated and the cohesion of each group to be precisely modeled. Using the learned typical topology pattern and group online update modules, we infer the birth/death and merging/splitting of dynamic groups. With the GST model, the proposed multi-person tracker can naturally facilitate the occlusion problem by treating the occluded object and other in-group members as a whole, while leveraging overall state transition. Experiments on different RGB-D and RGB datasets confirm that the proposed multi-person tracker improves the state-of-the-arts.

Show publication details

A Visualization Interface to Improve the Transparency of Collected Personal Data on the Internet

2021

IEEE Transactions on Visualization and Computer Graphics

Online services are used for all kinds of activities, like news, entertainment, publishing content or connecting with others. But information technology enables new threats to privacy by means of global mass surveillance, vast databases and fast distribution networks. Current news are full of misuses and data leakages. In most cases, users are powerless in such situations and develop an attitude of neglect for their online behaviour. On the other hand, the GDPR (General Data Protection Regulation) gives users the right to request a copy of all their personal data stored by a particular service, but the received data is hard to understand or analyze by the common internet user. This paper presents TransparencyVis - a web-based interface to support the visual and interactive exploration of data exports from different online services. With this approach, we aim at increasing the awareness of personal data stored by such online services and the effects of online behaviour. This design study provides an online accessible prototype and a best practice to unify data exports from different sources.

Show publication details
Schmidt, Tim; Kuijper, Arjan [1. Review]; Mukhopadhyay, Anirban [2. Review]

Adding Reason to the Blackbox: Building Surrogate Models Based on Adaptive Abstractions

2021

Darmstadt, TU, Master Thesis, 2021

everyday life [18]. Interpretability allows to validate the correct behaviour of such models and may prove key for the widespread acceptance of machine learning [64]. In this work a novel approach to add interpretability to the field of image classification is developed using surrogate modeling. This includes an image segmentation approach to transform image data into a suitable graph based representation. The human comprehensible representation is then used by a white box model to mimic the decisions of an arbitrary black box. Abstracting away the complexity of image data enables the white box model to reason about the image content in human understandable form. To assess the quality of the provided explanations an entropy based interpretability metric was developed. The developed system was evaluated using the Iris dataset [5, 29] and a self created image data set. The employed graph based representation proved powerful and flexible. Object occurrence, spatial relations and co-occurrence were spotted by the white box and could be visualised. The developed interpretability metric is stable and comparable. This thesis demonstrates that surrogate modeling can be applied to image classification. A powerful representation is necessary to deal with the complexity of image data. Graph based approaches are a suitable solution. Further an interpretability metric was developed allowing to assess the quality of the provided explanation. As this metric is promising it will be subject to future research.

Show publication details
Hashisho, Yousif; Dolereit, Tim; Segelken-Voigt, Alexandra; Bochert, Ralf; Vahl, Matthias

AI-assisted Automated Pipeline for Length Estimation, Visual Assessment of the Digestive Tract and Counting of Shrimp in Aquaculture Production

2021

Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications

International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP) <16, 2021, online>

Shrimp farming is a century-old practice in aquaculture production. In the past years, some improvements of the traditional farming methods have been made, however, it still involves mostly intensive manual work, which makes traditional farming a neither time nor cost efficient production process. Therefore, a continuous monitoring approach is required for increasing the efficiency of shrimp farming. This paper proposes a pipeline for automated shrimp monitoring using deep learning and image processing methods. The automated monitoring includes length estimation, assessment of the shrimp’s digestive tract and counting. Furthermore, a mobile system is designed for monitoring shrimp in various breeding tanks. This study shows promising results and unfolds the potential of artificial intelligence in automating shrimp monitoring.

Show publication details
Vatter, Jana; Noll, Matthias [1. Review]; Kuijper, Arjan [2. Review]

AI-based Segmentation of Human Hand Bone Structures and their Interconnection Areas in B-mode Ultrasound Image Data

2021

Darmstadt, TU, Master Thesis, 2021

To assess a patient’s arthritis condition, biopsies of the human synovial fluid may be performed under ultrasound guidance. To do this, the interconnection areas between the bones need to be targeted. With the aim of making this task easier, we develop an AI-based method to automatically extract human hand bone structures and their interconnections in b-mode ultrasound image data. Our method introduces a semi-automatic labeling process followed by training a neural network U-Net model. When deemed necessary, a post-processing step is performed. We combine a small manual labeled data set with the semi-automatically labeled data in a pre-training and fine-tuning step. Overall, good results are achieved when segmenting bones with a maximum Dice Similarity Coefficient of 0.7032 when pre-training on manually labeled data and fine-tuning on a subset of the semi-automatically labeled data. Experiments with interconnections resulted in a maximum score of 0.6892. Qualitative results when predicting on real clinical image data indicate that our method is suitable to be used in real-life applications. However, our approach is limited by low resolution image data and inconsistent fuzzy labels. Compared to former works, we did not only segment bones, but also their interconnection areas. Moreover, despite a small subset, we did not have any expert ground truth labels available, which differentiates us from other approaches.

Show publication details
Guthe, Stefan; Thürck, Daniel

Algorithm 1015: A Fast Scalable Solver for the Dense Linear (Sum) Assignment Problem

2021

ACM Transactions on Mathematical Software

We present a new algorithm for solving the dense linear (sum) assignment problem and an efficient, parallel implementation that is based on the successive shortest path algorithm. More specifically, we introduce the well-known epsilon scaling approach used in the Auction algorithm to approximate the dual variables of the successive shortest path algorithm prior to solving the assignment problem to limit the complexity of the path search. This improves the runtime by several orders of magnitude for hard-to-solve real-world problems, making the runtime virtually independent of how hard the assignment is to find. In addition, our approach allows for using accelerators and/or external compute resources to calculate individual rows of the cost matrix. This enables us to solve problems that are larger than what has been reported in the past, including the ability to efficiently solve problems whose cost matrix exceeds the available systems memory. To our knowledge, this is the first implementation that is able to solve problems with more than one trillion arcs in less than 100 hours on a single machine.

Show publication details
Bonneschky, Marco; Kuijper, Arjan [Betreuer]; Winner, Hermann [Betreuer]

Analysis of the Usability of IPGMovie in a CarMaker-Simulation for the Development of Perception Algorithms for Automated Driving

2021

Darmstadt, TU, Bachelor Thesis, 2021

Before a perception algorithm finds its way on autonomous driving vehicles, it has to be tested thoroughly. As real world tests are laborious and reproducible tests are beneficial, simulations are used to accelerate this process. Most simulations are highly capable to be used along sensor system models to develop automated driving features. In contrast perception algorithms are mainly developed on real world data. In this work a methodology is developed to measure the usability of a simulation for the development of perception algorithms for automated driving - called realism. This is accomplished by comparing the output of an exemplary perception algorithm on real and recreated virtual test drives and comparing the outputs. With the used procedure a metric is defined to rate the realism of the simulation. Before applying this methodology special requirements on the simulation for the use case are elaborated. Furthermore challenges for the development of perception algorithms using simulations and possible influences on the evaluation are described. Finally the methodology is applied on the vehicle distance estimation and IPG CarMaker using the NVIDIA DriveNet module developed by Continental AG. The analysis shows that even with a minimal test coverage the scenario specific behaviour is present in the simulation, but the errors are leveraged.

Show publication details

Analyzing and Improving the Parameterization Quality of Catmull-Clark Solids for Isogeometric Analysis

2021

IEEE Computer Graphics and Applications

In the field of physically based simulation, high quality of the simulation model is crucial for the correctness of the simulation results and the performance of the simulation algorithm. When working with spline or subdivision models in the context of isogeometric analysis, the quality of the parameterization has to be considered in addition to the geometric quality of the control mesh. Following Cohen et al.’s concept of model quality in addition to mesh quality, we present a parameterization quality metric tailored for Catmull–Clark (CC) solids. It measures the quality of the limit volume based on a quality measure for conformal mappings, revealing local distortions and singularities. We present topological operations that resolve these singularities by splitting certain types of boundary cells that typically occur in interactively designed CC-solid models. The improved models provide higher parameterization quality that positively affects the simulation results without additional computational costs for the solver.

Show publication details

Automatic Model-based 3-D Reconstruction of the Teeth from five Photographs with Predefined Viewing Directions

2021

Medical Imaging 2021: Image Processing

SPIE Medical Imaging Symposium <2021, online>

Proceedings of SPIE
11596, 1

Misalignment of teeth or jaws can impact the ability to chew or speak, increase the risk of gum disease or tooth decay, and potentially inuence a person's (psychological) well-being. Orthodontic treatments of misaligned teeth are complex procedures that employ dental braces to apply forces in order to move the teeth or jaws to their correct position. Photographs are typically used to document the treatment. An automatic analysis of those photographs could support the decision making and monitoring process. In this paper, we propose an automatic model-based end-to-end 3-D reconstruction approach of the teeth from _ve photographs with prede_ned viewing directions (i.e. the photographs used in orthodontic treatment documentation). It uses photo- or view-speci_c 2- D coupled shape models to extract the teeth contours from the images. The shape reconstruction is then carried out by a deformation-based reconstruction approach that utilizes 3-D coupled shape models and minimizes a silhouette-based loss. The optimal model parameters are determined by an optimization which maximizes the overlaps between the projected 2-D outlines of individual teeth of the 3-D model and the contours extracted from the corresponding photograph. After that the point displacements between the projected outline and segmented contour are used to iteratively deform the 3-D shape model of each tooth for all _ve views. Back-projection into shape space ensures that the 3-D coupled shape model consists of (statistically) valid teeth. Evaluation on 22 data sets shows promising results with an average symmetric surface distance of 0.848mm and an average DICE coe_cient of 0.659.

Show publication details
Oyarzun Laura, Cristina; Hartwig, Katrin; Distergoft, Alexander; Hoffmann, Tim; Scheckenbach, Kathrin; Brüsseler, Melanie; Wesarg, Stefan

Automatic Segmentation of the Structures in the Nasal Cavity and the Ethmoidal Sinus for the Quantification of Nasal Septal Deviations

2021

Medical Imaging 2021: Computer-Aided Diagnosis

SPIE Medical Imaging Symposium <2021, online>

Proceedings of SPIE
11597

Nasal septal deviations are a well-known and widespread problem. According to the American Academy of Otolaryngology 80% of the population have a nasal septal deviation. Its level of severity can range from the person not being aware of it to respiratory obstruction and choking. It is therefore necessary to distinguish those patients at risk. For a proper diagnosis, the amount and location of the deviation have to be considered, but also the shape and changes in the surrounding turbinates. The segmentation of the structures of interest is an important step to reduce subjectivity in the diagnosis. Unfortunately, due to their variable and tortuous shape manual segmentation is time consuming. In this paper, the _rst method for the automatic segmentation of the structures in the nasal cavity and ethmoidal sinus is presented. A coupled shape model of the nasal cavity and paranasal sinus regions is trained and used to detect the corresponding regions in new CT images. The nasal septum is then segmented using a novel slice-based propagation technique. This segmentation allows the additional separation and segmentation of the left and right nasal cavities and ethmoidal sinuses and their structures by means of an adaptive thresholding with varying boundary sizes. The method has been evaluated in 10 CT images obtaining promising results for the nasal septum (DICE: 87.71%) and for the remaining structures (DICE: 72.01% - 73.01%). Based on the resulting segmentations, a web-based diagnosis tool has been designed to quantify the septal deviation using three metrics proposed by clinical experts.

Show publication details
Rojtberg, Pavel; Kuijper, Arjan [Betreuer]; Fellner, Dieter W. [Betreuer]; Stricker, Didier [Betreuer]

Automation for camera-only 6D Object Detection

2021

Darmstadt, TU, Diss., 2021

Today a widespread deployment of Augmented Reality (AR) systems is only possible by means of computer vision frameworks like ARKit and ARCore, which abstract from specific devices, yet restrict the set of devices to the respective vendor. This thesis therefore investigates how to allow deploying AR systems to any device with an attached camera. One crucial part of an AR system is the detection of arbitrary objects in the camera frame and naturally accompanying the estimation of their 6D-pose. This increases the degree of scene understanding that AR applications require for placing augmentations in the real world. Currently, this is limited by a coarse segmentation of the scene into planes as provided by the aforementioned frameworks. Being able to reliably detect individual objects, allows attaching specific augmentations as required by e.g. AR maintenance applications. For this, we employ convolutional neural networks (CNNs) to estimate the 6D-pose of all visible objects from a single RGB image. Here, the addressed challenge is the automated training of the respective CNN models, given only the CAD geometry of the target object. First, we look at reconstructing the missing surface data in real-time before we turn to the more general problem of bridging the domain gap between the non-photorealistic representation and the real world appearance. To this end, we build upon generative adversarial network (GAN) models to formulate the domain gap as an unsupervised learning problem. Our evaluation shows an improvement in model performance, while providing a simplified handling compared to alternative solutions. Furthermore, the calibration data of the used camera must be known for precise pose estimation. This data, again, is only available for the restricted set of devices, that the proprietary frameworks support. To lift this restriction, we propose a web-based camera calibration service that not only aggregates calibration data, but also guides users in the calibration of new cameras. Here, we first present a novel calibration-pose selection framework that reduces the number of required calibration images by 30% compared to existing solutions, while ensuring a repeatable and reliable calibration outcome. Then, we present an evaluation of different userguidance strategies, which allows choosing a setting suitable for most users. This enables even novice users to perform a precise camera calibration in about 2 minutes. Finally, we propose an efficient client-server architecture to deploy the aforementioned guidance on the web, making it available to the widest possible range of devices. This service is not restricted to AR systems, but allows the general deployment of computer vision algorithms on the web that rely on camera calibration data, which was previously not possible. These elements combined, allow a semi-automatic deployment of AR systems with any camera to detect any object.

Show publication details
Endl, Barbara Sophie; Kuijper, Arjan [Prüfer]; Boine-Frankenheim, Oliver [Prüfer]; Schmitz, Benedikt [Betreuer]

Automatisierte Bilderverarbeitung für Ionenstrahldiagnostik

2021

Darmstadt, TU, Bachelor Thesis, 2021

Ziel der vorliegenden Arbeit ist es, die momentan manuelle bildtechnische Auswertung radiochromatischer Filme (kurz RCFs), die bei der Charakterisierung von laserbeschleunigten Ionenstrahlen zum Einsatz kommen, zu automatisieren.

Show publication details
Bieber, Gerald; Antony, Niklas; Kraft, Dimitri; Hölle, Bernd; Blenke, Dennis; Herrmann, Peter

Barcode-based Navigation Concept for Autonomous Wheelchairs and Walking Frames

2021

Proceedings of the 14th ACM International Conference on PErvasive Technologies Related to Assistive Environments

ACM International Conference on PErvasive Technologies Related to Assistive Environments (PETRA) <14, 2021, Online>

ACM International Conference Proceedings Series (ICPS)

After a knee or hip surgery, a fast mobilization of the patient is highly recommended. The best therapy would be individual physiotherapy or guided walks with personal assistance, but unfortunately, this is very quite expensive and clinics are suffering from a lack of well-educated personnel. With RoRo, an autonomous walking frame (rollator), the patient receives walking support, reminders for exercises, and a measurement of the gait and walking parameters. For the navigation purposes of the autonomous rollator within the building of a clinic, a precise indoor navigation concept is needed that even works without a continuous internet connection. This paper describes the location and navigation concept with optical barcodes that are stuck at the top of each floor. The information of the barcodes is coded in the way, that they contain orientation features and are spanning a hierarchical structure.

Show publication details
Kraft, Dimitri; Laerhoven, Kristof van; Bieber, Gerald

CareCam: Concept of a New Tool for Corporate Health Management

2021

Proceedings of the 14th ACM International Conference on PErvasive Technologies Related to Assistive Environments

ACM International Conference on PErvasive Technologies Related to Assistive Environments (PETRA) <14, 2021, Online>

Corporate health management is an important tool for preventing work-related absenteeism, increasing overall employee satisfaction and reducing the costs of absenteeism or presenteeism in the long term. Today, corporate health management is even more important because many employees work from home. Often, there is a lack of workspace or even a workplace that meets minimum occupational health and safety guidelines. Overwork, noise pollution, incorrect sitting posture and unstructured work breaks contribute negatively to the daily work routine, as does the general problem of separating work and leisure. Under these conditions, daily work is made even more difficult, which can lead to increased mental and physical stress. A concept for unobtrusive monitoring to increase long-term health, improve working conditions or at least to show the necessary adjustments to the new work situation can help to solve these problems. This paper presents a concept that shows how a simple webcam can be used to record essential vital signs during working hours, evaluate them using machine learning, and offer intervention recommendations based on these data to reduce psychological and physical stress. Work on continuous stress measurement and the challenges associated with it will be presented. This work serves as a starting point for the development of a camera-based tool for mental and physical stress measurement in theworkplace. Our approach demonstrates that the required parameters can be captured using a simple webcam and that interventions can be used to achieve long-term reductions in work-related mental and physical stress, provided that the proposed interventions are followed. The prototypical implementation shows that such a solution can work well in the workplace, but that data protection and technical limitations must be considered in the future in order to establish camera-based methods in the toolbox of workplace health management.

Show publication details
Boche, Benjamin; Kuijper, Arjan [1. Gutachten]; Gorschlüter, Felix [2. Gutachten]

Comparing RGBD-based 6D Pose Estimation

2021

Darmstadt, TU, Bachelor Thesis, 2021

In this work the suitability of the pose estimation method "PVN3D", by He et al.[3], is evaluated for industrial applications. It first gives an overview over the research eld of 6D pose estimation. Starting by explaining the basics of the eld this work goes over point pair features, machine learning and evaluation metrics as a background knowledge for the 6D pose estimation methods of He et al.[3] and Vidal et al.[2, 23] and the 6D pose estimation benchmark "BOP" introduced by Hodan et al. [6, 16]. The main contribution of this thesis is the evaluation of PVN3D[3] on the BOP[6] framework, so it can be compared to a multitude of other pose estimators, e.g. the methods benchmarked in the BOP 2020 challenge[16]. Here we focus on the T-LESS[11] dataset, as provided by BOP[16], it provides images of objects typical of industrial applications. We found that PVN3D[3] performs worst on T-LESS[11] compared to the BOP-results from 2020[16]. Specially it performs worse than the PPF based method by Vidal et al.[2, 23], itself ranking 6th in 2020[16]. However, it has to be noted, that our experiments were not fully conclusive, as there are strong indications that there is an unknown bug in the training of PVN3D[3].

Show publication details

Cross-database and Cross-attack Iris Presentation Attack Detection Using Micro Stripes Analyses

2021

Image and Vision Computing

With the widespread use of mobile devices, iris recognition systems encounter more challenges, such as the vulnerability of Presentation Attack Detection (PAD). Recent works pointed out the contact lens attacks, especially images captured under the uncontrolled environment, as a hard task for iris PAD. In this paper, we propose a novel framework for detecting iris presentation attacks that especially for detecting contact lenses based on the normalized multiple micro stripes. The classification decision is made by the majority vote of those micro-stripes. An in-depth experimental evaluation of this framework reveals a superior performance in three databases compared with state-of-the-art (SoTA) algorithms and baselines. Moreover, our solution minimizes the confusion between textured (attack) and transparent (bona fide) presentations in comparison to SoTA methods. We support the rationalization of our proposed method by studying the significance of different pupil-centered eye areas in iris PAD decisions under different experimental settings. In addition, extensive cross-database and cross-attack (unknown attack) detection evaluation experiments are demonstrated to explore the generalizability of our proposed method, texture-based method, and neural network based methods in three different databases. The results indicate that our Micro Stripes Analyses (MSA) method has, in most experiments, better generalizability compared to other baselines.

Show publication details
Nonnemann, Lars; Hogräfer, Marius; Schumann, Heidrun; Urban, Bodo; Schulz, Hans-Jörg

Customizable Coordination of Independent Visual Analytics Tools

2021

EuroVA 2021

International EuroVis Workshop on Visual Analytics (EuroVA) <2021, Online>

While it is common to use multiple independent analysis tools in combination, it is still cumbersome to carry out a cross-tool visual analysis. Some dedicated frameworks addressing this issue exist, yet in order to use them, a Visual Analytics tool must support their API or architecture. In this paper, we do not rely on a single predetermined exchange mechanism for the whole ensemble of VA tools. Instead, we propose using any available channel for exchanging data between two subsequently used VA tools. This effectively allows to mix and match different data exchange strategies within one cross-tool analysis, which considerably reduces the overhead of adding a new VA tool to a given tool ensemble. We demonstrate our approach with a first implementation called AnyProc and its application to a use case of three VA tools in a Health IT data analysis scenario.

Show publication details

Deep Learning Models for Optically Characterizing 3D Printers

2021

Optics Express

Multi-material 3D printers are able to create material arrangements possessing various optical properties. To reproduce these properties, an optical printer model that accurately predicts optical properties from the printer’s control values (tonals) is crucial. We present two deep learning-based models and training strategies for optically characterizing 3D printers that achieve both high accuracy with a moderate number of required training samples. The first one is a Pure Deep Learning (PDL) model that is essentially a black-box without any physical ground and the second one is a Deep-Learning-Linearized Cellular Neugebauer (DLLCN) model that uses deep-learning to multidimensionally linearize the tonal-value-space of a cellular Neugebauer model. We test the models on two six-material polyjetting 3D printers to predict both reflectances and translucency. Results show that both models can achieve accuracies sufficient for most applications with much fewer training prints compared to a regular cellular Neugebauer model.

Show publication details
Siegmund, Dirk; Fu, Biying; José-García, Adán; Salahuddin, Ahmad Masood; Kuijper, Arjan

Detection of Fiber Defects Using Keypoints and Deep Learning

2021

International Journal of Pattern Recognition and Artificial Intelligence

Due to the deforming and dynamically changing textile fibers, the quality assurance of cleaned industrial textiles is still a mostly manual task. Usually, textiles need to be spread flat, in order to detect defects using computer vision inspection methods. Already known methods for detecting defects on such inhomogeneous, voluminous surfaces use mainly supervised methods based on deep neural networks and require lots of labeled training data. In contrast, we present a novel unsupervised method, based on SURF keypoints, that does not require any training data. We propose using their location, number and orientation in order to group them into geographically close clusters. Keypoint clusters also indicate the exact position of the defect at the same time. We furthermore compared our approach to supervised methods using deep learning. The presented processing pipeline shows how normalization and classification methods need to be combined, in order to reliably detect fiber defects such as cuts and holes. We evaluate the performance of our system in real-world settings with images of piles of textiles, taken in stereo vision. Our results show that our novel unsupervised classification method using keypoint clustering achieves comparable results to other supervised methods.

Show publication details
Uecker, Marc; Linnhoff, Clemens [Supervisor]; Kuijper, Arjan [Supervisor]

Development of a Method for the Fusion of Environment Perception Sensor Data for an Automated Vehicle

2021

Darmstadt, TU, Master Thesis, 2021

Autonomous driving is currently one of the most anticipated future technologies of the automotive world, and researchers from all over the world are dedicated to this task. In the same pursuit, the aDDa project at TU Darmstadt is a collaboration of researchers and students, focused on jointly engineering a car into a fully autonomous vehicle. As such, the aDDa research vehicle is outfitted with a wide array of sensors for environment perception. Within the scope of the aDDa project, this thesis covers the fusion of data from LIDAR, RADAR and camera sensors into a unified environment model. Specifically, this work focuses on providing real-time environment perception, including fusion and interpretation of data from different sensors using only on-board hardware resources. The developed method is a software pipeline, consisting of an analytic low-level sensor fusion stage, a 3D semantic segmentation model based on deep learning, and analytical clustering and tracking methods, as well as a proof-of-concept for estimating drivable space. This method was designed to maximize robustness, by minimizing the influence of the used machine learning approach on the reliability of obstacle detection. The sensor fusion pipeline runs in real-time with an output frequency of 10 Hz, and a pipeline delay of 120 to 190 milliseconds in the encountered situations on public roads. An evaluation of several scenarios shows that the developed system can reliably detect a target vehicle in a variety of real-world situations. The full contributions of this work not only include the development of a sensor fusion pipeline, but also methods for sensor calibration, as well as a novel method for generating training data for the used machine learning approach. In contrast to existing manual methods of data annotation, this work presents a scalable solution for annotating real-world sensor recordings to generate training data for 3D machine perception approaches for autonomous driving.

Show publication details
Jestädt, Sebastian; Kuijper, Arjan [1. Gutachten]; Mukhopadhyay, Anirban [2. Gutachten]

Development of an AI Web Application to Train Radiologists in Deep Learning

2021

Darmstadt, TU, Master Thesis, 2021

In current news, the topic of AI is more present than ever. Accordingly, new studies on the subject are constantly being published. Similarly, the topic is also gaining more and more significance in digital healthcare. However, not everybody is prepared to work with the new technology. Based on this, a web application should be developed in this work to make radiologists “AI-ready”. For this reason, the state of the art is first discussed by presenting studies that deal with AI and radiologists. These illustrate the current state of knowledge and how radiologists see the future with AI. Especially, is shown which methods already exist to prepare them for this new technology. In the second part, currently existing web applications are presented and critically evaluated. Also included a work that developed a neural network for predicting pneumothorax disease. Next, a survey is conducted within the thesis to provide an overview of how easy it is for entry-level enthusiastic radiologists to learn about AI. Based on the findings, an analysis is carried out, which underlines the need for a practical solution for radiologists to be prepared for the daily routine with AI in the future. Then the concept is described how the application should look like to cover all requirements. Afterwards, the result of the implementation is presented. For this purpose, screenshots of developed application are shown and explanations are given how it was implemented. Subsequently, a user study of the application was also conducted to evaluate the result. The resulting outcomes are also presented. Finally, we critically reflect on whether the requirements have been met and whether the developed application serves its purpose. Furthermore, an outlook is given in which areas improvements can still be made.

Show publication details

Efficient Scheduling of Scientific Workflow Actions in the Cloud Based on Required Capabilities

2021

Data Management Technologies and Applications. Revised Selected Papers

International Conference on Data Management Technologies and Applications (DATA) <9, 2020, online>

Communications in Computer and Information Science
1446

Distributed scientific workflow management systems processing large data sets in the Cloud face the following challenges: (a) workflow tasks require different capabilities from the machines on which they run, but at the same time, the infrastructure is highly heterogeneous, (b) the environment is dynamic and new resources can be added and removed at any time, (c) scientific workflows can become very large with hundreds of thousands of tasks, (d) faults can happen at any time in a distributed system. In this paper, we present a software architecture and a capability-based scheduling algorithm that cover all these challenges in one design. Our architecture consists of loosely coupled components that can run on separate virtual machines and communicate with each other over an event bus and through a database. The scheduling algorithm matches capabilities required by the tasks (e.g. software, CPU power, main memory, graphics processing unit) with those offered by the available virtual machines and assigns them accordingly for processing. Our approach utilises heuristics to distribute the tasks evenly in the Cloud. This reduces the overall run time of workflows and makes efficient use of available resources. Our scheduling algorithm also implements optimisations to achieve a high scalability. We perform a thorough evaluation based on four experiments and test if our approach meets the challenges mentioned above. The paper finishes with a discussion, conclusions, and future research opportunities. An implementation of our algorithm and software architecture is publicly available with the open-source workflow management system “Steep”.

Show publication details
Staschewski, Tim; Kuijper, Arjan [1. Gutachten]; Appl, Stefan [2. Gutachten]

Entwicklung eines automatischen Laser-Kamera-Kalibrierungsalgorithmus auf komplexen Oberflächen

2021

Darmstadt, TU, Master Thesis, 2021

Show publication details
Jähnig, Alina; Schulz, Stefan M. [Erstprüfer]; Chodan, Wencke [Zweitprüferin]

Erfassung physiologischer Parameter im Kontext zur Erkennung von Stressreaktionen und Stressursachen von Pflegekräften in der Palliativmedizin

2021

Würzburg, Univ., Master Thesis, 2021

Stress am Arbeitsplatz ist häufig Grund für Krankheitsausfälle, Absentismus und Burnout. Die objektive Stressmessung durch physiologische Parameter stellt eine unauffällige und aussagekräftige Methode dar, Stressreaktionen zu ermitteln und entsprechenden Folgen vorzubeugen. Gleichzeitig ist die Identifizierung von Stressoren im Kontext wichtig, damit gezielt Interventionen entwickelt werden können. Im Vergleich zu anderen Berufsgruppen sind Pflegekräfte besonders oft krankgeschrieben. Ziel der vorliegenden Arbeit ist deshalb, die Stressreaktionen von Pflegekräften in der Palliativmedizin zu messen, ihre Belastung zu ermitteln und Stressursachen abzuleiten. Das Tragen einer Smartwatch während der Schicht ermöglicht das kontinuierliche Erfassen physiologischer Parameter; das Drücken eines Knopfs an der Seite der Smartwatch zeichnet unmittelbar subjektiv empfundenen Stress auf. Subjektive Stressbelastung und Burnout-Gefährdung der Probanden werden zu Beginn der Studie jeweils mit einem Fragebogen erfasst. Regelmäßig erhobene Ecological Momentary Assessments (EMAs) erlauben das Verfolgen des subjektiven Stresslevels während der Arbeit. NFC-Tags an verschiedenen Stellen der Palliativstation ermöglichen die Zuordnung von erlebtem Stress im Kontext. Die Ergebnisse zeigen, dass sich mithilfe von physiologischen Parametern Stressreaktionen erkennen, Unterschiede in der Burnout-Gefährdung feststellen und subjektive Stresslevel statistisch signifikant hervorsagen lassen. Die praktische Relevanz der Stressvorhersage ist allerdings noch eingeschränkt. Eine Kombination aus dem Auslesen der NFC-Tags und der Ergebnisse der Fragebögen erwies sich als nützlich, um „patienten- und angehörigenassoziierten Tätigkeiten“ sowie „Arbeitsorganisation und -struktur“ als Stressoren in Patientenzimmern sowie „Dienstübergaben“ als Stressoren zu erkennen. Individuelle Unterschiede in der stressauslösenden Wirkung der genannten Stressoren sind festzustellen und sollten in der Implementierung der Interventionen beachtet werden. Die Ergebnisse des Stress-Fragebogens ergeben, dass im Durchschnitt die Stressbelastung der Pflegekräfte im unteren Drittel der eingesetzten Skala liegt. Aus dem Maslach-Burnout Inventory geht hervor, dass eine von sechs Pflegekräften Burnout-gefährdet ist. Zukünftige Forschung ist notwendig, um auf subjektive Messungen zu verzichten und dennoch valide Ergebnisse zu erzielen. Eine Möglichkeit könnte maschinelles Lernen darstellen, das sich einer Automatisierung von Erfassung, Analyse und Interpretation objektiver physiologischer Parameter widmet.

Show publication details

Executing Cyclic Scientific Workflows in the Cloud

2021

Journal of Cloud Computing

We present an algorithm and a software architecture for a cloud-based system that executes cyclic scientific workflows whose structure may change during run time. Existing approaches either rely on workflow definitions based on directed acyclic graphs (DAGs) or require workarounds to implement cyclic structures. In contrast, our system supports cycles natively, avoids workarounds, and as such reduces the complexity of workflow modelling and maintenance. Our algorithm traverses workflow graphs and transforms them iteratively into linear sequences of executable actions. We call these sequences process chains. Our software architecture distributes the process chains to multiple compute nodes in the cloud and oversees their execution. We evaluate our approach by applying it to two practical use cases from the domains of astronomy and engineering. We also compare it with two existing workflow management systems. The evaluation demonstrates that our algorithm is able to execute dynamically changing workflows with cycles and that design and maintenance of complex workflows is easier than with existing solutions. It also shows that our software architecture can run process chains on multiple compute nodes in parallel to significantly speed up the workflow execution. An implementation of our algorithm and the software architecture is available with the Steep Workflow Management System that we released under an open-source license. The resources for the first practical use case are also available as open source for reproduction.

Show publication details
Huber, Marco; Kuijper, Arjan [1. Review]; Terhörst, Philipp [2. Review]

Explainable Face Image Quality Assessment

2021

Darmstadt, TU, Master Thesis, 2021

The high performance of today’s face recognition systems is driven by the quality of the used samples. To ensure a high quality of face images enrolled in a system, face image quality assessment is performed on these images. However, if an image is rejected due to low quality it is not obvious to the user why. Showing what led to the low quality is useful to increase trust and transparency in the system. Previous work has never addressed the explainability of their quality estimation, but only the explainability of their face recognition approaches. In this work, we propose a gradient-based method that explains which pixels contribute to the overall image quality and which do not. By adding a quality node to the end of the model, we can calculate quality-dependent gradients and visualize them. The qualitative experiments are conducted on three different datasets and we also propose a general framework for quantitative analysis of face image quality saliency maps. With our method, we can assign quality values to individual pixels and provide a meaningful explanation of how face recognition models work and how they respond to face image quality impairments. Our method provides pixel-level explanations, requires no training, applies to any face recognition model, and also takes model-specific behavior into account. By explaining how the poor quality is caused, concrete instructions can be given to people who take pictures suitable for face recognition, face image standards can be adapted, and low-quality areas can be inpainted.

Show publication details

Extended Evaluation of the Effect of Real and Simulated Masks on Face Recognition Performance

2021

IET Biometrics

Face recognition is an essential technology in our daily lives as a contactless and convenient method of accurate identity verification. Processes such as secure login to electronic devices or identity verification at automatic border control gates are increasingly dependent on such technologies. The recent COVID‐19 pandemic has increased the focus on hygienic and contactless identity verification methods. The pandemic has led to the wide use of face masks, essential to keep the pandemic under control. The effect of mask‐wearing on face recognition in a collaborative environment is currently a sensitive yet understudied issue. Recent reports have tackled this by using face images with synthetic mask‐like face occlusions without exclusively assessing how representative they are of real face masks. These issues are addressed by presenting a specifically collected database containing three sessions, each with three different capture instructions, to simulate real use cases. The data are augmented to include previously used synthetic mask occlusions. Further studied is the effect of masked face probes on the behaviour of four face recognition systems—three academic and one commercial. This study evaluates both masked‐to‐non‐masked and masked‐to‐masked face comparisons. In addition, real masks in the database are compared with simulated masks to determine their comparative effects on face recognition performance.

Show publication details
Rus, Silvia; Fellner, Dieter W. [Referent]; Augusto, Juan Carlos [Referent]; Kuijper, Arjan [Referent]

Extending the Design Space of E-Textile Assistive Smart Environment Applications

2021

Darmstadt, TU, Diss., 2021

The thriving field of Smart Environments has allowed computing devices to gain new capabilities and develop new interfaces, thus becoming more and more part of our lives. In many of these areas it is unthinkable to renounce to the assisting functionality such as e.g. comfort and safety functions during driving, safety functionality while working in an industrial plant, or self-optimization of daily activities with a Smartwatch. Adults spend a lot of time on flexible surfaces such as in the office chair, in bed or in the car seat. These are crucial parts of our environments. Even though environments have become smarter with integrated computing gaining new capabilities and new interfaces, mostly rigid surfaces and objects have become smarter. In this thesis, I build on the advantages flexible and bendable surfaces have to offer and look into the creation process of assistive Smart Environment applications leveraging these surfaces. I have done this with three main contributions. First, since most Smart Environment applications are built-in into rigid surfaces, I extend the body of knowledge by designing new assistive applications integrated in flexible surfaces such as comfortable chairs, beds, or any type of soft, flexible objects. These developed applications offer assistance e.g. through preventive functionality such as decubitus ulcer prevention while lying in bed, back pain prevention while sitting on a chair or emotion detection while detecting movements on a couch. Second, I propose a new framework for the design process of flexible surface prototypes and its challenges of creating hardware prototypes in multiple iterations, using resources such as work time and material costs. I address this research challenge by creating a simulation framework which can be used to design applications with changing surface shape. In a first step I validate the simulation framework by building a real prototype and a simulated prototype and compare the results in terms of sensor amount and sensor placement. Furthermore, I use this developed simulation framework to analyse the influence it has on an application design if the developer is experienced or not. Finally, since sensor capabilities play a major role during the design process, and humans come often in contact with surfaces made of fabric, I combine the integration advantages of fabric and those of capacitive proximity sensing electrodes. By conducting a multitude of capacitive proximity sensing measurements, I determine the performance of electrodes made by varying properties such as material, shape, size, pattern density, stitching type, or supporting fabric. I discuss the results from this performance evaluation and condense them into e-textile capacitive sensing electrode guidelines, applied exemplary on the use case of creating a bedsheet for breathing rate detection.

Show publication details
Purnapatra, Sandip; Smalt, Nic; Bahmani, Keivan; Das, Priyanka; Yambay, David; Mohammadi, Amir; George, Anjith; Bourlai, Thirimachos; Marcel, Sébastien; Schuckers, Stephanie; Fang, Meiling; Damer, Naser; Boutros, Fadi; Kuijper, Arjan; Kantarci, Alperen; Demir, Başar; Yildiz, Zafer; Ghafoory, Zabi; Dertli, Hasan; Ekenel, Hazım Kemal; Vu, Son; Christophides, Vassilis; Dashuang, Liang; Guanghao, Zhang; Zhanlong, Hao; Junfu, Liu; Yufeng, Jin; Liu, Samo; Huang, Samuel; Kuei, Salieri; Singh, Jag Mohan; Ramachandra, Raghavendra

Face Liveness Detection Competition (LivDet-Face) - 2021

2021

IJCB 2021. IEEE/IARP International Joint Conference on Biometrics

IEEE International Joint Conference on Biometrics (IJCB) <2021, online>

Liveness Detection (LivDet)-Face is an international competition series open to academia and industry. The competition’s objective is to assess and report state-of-the-art in liveness / Presentation Attack Detection (PAD) for face recognition. Impersonation and presentation of false samples to the sensors can be classified as presentation attacks and the ability for the sensors to detect such attempts is known as PAD. LivDet-Face 2021 * will be the first edition of the face liveness competition. This competition serves as an important benchmark in face presentation attack detection, offering (a) an independent assessment of the current state of the art in face PAD, and (b) a common evaluation protocol, availability of Presentation Attack Instruments (PAI) and live face image dataset through the Biometric Evaluation and Testing (BEAT) platform. The competition can be easily followed by researchers after it is closed, in a platform in which participants can compare their solutions against the LivDet-Face winners.

Show publication details
Kessler, Roman; Henniger, Olaf; Busch, Christoph

Fingerprints, forever young ?

2021

Pattern Recognition. ICPR International Workshops and Challenges. Proceedings

International Conference on Pattern Recognition (ICPR) <25, 2020, Online>

Lecture Notes in Computer Science (LNCS)
12664

In this study we analyzed longitudinal fingerprint data of 20 data subjects, acquired over a time span of up to 12 years. Using hierarchical linear modeling, we aimed to delineate mated similarity scores as a function of fingerprint quality and of the time interval between reference and probe images. Our results did not reveal effects on mated similarity scores caused by an increasing time interval across subjects, but rather individual effects on mated similarity scores. The results are in line with the general assumption that the fingerprint as a biometric characteristic and the features extracted from it do not change over the adult life span. However, it contradicts several related studies that reported noticeable template ageing effects. We discuss why different findings regarding ageing of references in fingerprint recognition systems were made.

Show publication details
Callmer, Pia; Kuijper, Arjan [1. Review]; Noll, Matthias [2. Review]

Fluid Detection and Analysis in Ultrasonic Image Data Applying Artificial Intelligence

2021

Darmstadt, TU, Bachelor Thesis, 2021

The fast recognition and treatment of internal injuries can have an essential impact on the patient’s disease process. The application of an ultrasound device and the associated FAST protocol have been established standards used for decades for the initial detection of free fluids. Especially with ultrasound, the acquisition and evaluation of ultrasound data is very dependent on the experience of the ultrasound operator. Therefore, the automatic detection of fluids would be especially important for less experienced operators. In this thesis, the feasibility of artificial intelligence (AI) methods to automatically detect fluids in ultrasound image data is addressed. It is also evaluated how accurate the methods can distinguish between free and nonfree fluids. In particular, different variations of the state-of-the-art U-net and its effect on the results are investigated. The success of supervised learning methods is highly dependent on the design of the method, such as the architecture of the neural network, but also on the given input data. AI methods, such as the U-net, are characterized by the fact that they can be applied to different problems. Overall, the study shows that the U-net architecture provides promising results for this problem on the given data set. The potential as well as the weaknesses of the selected methods are discussed in detail in the following.

Show publication details
Spiller, Noémie Catherine Hélène; Kuijper, Arjan [Betreuer]; Damer, Naser [Betreuer]

Generalization of Face Morphing Attack Detection

2021

Darmstadt, TU, Bachelor Thesis, 2021

Researchers found that faces can be morphed together to create a new face image which resembles both original identities. These morphs can be submitted for e.g. a passport that both persons are able to use since both match the morphed image. The hazardous vulnerability of face recognition systems (FRS) towards morphing attacks can be encountered with automatic Morphing Attack Detection (MAD). The aim of this work is to evaluate a new approach to face MAD and comparing it to state-of-the-art baselines on a created digital and print and scan (P&S) database. The database consists of 276 morphs and 364 non-morphed (bona fide) images and was entirely printed and scanned. From the databases, five different splits were formed to train and test the methods, namely digital, P&S, mixed, train digital, test P&S and train P&S, test digital. The novel pixel-wise supervised MAD solution (PW-MAD) differs from traditional supervised solutions as it outputs a feature map classifying each pixel/patch to either bona fide or attack. This map is computed to a binary score by a fully connected layer with sigmoid activation. PW-MAD yields good results across all intra and cross-database experiments and outperforms all baselines, showing good generalizability and reaching 6.45%, 8.13%, 8.87%, 12.19%, and 6.45% BPCER at APCER=10% for the digital, P&S, mixed, train digital, test P&s and train P&S, test digital, respectively

Show publication details

Generalization of Fitness Exercise Recognition from Doppler Measurements by Domain-Adaption and Few-Shot Learning

2021

Pattern Recognition. ICPR International Workshops and Challenges. Proceedings, Part IV

International Conference on Pattern Recognition (ICPR) <25, 2020, Online>

Lecture Notes in Computer Science (LNCS)
12664

In previous works, a mobile application was developed using an unmodified commercial smartphone to recognize whole-body exercises. The working principle was based on the ultrasound Doppler sensing with the device built-in hardware. Applying such a lab environment trained model on realistic application variations causes a significant drop in performance, and thus decimate its applicability. The reason of the reduced performance can be manifold. It could be induced by the user, environment, and device variations in realistic scenarios. Such scenarios are often more complex and diverse, which can be challenging to anticipate in the initial training data. To study and overcome this issue, this paper presents a database with controlled and uncontrolled subsets of fitness exercises. We propose two concepts to utilize small adaption data to successfully improve model generalization in an uncontrolled environment, increasing the recognition accuracy by two to six folds compared to the baseline for different users.

Show publication details
Guski, Florian; Kuijper, Arjan [1. Review]; Schufrin, Marija [2. Review]

Generalizing a Clustering of Units Visualization for Personal Cyber Analytics

2021

Darmstadt, TU, Master Thesis, 2021

Personal cyber security and privacy are topics that can often be difficult to understand for non-experts. Information Visualization is a method to support understanding data and can therefore be used in this field to make those topics more accessible. Nonetheless, the effectivity of visualizations varies greatly depending on the application context. In this thesis, a prototype that originally was designed for visualizing firewall logs was generalized to make it applicable for various personal cyber analytics use cases. To achieve that, the visualization was analyzed and reverse engineered and afterwards a generalized paradigm was designed. This paradigm was implemented and a user study was conducted to evaluate the implemented paradigm and validate it as a generalization for personal cyber analytics. The results showed that the visualization is able to support users to deal with personal cyber data and enables them to gain more knowledge. Moreover, we presented a varied set of possible application fields for the visualization.

Show publication details
Piatkowski, Nico; Mueller-Roemer, Johannes; Hasse, Peter; Bachorek, Adam; Werner, Tim; Birnstill, Pascal; Morgenstern, Andreas; Stobbe, Lutz

Generative Machine Learning for Resource-Aware 5G and IoT Systems

2021

2021 IEEE International Conference on Communications Workshops (ICC Workshops). Proceedings

IEEE International Conference on Communications Workshops (ICC Workshop) <2021, Online>

Extrapolations predict that the sheer number of Internet-of-Things (IoT) devices will exceed 40 billion in the next five years. Hand-crafting specialized energy models and monitoring sub-systems for each type of device is error prone, costly, and sometimes infeasible. In order to detect abnormal or faulty behavior as well as inefficient resource usage autonomously, it is of tremendous importance to endow upcoming IoT and 5G devices with sufficient intelligence to deduce an energy model from their own resource usage data. Such models can in-turn be applied to predict upcoming resource consumption and to detect system behavior that deviates from normal states. To this end, we investigate a special class of undirected probabilistic graphical model, the so-called integer Markov random fields (IntMRF). On the one hand, this model learns a full generative probability distribution over all possible states of the system—allowing us to predict system states and to measure the probability of observed states. On the other hand, IntMRFs are themselves designed to consume as less resources as possible—e.g., faithful modelling of systems with an exponentially large number of states, by using only 8-bit unsigned integer arithmetic and less than 16KB memory. We explain how IntMRFs can be applied to model the resource consumption and the system behavior of an IoT device and a 5G core network component, both under various workloads. Our results suggest, that the machine learning model can represent important characteristics of our two test systems and deliver reasonable predictions of the power consumption.

Show publication details

Graph Matching Survey for Medical Imaging: on the Way to Deep Learning

2021

Methods

The interest on graph matching has not stopped growing since the late seventies. The basic idea of graph matching consists of generating graph representations of different data or structures and compare those representations by searching correspondences between them. There are manifold techniques that have been developed to find those correspondences and the choice of one or another depends on the characteristics of the application of interest. These applications range from pattern recognition (e.g. biometric identification) to signal processing or artificial intelligence. One of the aspects that make graph matching so attractive is its ability to facilitate data analysis, and medical imaging is one of the fields that can benefit from this in a greater extent. The potential of graph matching to find similarities and differences between data acquired at different points in time shows its potential to improve diagnosis, follow-up of human diseases or any other of the clinical scenarios that require comparison between different datasets. In spite of the large amount of papers that were published in this field to the date there is no survey paper of graph matching for clinical applications. This survey aims to fill this gap.

Show publication details
Ramachandra, Raghavendra; Damer, Naser; Raja, Kiran; Rattani, Ajita

Guest Editorial: WACV 2020 - Presentation Attacks on Biometric Systems

2021

IET Biometrics

IEEE Winter Conference on Applications of Computer Vision (WACV) <2020, Online>

Welcome message from the proceedings WACV 2020

Show publication details
Gottschalk, Maximilian; Kuijper, Arjan [1. Review]; Jansen, Nils [2. Review]

Handling of Backgrounds in Furniture Recognition with Neural Networks

2021

Darmstadt, TU, Master Thesis, 2021

This work explores different object segmentation methods for the use-case of a visual furniture recommender system. The two main contributions are a novel method of synthetic furniture image generation to be used for semantic segmentation for different furniture classes and foreground-background separation. Secondly, an evaluation of the best performing segmentation network architectures in a recommender system. The synthetic dataset proves to be a suitable way to train image segmentation and to be a fitting way to represent real world data of furniture. The setup of the pipeline for generating the data provides a scalable and diverse data set through numerous augmentations. Evaluating the best performing segmentation network architectures in a visual furniture recommender system shows that significant improvements are achieved for recommendations compared to recommendations based on the recommender system without background subtraction. Recommendations, based on background subtraction, are chosen two to three times more often.

Show publication details
Koch, Vitali; Albrecht, Moritz H.; Gruenewald, Leon D.; Yel, Ibrahim; Eichler, Katrin; Gruber-Rouh, Tatjana; Hammerstingl, Renate; Burck, Iris; Wichmann, Julian L.; Alizadeh, Leona S.; Vogl, Thomas J.; Lenga, Lukas; Wesarg, Stefan; Martin, Simon S.; Mader, Christoph; Dimitrova, Mirela; D'Angelo, Tommaso; Booz, Christian

Impact of Intravenously Injected Contrast Agent on Bone Mineral Density Measurement in Dual-Source Dual-Energy CT

2021

Academic Radiology

Purpose: To assess the influence of intravenously injected contrast agent on bone mineral density (BMD) assessment in dual-source dual-energy CT. Methods: This retrospective study included 1,031 patients (mean age, 53 ± 7 years; 519 women) who had undergone third-generation dual-source dual-energy CT in context of tumor staging between January 2019 and December 2019. Dedicated postprocessing software based on material decomposition was used for phantomless volumetric BMD assessment of trabecular bone of the lumbar spine. Volumetric trabecular BMD values derived from unenhanced and contrast-enhanced portal venous phase were compared by calculating correlation and agreement analyses using Pearson product-moment correlation, linear regression, and Bland-Altman plots. Results: Mean BMD values were 115.53 ± 37.23 and 116.10 ± 37.78 mg/cm3 in unenhanced and contrast-enhanced dual-energy CT series, respectively. Values from contrast-enhanced portal venous phase differed not significantly from those of the unenhanced phase (p = 0.44) and showed high correlation (r = 0.971 [95% CI, 0.969–0.973]) with excellent agreement in Bland-Altman plots. Mean difference of the two phases was 0.61 mg/cm3 (95% limits of agreement, -17.14 and 18.36 mg/cm3). Conclusion: Portal venous phase dual-source dual-energy CT allows for accurate opportunistic BMD assessment of trabecular bone of the lumbar spine compared to unenhanced imaging. Therefore, dual-source CT may provide greater flexibility regarding BMD assessment in clinical routine and reduce radiation exposure by avoiding additional osteodensitometry examinations, as contrast-enhanced CT scans in context of tumor staging are increasingly performed in dual-energy mode.

Show publication details
Baumgartl, Tom; Petzold, Markus; Wunderlich, Marcel; Höhn, Markus; Archambault, Daniel; Lieser, M.; Dalpke, Alexander; Scheithauer, Simone; Marschollek, Michael; Eichel, V. M.; Mutters, Nico T.; Landesberger, Tatiana von

In Search of Patient Zero: Visual Analytics of Pathogen Transmission Pathways in Hospitals

2021

IEEE Transactions on Visualization and Computer Graphics

Pathogen outbreaks (i.e., outbreaks of bacteria and viruses) in hospitals can cause high mortality rates and increase costs for hospitals significantly. An outbreak is generally noticed when the number of infected patients rises above an endemic level or the usual prevalence of a pathogen in a defined population. Reconstructing transmission pathways back to the source of an outbreak – the patient zero or index patient – requires the analysis of microbiological data and patient contacts. This is often manually completed by infection control experts. We present a novel visual analytics approach to support the analysis of transmission pathways, patient contacts, the progression of the outbreak, and patient timelines during hospitalization. Infection control experts applied our solution to a real outbreak of Klebsiella pneumoniae in a large German hospital. Using our system, our experts were able to scale the analysis of transmission pathways to longer time intervals (i.e., several years of data instead of days) and across a larger number of wards. Also, the system is able to reduce the analysis time from days to hours. In our final study, feedback from twenty-five experts from seven German hospitals provides evidence that our solution brings significant benefits for analyzing outbreaks.

Show publication details
Wulf, Conrad; Kraft, Dimitri [Betreuer]; Urban, Bodo [Erstgutachter]; Lukas, Uwe von [Zweitgutachter]

Interaktive Vereinheitlichung von unterschiedlichen Parametern durch eine KI-basierte Ontologie

2021

Rostock, Univ., Master Thesis, 2021

Die vorliegende Arbeit präsentiert ein System zur Vereinheitlichung von Data Sets aus heterogenen Quellen. Mittels Techniken des Ontology-Matchings erstellt dieses eine Ontologie der Beziehungen zwischen Data Sets, welche in einem Kreis-Layout visualisiert wird und bearbeitet werden kann. Anschließend benennt das System Variablen der Data Sets entsprechend der Ontologie um. Um die Eignung unterschiedlicher Matching-Techniken für die Erstellung der Ontologie zu untersuchen, werden mehrere Ansätze implementiert. Darunter sind Methoden auf Basis von Zeichenkettenvergleichen, Word-Embeddings mit GloVe und BERT und Techniken, die Instanzdaten berücksichtigen. Anhand einer aus realen Daten erstellten Ground-Truth werden diese hinsichtlich F1-Score, Precision, Recall und Average-Precision evaluiert. Alle Ansätze schneiden besser ab als ein Weighted Guessing Classifier und eignen sich daher grundlegend zur Unterstützung der Vereinheitlichung. Hinsichtlich des F1-Score und Recall schneidet eine Methode auf Basis erlernter GloVe-Embeddings am besten ab, ihr F1-Score von 0.387 und Recall von 0.319 lassen jedoch Raum für Verbesserungen. Eine Methode auf Basis von Zeichenkettenvergleichen liefert mit einem Wert von 0.358 das zweitbeste Ergebnis im Hinblick auf den F1-Score und mit einem Wert von 0.739 die beste Precision. Die Berücksichtigung von Instanzdaten verbessert nur bei einer von vier implementierten Methoden die Leistung hinsichtlich Average Precision. Dadurch liefert der Ansatz, welcher Zeichenkettenvergleiche mit der Berücksichtigung von Instanzdaten kombiniert, mit einem Wert von 0.391 die beste Average Precision aller verglichenen Techniken.

Show publication details
Cibulski, Lena; Mitterhofer, Hubert

Interaktive Visualisierung: Durchblick beim Produktdesign

2021

wt Werkstattstechnik online

In der Produktentwicklung und Produktionsplanung treten häufig Konflikte zwischen verschiedenen Zielgrößen auf. So lassen sich manche Zielgrößen nicht optimieren, ohne bei anderen Kompromisse eingehen zu müssen. Das Visualisierungs-Tool „Paved“ (Pareto Front Visualization for Engineering Design) hilft, Unterschiede zwischen Alternativen besser zu verstehen und so tragfähigere Entscheidungen zu treffen.

Show publication details

Iris Presentation Attack Detection by Attention-based and Deep Pixel-wise Binary Supervision Network

2021

IJCB 2021. IEEE/IARP International Joint Conference on Biometrics

IEEE International Joint Conference on Biometrics (IJCB) <2021, online>

Iris presentation attack detection (PAD) plays a vital role in iris recognition systems. Most existing CNN-based iris PAD solutions 1) perform only binary label supervision during the training of CNNs, serving global information learning but weakening the capture of local discriminative features, 2) prefer the stacked deeper convolutions or expert-designed networks, raising the risk of overfitting, 3) fuse multiple PAD systems or various types of features, increasing difficulty for deployment on mobile devices. Hence, we propose a novel attention-based deep pixel-wise bi-nary supervision (A-PBS) method. Pixel-wise supervision is first able to capture the fine-grained pixel/patch-level cues. Then, the attention mechanism guides the network to automatically find regions that most contribute to an accurate PAD decision. Extensive experiments are performed on LivDet-Iris 2017 and three other publicly available databases to show the effectiveness and robustness of proposed A-PBS methods. For instance, the A-PBS model achieves an HTER of 6.50% on the IIITD-WVU database outperforming state-of-the-art methods.

Show publication details
Fellner, Dieter W. [Hg.]; Sihn, Wilfried [Betreuer]

Jahresbericht 2020

2021

Show publication details
Krumb, Henry John; Kuijper, Arjan [1. Review]; Mukhopadhyay, Anirban [2. Review]

Learning to Track Permanent Magnets by Example

2021

Darmstadt, TU, Master Thesis, 2021

Minimally invasive surgeries require navigation modalities without the requirement for a free line-of-sight. The current gold standard for minimally invasive endovascular interventions is fluoroscopic guidance, which provides visual feedback to the interventional radiologist. To minimize radiation exposure, previous work proposes to complement fluoroscopy by Electromagnetic Tracking (EMT) as a second navigation modality. However, EMT systems require tethered sensors, which are expensive and difficult in the handling. This thesis investigates Permanent Magnet Tracking (PMT) as a potential wireless and inexpensive alternative to EMT for intra-operative navigation. Instead of locating a tethered sensor inside of the patient, a strong permanent magnet is attached to a surgical instrument and tracked by external sensors. These wired sensors on the exterior sense the magnet inside the patient, enabling wireless navigation at a low cost. The major challenge of this approach is the localization of the permanent magnet, merely based on a few simultaneous sensor readings. In this thesis, an inexpensive and reproducible PMT setup is proposed. This setup uses four magnetoresistive sensors to track a cylindrical Neodymium magnet in up to six degrees of freedom. Unlike most of its predecessors, this tracker uses a data-driven approach to localize the permanent magnet. In particular, neural networks are employed as general function approximators, which are trained to deduce magnet positions from sensor readings. Hand-collected data are fed to the neural networks to learn a mapping from magnetic field measurements to positions in 3D space, together with orientation around the x-, y- and z-axes. In an experimental phase, different optimizations for the data driven approach are proposed. These experimental results suggest that incorporating temperature data, captured by the sensors, increases prediction accuracy. Similarly, converting input points to spherical coordinates increases accuracy. However, augmenting the data by either simulated or interpolated data does not yield satisfying results.

Show publication details

LFPeers: Temporal Similarity Search in Covid-19 Data

2021

EuroVA 2021

International EuroVis Workshop on Visual Analytics (EuroVA) <2021, Online>

While there is a wide variety of visualizations and dashboards to help understand the data of the Covid-19 pandemic, hardly any of these support important analytical tasks, especially of temporal attributes. In this paper, we introduce a general concept for the analysis of temporal and multimodal data and the system LFPeers that applies this concept to the analysis of countries in a Covid-19 dataset. Our concept divides the analysis in two phases: a search phase to find the most similar objects to a target object before a time point t0, and an exploration phase to analyze this subset of objects after t0. LFPeers targets epidemiologists and the public who want to learn from the Covid-19 pandemic and distinguish successful and ineffective measures.

Show publication details

Linoc: A Prototyping Platform for Capacitive and Passive Electrical Field Sensing

2021

Proceedings of the 10th International Conference on Sensor Network

International Conference on Sensor Networks (SENSORNETs) <10, 2021, online>

In this paper the Linoc prototyping toolkit is presented. It is a sensor toolkit that focuses on fast prototypingof sensor systems, especially on capacitive ones. The toolkit is built around two capacitive and two ElectricPotential Sensing (EPS) groups providing unobtrusive proximity detection in the field of Human ComputerInterface (HCI). The toolkits focus lies on its usability and connectivity in order to be adapted in future researchand novel use cases. A common obstacle in the beginning of a project is the time required to familiarize withpresent tools and systems, before the actual project can be attended to. Another obstacle while tackling newtasks is the actual physical connection of sensors to the processing unit. This situation can be even worsedue to dependencies on previous work, most of the times not fully documented and missing knowledge evenif the the original designer is involved. Good toolkits can help to overcome this problem by providing alayer of abstraction and allowing to work on a higher level. If the toolkit however requires too much time tofamiliarize or behaves too restrictive, its goal has been missed and no benefits are generated. To assess thequality of the Linoc prototyping toolkit, it was evaluated in terms of three different aspects: demonstration,usage and technical performance. The usage study found good reception, a fast learning curve and an interestto use the toolkit in the future. Technical benchmarks for the capacitive sensors show a detectable range equalto its predecessors and several operational prototypes prove that the toolkit can actually be used in projects.

Show publication details

MAAD-Face: A Massively Annotated Attribute Dataset for Face Images

2021

IEEE Transactions on Information Forensics and Security

Soft-biometrics play an important role in face biometrics and related fields since these might lead to biased performances, threaten the user’s privacy, or are valuable for commercial aspects. Current face databases are specifically constructed for the development of face recognition applications. Consequently, these databases contain a large number of face images but lack in the number of attribute annotations and the overall annotation correctness. In this work, we propose a novel annotation-transfer pipeline that allows to accurately transfer attribute annotations from multiple source datasets to a target dataset. The transfer is based on a massive attribute classifier that can accurately state its prediction confidence. Using these prediction confidences, a high correctness of the transferred annotations is ensured. Applying this pipeline to the VGGFace2 database, we propose the MAAD-Face annotation database. It consists of 3.3M faces of over 9k individuals and provides 123.9M attribute annotations of 47 different binary attributes. Consequently, it provides 15 and 137 times more attribute annotations than CelebA and LFW. Our investigation on the annotation quality by three human evaluators demonstrated the superiority of the MAAD-Face annotations over existing databases. Additionally, we make use of the large number of high-quality annotations from MAAD-Face to study the viability of soft-biometrics for recognition, providing insights into which attributes support genuine and imposter decisions. The MAAD-Face annotations dataset is publicly available.

Show publication details
Wiesmaier, Luca; Kuijper, Arjan [1. Review]; Yeste Magdaleno, Javier [2. Review]

Material Segmentation for Visual Aware Recommender Systems

2021

Darmstadt, TU, Master Thesis, 2021

People nowadays have the possibility to get recommendations for almost anything based on things they previously purchased or liked. These recommendations are often based on categories, simple colors, or other user interactions. This work presents a different approach by using precise material recognition to recommend furniture as well as clothes. These so called visual aware recommender systems are fairly unknown and have only recently gained attention. A visual aware recommender system extracts visual features from its input and uses these features to recommend accordingly. One of the biggest advantages is that these systems do not suffer from the cold start problem that many modern recommender systems have, since they do not require any other information except the visual input. In order to use material information for recommendations, precise semantic segmentation is required. Therefore, the two best performing state-of-the-art neural networks for this task are compared and evaluated, while the better model is then used in the recommender system. Performance is demonstrated by using not just one approach, but two approaches. One uses a user study to evaluate the performance gain compared to a recommender system without material recognition, and the other uses expert data known to be true to evaluate the total precision on a real live task. Both of them confirm the assumption that material recognition not only works, but also substantially improves recommendation performance especially on certain combinations.

Show publication details

MiDeCon: Unsupervised and Accurate Fingerprint and Minutia Quality Assessment based on Minutia Detection Confidence

2021

IJCB 2021. IEEE/IARP International Joint Conference on Biometrics

IEEE International Joint Conference on Biometrics (IJCB) <2021, online>

An essential factor to achieve high accuracies in finger-print recognition systems is the quality of its samples. Previous works mainly proposed supervised solutions based on image properties that neglects the minutiae extraction process, despite that most fingerprint recognition techniques are based on detected minutiae. Consequently, a fingerprint image might be assigned a high quality even if the utilized minutia extractor produces unreliable information. In this work, we propose a novel concept of assessing minutia and fingerprint quality based on minutia detection confidence (MiDeCon). MiDeCon can be applied to an arbitrary deep learning based minutia extractor and does not require quality labels for learning. We propose using the detection reliability of the extracted minutia as its quality indicator. By combining the highest minutia qualities, MiDeCon also accurately determines the quality of a full fingerprint. Experiments are conducted on the publicly available databases of the FVC 2006 and compared against several baselines, such as NIST’s widely-used fingerprint image quality software NFIQ1 and NFIQ2. The results demonstrate a significantly stronger quality assessment performance of the proposed MiDeCon-qualities as related works on both, minutia- and fingerprint-level. The implementation is publicly available.

Show publication details
Zhang, Haoyu; Venkatesh, Sushma; Ramachandra, Raghavendra; Raja, Kiran; Damer, Naser; Busch, Christoph

MIPGAN - Generating Strong and High Quality Morphing Attacks Using Identity Prior Driven GAN

2021

IEEE Transactions on Biometrics, Behavior, and Identity Science

Face morphing attacks target to circumvent Face Recognition Systems (FRS) by employing face images derived from multiple data subjects (e.g., accomplices and malicious actors). Morphed images can be verified against contributing data subjects with a reasonable success rate, given they have a high degree of facial resemblance. The success of morphing attacks is directly dependent on the quality of the generated morph images. We present a new approach for generating strong attacks extending our earlier framework for generating face morphs. We present a new approach using an Identity Prior Driven Generative Adversarial Network, which we refer to as MIPGAN (Morphing through Identity Prior driven GAN). The proposed MIPGAN is derived from the StyleGAN with a newly formulated loss function exploiting perceptual quality and identity factor to generate a high quality morphed facial image with minimal artefacts and with high resolution. We demonstrate the proposed approach’s applicability to generate strong morphing attacks by evaluating its vulnerability against both commercial and deep learning based Face Recognition System (FRS) and demonstrate the success rate of attacks. Extensive experiments are carried out to assess the FRS’s vulnerability against the proposed morphed face generation technique on three types of data such as digital images, re-digitized (printed and scanned) images, and compressed images after re-digitization from newly generated MIPGAN Face Morph Dataset. The obtained results demonstrate that the proposed approach of morph generation poses a high threat to FRS.

Show publication details
Terhörst, Philipp; Kuijper, Arjan [1. Review]; Fellner, Dieter W. [2. Review]; Struc, Vitomir [3. Review]

Mitigating Soft-Biometric Driven Bias and Privacy Concerns in Face Recognition Systems

2021

Darmstadt, TU, Diss., 2021

Biometric verification refers to the automatic verification of a person’s identity based on their behavioural and biological characteristics. Among various biometric modalities, the face is one of the most widely used since it is easily acquirable in unconstrained environments and provides a strong uniqueness. In recent years, face recognition systems spread world-wide and are increasingly involved in critical decision-making processes such as finance, public security, and forensics. The growing effect of these systems on everybody’s daily life is driven by the strong enhancements in their recognition performance. The advances in extracting deeply-learned feature representations from face images enabled the high-performance of current face recognition systems. However, the success of these representations came at the cost of two major discriminatory concerns. These concerns are driven by soft-biometric attributes such as demographics, accessories, health conditions, or hairstyles. The first concern is about bias in face recognition. Current face recognition solutions are built on representation-learning strategies that optimize total recognition performance. These learning strategies often depend on the underlying distribution of soft-biometric attributes in the training data. Consequently, the behaviour of the learned face recognition solutions strongly varies depending on the individual’s soft-biometrics (e.g. based on the individual’s ethnicity). The second concern tackles the user’s privacy in such systems. Although face recognition systems are trained to recognize individuals based on face images, the deeply-learned representation of an individual contains more information than just the person’s identity. Privacy-sensitive information such as demographics, sexual orientation, or health status, is encoded in such representations. However, for many applications, the biometric data is expected to be used for recognition only and thus, raises major privacy issues. The unauthorized access of such individual’s privacy-sensitive information can lead to unfair or unequal treatment of this individual. Both issues are caused by the presence of soft-biometric attribute information in the face images. Previous research focused on investigating the influence of demographic attributes on both concerns. Consequently, the solutions from previous works focused on the mitigation of demographic-concerns only as well. Moreover, these approaches require computationally-heavy retraining of the deployed face recognition model and thus, are hardly-integrable into existing systems. Unlike previous works, this thesis proposes solutions to mitigating soft-biometric driven bias and privacy concerns in face recognition systems that are easily-integrable in existing systems and aim for more comprehensive mitigation, not limited to pre-defined demographic attributes. This aims at enhancing the reliability, trust, and dissemination of these systems. The first part of this work provides in-depth investigations on soft-biometric driven bias and privacy concerns in face recognition over a wide range of soft-biometric attributes. The findings of these investigations guided the development of the proposed solutions. The investigations showed that a high number of soft-biometric and privacy-sensitive attributes are encoded in face representations. Moreover, the presence of these soft-biometric attributes strongly influences the behaviour of face recognition systems. This demonstrates the strong need for more comprehensive privacy-enhancing and bias-mitigating technologies that are not limited to pre-defined (demographic) attributes. Guided by these findings, this work proposes solutions for mitigating bias in face recognition systems and solutions for the enhancement of soft-biometric privacy in these systems. The proposed bias-mitigating solutions operate on the comparison- and scorelevel of recognition system and thus, can be easily integrated. Incorporating the notation of individual fairness, that aims at treating similar individuals similarly, strongly mitigates bias of unknown origins and further improves the overall-recognition performance of the system. The proposed solutions for enhancing the soft-biometric privacy in face recognition systems either manipulate existing face representations directly or changes the representation type including the inference process for verification. The manipulation of existing face representations aims at directly suppressing the encoded privacy-risk information in an easily-integrable manner. Contrarily, the inference-level solutions indirectly suppress this privacy-risk information by changing the way of how this information is encoded. To summarise, this work investigates soft-biometric driven bias and privacy concerns in face recognition systems and proposed solutions to mitigate these. Unlike previous works, the proposed approaches are (a) highly effective in mitigating these concerns, (b) not limited to the mitigation of concerns origin from specific attributes, and (c) easilyintegrable into existing systems. Moreover, the presented solutions are not limited to face biometrics and thus, aim at enhancing the reliability, trust, and dissemination of biometric systems in general.

Show publication details

MixFaceNets: Extremely Efficient Face Recognition Networks

2021

IJCB 2021. IEEE/IARP International Joint Conference on Biometrics

IEEE International Joint Conference on Biometrics (IJCB) <2021, online>

In this paper, we present a set of extremely efficient and high throughput models for accurate face verification, Mix-FaceNets which are inspired by Mixed Depthwise Convolutional Kernels. Extensive experiment evaluations on Label Face in the Wild (LFW), Age-DB, MegaFace, and IARPA Janus Benchmarks IJB-B and IJB-C datasets have shown the effectiveness of our MixFaceNets for applications requiring extremely low computational complexity. Under the same level of computation complexity (≤ 500M FLOPs), our MixFaceNets outperform MobileFaceNets on all the evaluated datasets, achieving 99.60% accuracy on LFW, 97.05% accuracy on AgeDB-30, 93.60 TAR (at FAR1e-6) on MegaFace, 90.94 TAR (at FAR1e-4) on IJB-B and 93.08 TAR (at FAR1e-4) on IJB-C. With computational complexity between 500M and 1G FLOPs, our MixFaceNets achieved results comparable to the top-ranked models, while using significantly fewer FLOPs and less computation over-head, which proves the practical value of our proposed Mix-FaceNets. All training codes, pre-trained models, and training logs have been made available https://github.com/fdbtrs/mixfacenets.

Show publication details
Siebke, Patrick; Kuijper, Arjan [1. Review]; Boutros, Fadi [2. Review]

Neural Architecture Search for Mobile Face Recognition

2021

Darmstadt, TU, Bachelor Thesis, 2021

Biometrics is a rapidly growing technology that aims to identify or verify people’s identities based on their physical or behavioral properties. With the rapid growth of smartphone users, the interest in secure authentication using biometric technology to authorize and identify the application user has been increased. With its high social acceptability, face recognition is one of the most convenient and accurate biometric recognition technologies which is used more and more in mobile and embedded systems for unlocking, application login and mobile payment. To enable face recognition on low computational powered devices, the model needs to be accurate, small and fast. With the rise of deep learning technology, face recognition systems were able to achieve notable verification performances. However, most high-accurate face recognition models rely on very deep Convolutional Neural Networks (CNNs) and therefore, require a high amount of computational resources, which makes them unfeasible for mobile and embedded systems. Recently, a great progess has been made in designing efficient face recognition systems by utilizing lightweight deep learning network architectures designed for common computer vision tasks for face recognition. However, none of these works designed a network specifically for the face recognition task, rather than adopting existing architectures designed for common computer vision tasks. With the development of AutoML, Neural Architecture Search (NAS) has shown excellent performance in many computer vision tasks and was able to automatically design highly efficient network architectures that outperform existing manually designed architectures. This thesis utilizes Neural Architecture Search (NAS) to automate the process of designing highly efficient neural architectures for face recognition. While other works focused on manually designing efficient architectures, this process has not been made automatic for the face recognition setting. This is the first work that utilizes Differentiable Architecture Search (DARTS) for face recognition and introduces a new search space. Based on DARTS, this thesis introduces a new network architecture named DartFaceNet. Evaluation on a variety of large-scale databases proves that DartFaceNet is able to achieve high performance on major face recognition benchmarks. Using DartFaceNet architecture and different embedding sizes, this thesis introduces three face recognition models with less than 2 million parameters. Trained with ArcFace loss on MS1MV2 dataset, DartFaceNet-256 achieves 99.5% on LFW with only 0.991 million parameters and 587.11 FLOPs which is comparable to the efficient light-weight architecture from MobileFaceNets. With less than 2 million parameters, DartFaceNet-512 outperforms existing light-weight models under 5 million parameters on CA-LFW with 95.333%. Also this thesis provides evaluation on the large-scale databases IJB-B, IJB-C and the MegaFace challenge. With only 0.925 million parameters and a memory footprint of 3.7 Megabytes, DartFaceNet-128 achieves the best trade-off between performance and model size among all DartFaceNet models

Show publication details
Tabassi, Elham; Olsen, Martin; Bausinger, Oliver; Busch, Christoph; Figlarz, Andrew; Fiumara, Gregory; Henniger, Olaf; Merkle, Johannes; Ruhland, Timo; Schiel, Christopher; Schwaiger, Michael

NFIQ 2 NIST Fingerprint Image Quality

2021

NIST Fingerprint Image Quality (NFIQ 2) is open source software that links image quality of optical and ink 500 pixel per inch fingerprints to operational recognition performance. This allows quality values to be tightly defined and then numerically calibrated, which in turn allows for the standardization needed to support a worldwide deployment of fingerprint sensors with universally interpretable image qualities. NFIQ 2 quality features are formally standardized as part of ISO/IEC 29794-4 and serve as the reference implementation of the standard.

Show publication details

On Soft-Biometric Information Stored in Biometric Face Embeddings

2021

IEEE Transactions on Biometrics, Behavior, and Identity Science

The success of modern face recognition systems is based on the advances of deeply-learned features. These embeddings aim to encode the identity of an individual such that these can be used for recognition. However, recent works have shown that more information beyond the user’s identity is stored in these embeddings, such as demographics, image characteristics, and social traits. This raises privacy and bias concerns in face recognition. We investigate the predictability of 73 different soft-biometric attributes on three popular face embeddings with different learning principles. The experiments were conducted on two publicly available databases. For the evaluation, we trained a massive attribute classifier such that can accurately state the confidence of its predictions. This enables us to derive more sophisticated statements about the attribute predictability. The results demonstrate that the majority of the investigated attributes are encoded in face embeddings. For instance, a strong encoding was found for demographics, haircolors, hairstyles, beards, and accessories. Although face recognition embeddings are trained to be robust against non-permanent factors, we found that specifically these attributes are easily-predictable from face embeddings. We hope our findings will guide future works to develop more privacy-preserving and bias-mitigating face recognition technologies.

Show publication details
Gerding, Michael; Baumgarten, Gerd; Zecha, Marius; Luebken, Franz-Josef; Baumgarten, Kathrin; Latteck, Ralph

On the Unusually Bright and Frequent Noctilucent Clouds in Summer 2019 above Northern Germany

2021

Journal of Atmospheric and Solar-Terrestrial Physics

Noctilucent Clouds (NLC) and Mesospheric Summer Echoes (MSE) are ice-related phenomena that occur occasionally in the mid-latitude summer mesopause region and more frequently in the polar regions. We observe both phenomena above our site at Kühlungsborn (Germany, 54.1°N, 11.8°E) by lidar and radar since 1997 and 1998, respectively. The NLC season 2019 turned out to be record-breaking with respect to different parameters. We observed the brightest NLC (backscatter coefficient at 532 nm of ), the longest continuous NLC (11 h) and the largest occurrence rates in June (20%). The seasonally averaged NLC height was found 600 m lower in altitude in 2019 compared to our long-term record. Consistent with the NLC data, radar observations of MSE showed unusually long-lasting echoes and a higher occurrence rate in June 2019. In contrast to our initial expectations, this increase of ice abundance in 2019 was not related to a generally stronger advection from higher latitudes. Mean winds observed by a nearby meteor radar were essentially weaker than in previous years, even though the winds were still typically southward during NLC. Instead, we found unusually low mean temperatures below 83 km altitude (and down to 75 km) being the main reason for these extraordinary observations. Furthermore, water vapor concentrations were slightly enhanced in June 2019. Low temperatures and enhanced water vapor may be caused by stronger upwelling in the upper mesosphere. However, there is no vertical wind data available. Low solar activity was also a factor that promoted these good NLC conditions. Overall, we judge this a singular event and not as an indicator for climate change. Temperature data in the mesopause region and below are taken from Sounding of the Atmosphere using Broadband Emission Radiometry (SABER) on NASA's Thermosphere Ionosphere Mesosphere Energetics Dynamics (TIMED) satellite as well as Earth Observing System (EOS) Microwave Limb Sounder (MLS) onboard NASA's Aura satellite.

Show publication details
Fellner, Dieter W. [Hrsg.]; Welling, Daniela [Red.]; Ackeren, Janine van [Red.]; Bergstedt, Bettina [Red.]; Krüger, Kathrin [Red.]; Prasche, Svenja [Betreuer]; Bornemann, Heidrun [Red.]; Roth, Anahit [Red.]

Our Year 2020

2021

Show publication details
Meden, Blaž; Rot, Peter; Terhörst, Philipp; Damer, Naser; Kuijper, Arjan; Scheirer, Walter J.; Ross, Arun A.; Peer, Peter; Struc, Vitomir

Privacy-Enhancing Face Biometrics: A Comprehensive Survey

2021

IEEE Transactions on Information Forensics and Security

Biometric recognition technology has made significant advances over the last decade and is now used across a number of services and applications. However, this widespread deployment has also resulted in privacy concerns and evolving societal expectations about the appropriate use of the technology. For example, the ability to automatically extract age, gender, race, and health cues from biometric data has heightened concerns about privacy leakage. Face recognition technology, in particular, has been in the spotlight, and is now seen by many as posing a considerable risk to personal privacy. In response to these and similar concerns, researchers have intensified efforts towards developing techniques and computational models capable of ensuring privacy to individuals, while still facilitating the utility of face recognition technology in several application scenarios. These efforts have resulted in a multitude of privacy–enhancing techniques that aim at addressing privacy risks originating from biometric systems and providing technological solutions for legislative requirements set forth in privacy laws and regulations, such as GDPR. The goal of this overview paper is to provide a comprehensive introduction into privacy–related research in the area of biometrics and review existing work on Biometric Privacy–Enhancing Techniques (B–PETs) applied to face biometrics. To make this work useful for as wide of an audience as possible, several key topics are covered as well, including evaluation strategies used with B–PETs, existing datasets, relevant standards, and regulations and critical open issues that will have to be addressed in the future.

Show publication details

PrivInferVis: Towards Enhancing Transparency over Attribute Inference in Online Social Networks

2021

Proceedings

IEEE International Conference on Computer Communications (INFOCOM) <2021, Online>

The European GDPR calls, besides other things, for innovative tools to empower online social networks (OSN) users with transparency over risks of attribute inferences. In this work, we propose a novel transparency-enhancing framework for OSN, PrivInferVis, to help people assess and visualize their individual risks of attribute inference derived from public details from their social graphs in different OSN domains. We propose a weighted Bayesian model as the underlying method for attribute inference. A preliminary evaluation shows that our proposal outperforms baseline algorithms on several evaluation metrics significantly. PrivInferVis provides visual interfaces that allow users to explore details about their (inferred and self-disclosed) data and to understand how inference estimates and related scores are derived.

Show publication details

ProBGP: Progressive Visual Analytics of Live BGP Updates

2021

EuroVis 2021. 23rd Eurographics / IEEE VGTC Conference on Visualization 2021

Eurographics / IEEE VGTC Conference on Visualization (EuroVis) <23, 2021, online>

The global routing network is the backbone of the Internet. However, it is quite vulnerable to attacks that cause major disruptions or routing manipulations. Prior related works have visualized routing path changes with node link diagrams, but it requires strong domain expertise to understand if a routing change between autonomous systems is suspicious. Geographic visualization has an advantage over conventional node-link diagrams by helping uncover such suspicious routes as the user can immediately see if a path is the shortest path to the target or an unreasonable detour. In this paper, we present ProBGP, a web-based progressive approach to visually analyze BGP update routes. We created a novel progressive data processing algorithm for the geographic approximation of autonomous systems and combined it with a progressively updating visualization. While the newest log data is continuously loaded, our approach also allows querying the entire log recordings since 1999. We present the usefulness of our approach with a real use case of a major route leak from June 2019. We report on multiple interviews with domain experts throughout the development. Finally, we evaluated our algorithm quantitatively against a public peering database and qualitatively against AS network maps.

Show publication details
Cao, Min; Chen, Chen; Dou, Hao; Hu, Xiyuan; Peng, Silong; Kuijper, Arjan

Progressive Bilateral-Context Driven Model for Post-Processing Person Re-Identification

2021

IEEE Transactions on Multimedia

Most existing person re-identification methods compute pairwise similarity by extracting robust visual features and learning the discriminative metric. Owing to visual ambiguities, these content-based methods that determine the pairwise relationship only based on the similarity between them, inevitably produce a suboptimal ranking list. Instead, the pairwise similarity can be estimated more accurately along the geodesic path of the underlying data manifold by exploring the rich contextual information of the sample. In this paper, we propose a lightweight post-processing person re-identification method in which the pairwise measure is determined by the relationship between the sample and the counterpart's context in an unsupervised way. We translate the point-to-point comparison into the bilateral point-to-set comparison. The sample's context is composed of its neighbor samples with two different definition ways: the first order context and the second order context, which are used to compute the pairwise similarity in sequence, resulting in a progressive post-processing model. The experiments on four large-scale person re-identification benchmark datasets indicate that (1) the proposed method can consistently achieve higher accuracies by serving as a post-processing procedure after the content-based person re-identification methods, showing its state-of-the-art results, (2) the proposed lightweight method only needs about 6 milliseconds for optimizing the ranking results of one sample, showing its high-efficiency. Code is available at: https://github.com/123ci/PBCmodel.

Show publication details
Kocon, Kevin; Kuijper, Arjan [1. Gutachten]; Bormann, Pascal [2. Gutachten]

Progressives Indexieren von massiven Punktwolken mit Big-Data-Technologien

2021

Darmstadt, TU, Master Thesis, 2021

Aufgrund der steigenden Beliebtheit von Punktwolken und der genauer werdenden LiDAR Aufnahmeverfahren wachsen die Punktwolkengrößen exponentiell. Ein wichtiger, in dieser Thesis betrachteter Anwendungsfall, ist der Aufbau eines räumlichen Index für die Web-basierte Echtzeitvisualisierung. Bereits bei aktuellen Punktwolken dauert dieser zeitintensive Prozess mit derzeitigen Verfahren bis zu mehreren Tagen. In dieser Thesis werden zwei Ansätze vorgestellt, die die Indexierungsdauer deutlich reduzieren. Zum einen wird durch die Verwendung von Big-Data-Technologien und der daraus folgenden Skalierbarkeit eine Reduktion der Gesamtindexierungsdauer erreicht. Zum anderen werden durch einen progressiven Ansatz relevante Teilbereiche der Punktwolke sukzessiv indexiert, wodurch diese bereits vor dem Verarbeiten der gesamten Punktwolke visualisiert werden können. Als Big-Data-Technologien werden Apache Spark und Apache Cassandra verwendet. Die Grundvoraussetzung für die Verwendung von Spark bietet der implementierte Top-Down Ansatz, der den Indexierungsvorgang auf das Map-Reduce Schema abbildet. Für den progressiven Ansatz werden für den Nutzer relevante Teilbereiche der Punktwolke bestimmt und nacheinander verarbeitet. Die neu eingeführte Datenstruktur des hybriden Nested-Octree Gitters ermöglicht dafür unter Anderem das sukzessive Erweitern des räumlichen Index. Die Ergebnisse zeigen, dass bei horizontaler Hardwareerweiterung eine lineare Skalierbarkeit um den Faktor 0.58 gegeben ist. Dadurch konnten massive Punktwolken im Vergleich zu lokalen Anwendungen wie dem PotreeConverter 2.0, Schwarzwald oder Entwine deutlich schneller indexiert werden. Zusätzlich ist durch den progressiven Ansatz eine Visualisierung der aktuell vom Nutzer betrachteten Teile der Punktewolke schon nach einem Bruchteil der Gesamtverarbeitungsdauer möglich.

Show publication details
Grotz, Dennis; Kuijper, Arjan [1. Review]; Mukhopadhyay, Anirban [2. Review]

Registration of EMT Positions and X-Ray Images in an Aortic Phantom

2021

Darmstadt, TU, Bachelor Thesis, 2021

In this work a real-time object detection model based on the state-of-the-art architecture You Only Look Once (YOLOv3) was trained, in order to detect an Electromagnetic Tracking (EMT) sensor inside an aortic phantom. An EMT system consists of a tracking system, a field generator and an EMT sensor [4]. A problem with EMT is that magnetic fields interfere with ferromagnetic objects in the surrounding area, which corrupts the EMT positional data [12]. The object detection model can be used to detect and recalibrate the EMT sensor, in order to counteract the electromagnetic interference. This can result in a robust model to support surgeons in minimally invasive surgeries, which have advantages over open surgeries like less pain for the patients, faster recovery after operations or better cosmetic results [7]. In this work different datasets containing webcam images were collected, in order to train and test two models. One model was trained on raw webcam images, while the other one was additionally trained on augmented images. Both models were evaluated quantitatively and qualitatively for different hyperparameter configurations. They show good results on the test dataset with achieved mean Average Precision (mAP) scores of 99.89% (model without augmentations) and 99.67% (model with augmentations) and they run very fast with an interference of 26 ms and 35 Frames Per Second (FPS) on a video file. However, the model trained with augmentations generalizes better on images with different backgrounds. Furthermore, a study with some sample x-ray images showed, that the model trained with augmentations is capable of transferring from webcam images to x-ray images. Finally, this work presented a real-time object detection model for detecting an EMT sensor inside an aortic phantom with very precise detections and the capability of transferring to different domains like x-ray images.

Show publication details
Strelnikova, Irina; Almowafy, Marwa; Baumgarten, Gerd; Baumgarten, Kathrin; Ern, Manfred; Gerding, Michael; Luebken, Franz-Josef

Seasonal Cycle of Gravity Wave Potential Energy Densities from Lidar and Satellite Observations at 54° and 69°N

2021

Journal of the Atmospheric Sciences (JAS)

We present gravity wave climatologies based on 7 years (2012–18) of lidar and Sounding of the Atmosphere using Broadband Emission Radiometry (SABER) temperatures and reanalysis data at 54° and 69°N in the altitude range 30–70 km. We use 9452 (5044) h of lidar observations at Kühlungsborn [Arctic Lidar Observatory for Middle Atmosphere Research (ALOMAR)]. Filtering according to vertical wavelength (λz < 15 km) or period (τ < 8 h) is applied. Gravity wave potential energy densities (GWPED) per unit volume (EpV) and per unit mass (Epm) are derived. GWPED from reanalysis are smaller compared to lidar. The difference increases with altitude in winter and reaches almost two orders of magnitude around 70 km. A seasonal cycle of EpV with maximum values in winter is present at both stations in nearly all lidar and SABER measurements and in reanalysis data. For SABER and for lidar (with λ < 15 km) the winter/summer ratios are a factor of ~2–4, but are significantly smaller for lidar with τ < 8 h. The winter/summer ratios are nearly identical at both stations and are significantly larger for Epm compared to EpV. Lidar and SABER observations show that EpV is larger by a factor of ~2 at Kühlungsborn compared to ALOMAR, independent of season and altitude. Comparison with mean background winds shows that simple scenarios regarding GW filtering, etc., cannot explain the Kühlungsborn–ALOMAR differences. The value of EpV decreases with altitude in nearly all cases. Corresponding EpV-scale heights from lidar are generally larger in winter compared to summer. Above ~55 km, EpV in summer is almost constant with altitude at both stations. The winter–summer difference of EpV scale heights is much smaller or absent in SABER and in reanalysis data.

Show publication details
Stober, Gunter; Janches, Diego; Matthias, Vivien; Fritts, Dave; Marino, John; Moffat-Griffin, Tracy; Baumgarten, Kathrin; Lee, Wonseok; Murphy, Damian; Kim, Yong Ha; Mitchell, Nicholas; Palo, Scott

Seasonal Evolution of Winds, Atmospheric Tides, and Reynolds Stress omponents in the Southern Hemisphere Mesosphere–Lower Thermosphere in 2019

2021

Annales Geophysicae (ANGEO)

In this study we explore the seasonal variability of the mean winds and diurnal and semidiurnal tidal amplitude and phases, as well as the Reynolds stress components during 2019, utilizing meteor radars at six Southern Hemisphere locations ranging from midlatitudes to polar latitudes. These include Tierra del Fuego, King Edward Point on South Georgia island, King Sejong Station, Rothera, Davis, and McMurdo stations. The year 2019 was exceptional in the Southern Hemisphere, due to the occurrence of a rare minor stratospheric warming in September. Our results show a substantial longitudinal and latitudinal seasonal variability of mean winds and tides, pointing towards a wobbling and asymmetric polar vortex. Furthermore, the derived momentum fluxes and wind variances, utilizing a recently developed algorithm, reveal a characteristic seasonal pattern at each location included in this study. The longitudinal and latitudinal variability of vertical flux of zonal and meridional momentum is discussed in the context of polar vortex asymmetry, spatial and temporal variability, and the longitude and latitude dependence of the vertical propagation conditions of gravity waves. The horizontal momentum fluxes exhibit a rather consistent seasonal structure between the stations, while the wind variances indicate a clear seasonal behavior and altitude dependence, showing the largest values at higher altitudes during the hemispheric winter and two variance minima during the equinoxes. Also the hemispheric summer mesopause and the zonal wind reversal can be identified in the wind variances.

Show publication details
Bauer, Christoph; Kuijper, Arjan [Betreuer]; Kohlhammer, Jörn [Betreuer]; Bergmann, Tim Alexander [Betreuer]

Semiautomatische Segmentierung von relevanten Gelenken zur Frühdiagnose von Psoriasis-Arthritis in Röntgenbildern

2021

Darmstadt, TU, Master Thesis, 2021

In dieser Arbeit wird ein dreistufiges semi-automatisches Verfahren zur Segmentierung von relevanten Gelenken für die Frühdiagnose von Psoriasis-Arthritis in Röntgenbildern vorgestellt. Psoriasis-Arthritis ist eine entzündliche Gelenkerkrankung, welche eine Verringerung des Gelenkabstandes, Gelenkfehlstellungen und bis zu Erosionen von Knochengliedern führen kann. Um dies zu detektieren werden die Knochen von Hand und Fuß segmentiert. Hierfür wird im ersten Schritt des Verfahrens das Bild vorverarbeitet. So wird mithilfe eines Medianfilters Rauschen entfernt und der Kontrast des Bildes durch Anwendung eines adaptiven Histogrammausgleichs verbessert. Für eine Basissegmentierung sorgen dann zwei adaptive Schwellwertverfahren, welche zusammengefügt werden. Für die endgültige Segmentierung wird im letzten Schritt jedes Knochenglied einzeln betrachtet und über eine Verknüpfung der Basissegmentierung mit einem erstellten Kantenbild verbessert. Somit besteht das Resultat dieses Verfahrens aus 19 Segmentierungen der einzelnen Knochenglieder, die für eine bessere Übersicht beliebig kombiniert werden können. Die Resultate werden im Anschluss mithilfe des Dice-Koeffizienten und der Hausdorff-Distanz anhand von Ground-Truth-Daten ausgewertet. Diese zeigen gerade im Bereich der Hand, mit einem durchschnittlichen Dice-Koeffizienten von 0.923 und einer durchschnittlichen Hausdorff-Distanz von 23.673, vielversprechende Ergebnisse. Im Bereich der Füße zeigen die Ergebnisse, mit einem durchschnittlichen Dice-Koeffizienten von 0.74 und einer durchschnittlichen Hausdorff-Distanz von 54.462, noch ausbaufähige Resultate.

Show publication details
Ströter, Daniel; Krispel, Ulrich; Mueller-Roemer, Johannes; Fellner, Dieter W.

TEdit: A Distributed Tetrahedral Mesh Editor with Immediate Simulation Feedback

2021

Proceedings of the 11th International Conference on Simulation and Modeling Methodologies, Technologies and Applications (SIMULTECH 2021)

International Conference on Simulation and Modeling Methodologies, Technologies and Applications (SIMULTECH) <11, 2021, Online>

The cycle of computer aided design and verification via physics simulation is often burdened by the use of separate tools for modeling and simulation, which requires conversion between formats, e.g. meshing for finite element simulation. This separation is often unavoidable because the tools contain specific domain knowledge which is mandatory for the task, for example a specific CAD modeling suite. We propose a distributed application that allows interactive modification of tetrahedral meshes, derived from existing CAD models. It provides immediate simulation feedback by offloading resource-intensive tasks onto multiple machines thereby enabling fast design cycles for individualized versions of mass-produced parts.

Show publication details

The Effect of Alignment on Peoples Ability to Judge Event Sequence Similarity

2021

IEEE Transactions on Visualization and Computer Graphics

Event sequences are central to the analysis of data in domains that range from biology and health, to logfile analysis and people's everyday behavior. Many visualization tools have been created for such data, but people are error-prone when asked to judge the similarity of event sequences with basic presentation methods. This paper describes an experiment that investigates whether local and global alignment techniques improve people's performance when judging sequence similarity. Participants were divided into three groups (basic vs. local vs. global alignment), and each participant judged the similarity of 180 sets of pseudo-randomly generated sequences. Each set comprised a target, a correct choice and a wrong choice. After training, the global alignment group was more accurate than the local alignment group (98% vs. 93% correct), with the basic group getting 95% correct. Participants' response times were primarily affected by the number of event types, the similarity of sequences (measured by the Levenshtein distance) and the edit types (nine combinations of deletion, insertion and substitution). In summary, global alignment is superior and people's performance could be further improved by choosing alignment parameters that explicitly penalize sequence mismatches.

Show publication details

The Effect of Alignment on People’s Ability to Judge Event Sequence Similarity

2021

IEEE Transactions on Visualization and Computer Graphics

Event sequences are central to the analysis of data in domains that range from biology and health, to logfile analysis and people's everyday behavior. Many visualization tools have been created for such data, but people are error-prone when asked to judge the similarity of event sequences with basic presentation methods. This paper describes an experiment that investigates whether local and global alignment techniques improve people's performance when judging sequence similarity. Participants were divided into three groups (basic vs. local vs. global alignment), and each participant judged the similarity of 180 sets of pseudo-randomly generated sequences. Each set comprised a target, a correct choice and a wrong choice. After training, the global alignment group was more accurate than the local alignment group (98% vs. 93% correct), with the basic group getting 95% correct. Participants' response times were primarily affected by the number of event types, the similarity of sequences (measured by the Levenshtein distance) and the edit types (nine combinations of deletion, insertion and substitution). In summary, global alignment is superior and people's performance could be further improved by choosing alignment parameters that explicitly penalize sequence mismatches.

Show publication details

Towards Combined Open Set Recognition and Out-of-Distribution Detection for Fine-grained Classification

2021

Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications

International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP) <16, 2021, online>

We analyze the two very similar problems of Out-of-Distribution (OOD) Detection and Open Set Recognition (OSR) in the context of fine-grained classification. Both problems are about detecting object classes that a classifier was not trained on, but while the former aims to reject invalid inputs, the latter aims to detect valid but unknown classes. Previous works on OOD detection and OSR methods are evaluated mostly on very simple datasets or datasets with large inter-class variance and perform poorly in the fine-grained setting. In our experiments, we show that object detection works well to recognize invalid inputs and techniques from the field of fine-grained classification, like individual part detection or zooming into discriminative local regions, are helpful for fine-grained OSR.

Show publication details
Reiz, Achim; Albadawi, Mohamad; Sandkuhl, Kurt; Vahl, Matthias; Sidin, Dennis

Towards More Robust Fashion Recognition by Combining Deep-Learning-Based Detection with Semantic Reasoning

2021

Proceedings of the AAAI 2021 Spring Symposium on Combining Machine Learning and Knowledge Engineering (AAAI-MAKE 2021)

AAAI 2021 Spring Symposium on Combining Machine Learning and Knowledge Engineering (AAAI-MAKE 2021) <2021, Online>

CEUR Workshop Proceedings
2846

The company FutureTV produces and distributes self-produced videos in the fashion domain. It creates revenue through the placement of relevant advertising. The placement of apposite ads, though, requires an understanding of the contents of the videos. Until now, this tagging is created manually in a labor-intensive process. We believe that image recognition technologies can significantly decrease the need for manual involvement in the tagging process. However, the tagging of videos comes with additional challenges: Preliminary, new deep-learning models need to be trained on vast amounts of data obtained in a labor-intensive data-collection process. We suggest a new approach for the combining of deep-learning-based recognition with a semantic reasoning engine. Through the explicit declaration of knowledge fitting to the fashion categories present in the training data of the recognition system, we argue that it is possible to refine the recognition results and win extra knowledge beyond what is found in the neural net.

Show publication details
Boller, André; Kuijper, Arjan [1. Review]; Terhörst, Philipp [2. Review]

Towards Unsupervised Fingerprint Image Quality Assessment

2021

Darmstadt, TU, Bachelor Thesis, 2021

Fingerprint matching is one of the most popular and reliable biometric techniques used in automatic verification of a person. During the verification process of a fingerprint, the areas and in more detail the information in these areas (minutiae) of the fingerprint are compared with the corresponding information of a second fingerprint, which is called fingerprint matching. However, the quality of the fingerprint images highly affects the recognition process. Generally, the biometric sample quality is defined on its impact on a biometric recognition system, the similarity between the sample and its source, and the quality of its physical features. Poor quality fingerprints often have areas or regions that are not clear or even missing, resulting in arbitrary changes in the structure of these areas. That results in an inaccurate matching caused by comparisons of damaged areas, which leads to inaccurate matching influencing the verification results again. That is the reason why assessing the fingerprint image quality is very important because the matching performance could be significantly affected by poor quality samples. One application area that is very much affected by this problem is forensics. Forensics often deal with partial fingerprints lifted from a surface, known as latent fingerprints. Since latent fingerprints are often of poor quality, the matching performance during the recognition process is often severely impaired. Quality assessment can be used to improve biometric systems that perform automatic recognition tasks like identification or verification. Quality estimation during the enrollment process is used to ensure the best possible quality of the biometric data, thus guaranteeing a good training of the biometric systems as well as a good performance. This is a typical reason why image quality assessment is required to evaluate the quality of the images and improve the recognition process. In this thesis, two new methods are proposed to accurately assess (a) the quality of a single minutia and (b) the quality of a fingerprint. The proposed minutia quality is based on its detection reliability. A minutia is classified by randomly generated subnetworks of a minutiae classifier that determines if a minutia is a true minutia or not. The various classification results are used to determine a robustness score, considered as the detection reliability. The proposed fingerprint quality assessment method applies a stochastic method to the detection reliabilities of the minutiae to determine the quality of the fingerprint. Since this thesis addresses these two problems, both kinds of quality assessments have to be evaluated separately. The proposed minutia quality assessment method (a) is compared with Mindtct. For evaluation purposes of (b), the proposed method is compared with the current state-of-the-art quality assessment method NFIQ2 as well as its predecessor NFIQ. All experiments are evaluated on the FVC2006 database using the Bozorth3 and MCC fingerprint matchers. It can be shown that the proposed method assesses the quality on minutiae-level (a) just as good and better as Mindtct on the most experiments without the need of handcrafted quality labels. Experiments on data acquired from an electrical field sensor, for example, show that the proposed method achieves on average a 0.03 lower FNMR at a FMR of 10−2 using Bozorth3. Furthermore, the quality assessment on fingerprint-level (b) outperforms the state-of-the-art quality assessment methods NFIQ2 and NFIQ. An improvement of the recognition performance on all databases captured by real sensor types could be achieved. Observations of experimental results at a FMR of 10−2 using Bozorth3 show that the proposed method achieves a FNMR that is about 0.003 lower than the FNMR achieved by NFIQ2 on optical sensor data after rejecting the 20% worst fingerprints. Furthermore, a 0.025 lower FNMR can also be achieved on data captured by a thermal sweeping sensor.

Show publication details
Hermawati, Setia; Cibulski, Lena; Lawson, Glyn

Towards Using the Critical Decision Method for Studying Visualisation-Based Decision-Making

2021

Ergonomics & Human Factors 2021. Proceedings

Ergonomics & Human Factors (CIEHF) <2021, Online>

Visualisations provide significant support for effective reasoning and decision-making processes. Its value mainly lies in its ability to turn raw data into actionable insights that lead to a decision. This requires appropriate visual representations that are designed with the decision-maker's way of reasoning in mind. Understanding the cognitive aspects underlying decision-making with visualisations is therefore crucial. Cognitive task analysis methods have been used to elicit expert knowledge in a variety of decision-making scenarios, with the Critical Decision Method (CDM) focusing on the cognitive bases in naturalistic non-routine incidents. In this study, we aim to determine the feasibility of CDM for capturing the expert knowledge, strategies, and cues involved with visualisation-based decision-making processes. Based on an analysis of four semi-structured interviews, we evaluate the method’s potential to inform the role of visualisation for human decision-making. We anticipate that our reflections on methodological insights can serve as a starting point for other human factors and visualisation researchers, who aim at studying strategies for higher-level decision-making and problem-solving tasks.

Show publication details
Fellner, Dieter W. [Hrsg.]; Welling, Daniela [Red.]; Ackeren, Janine van [Red.]; Bergstedt, Bettina [Red.]; Krüger, Kathrin [Red.]; Prasche, Svenja [Betreuer]; Bornemann, Heidrun [Red.]; Roth, Anahit [Red.]

Unser Jahr 2020

2021

Show publication details
Schäfer, Lukas; Kuijper, Arjan [1. Gutachten]; Renzel, Christian [2. Gutachten]

Visualisierung von Laufzeitdaten zur Unterstützung der Analyse von Softwarekomponenten in der AUTOSAR-basierten Entwicklung

2021

Darmstadt, TU, Bachelor Thesis, 2021

The fulfillment of the requirements for runtime limits and controller workload is very important in the context of embedded software development, especially in the automotive industry. Detecting critical system workload situations at an early stage of the development process and locating the causing components are critical and demanding tasks in a system with many ECUs and the associated software. In this work, the visual representability of the measurable data of an AUTOSAR-based software system will be addressed for the first time. For this purpose, possibilities for creating requirements in such a visualization process are presented and discussed. Use cases that occur in industrial practice will be worked out and a prototypical software for data visualization will be developed. The intention of this thesis is to support the development in the area of diagnosis and evaluation of software component runtimes. Finally, the use and the benefit of this software will be evaluated in practical use.

Show publication details
Starčević, Nataša; Kuijper, Arjan [Betreuer]; Schufrin, Marija [Betreuer]

Visualizing Online Personal Data based on the Mental Models of Common Internet User Types

2021

Darmstadt, TU, Master Thesis, 2021

Der Alltag und das Leben vieler Menschen wird zunehmend von der Online Welt geprägt und beeinflusst. Doch das Internet und die Vielzahl an Online Diensten, die wir alle täglich nutzen, die uns einerseits den Alltag erleichtern können, bergen andererseits ein oftmals schwer abschätzbares Risiko, aufgrund der zunehmenden Datensammlung und der damit verbundenen Beeinträchtigung der Privatsphäre. Viele Nutzer sind sich des Ausmaßes dieser Datensammlung nicht bewusst und wissen oft nicht, welche Daten jeweils erhoben und gespeichert werden. Daher werden Techniken und Methoden entwickelt, um den Menschen bei dieser Aufgabe zu unterstützen. Jedoch sind viele Ansätze nicht an den Vorstellungen und Bedürfnissen des Nutzers ausgerichtet. Aus diesem Grund wurde im Rahmen dieser Master-Arbeit die Web-Applikation TransparencyVis zur Visualisierung personenbezogener Daten erweitert. Die bestehende Anwendung dient zur Verbesserung der Datentransparenz und erlaubt die Erkundung personenbezogener Daten von populären Online-Diensten, sogenannte DSGVO Datenexporte, die aufgrund des Auskunftsrechts des Art. 15 DSGVO von den Nutzern angefordert werden können. Auf Basis von mentalen Modellen von typischen Internetnutzern wurden Visualisierungskonzepte entworfen und prototypisch in die bestehende Anwendung implementiert. Zusätzlich wurden die Nutzer anhand ihrer Datenschutzbedenken in drei Klassen unterteilt. Mit Hilfe interaktiver Visualisierungen haben die Nutzer die Möglichkeit, ihre persönlichen DSGVO Datenexporte ausgewählter Online-Dienste zu erkunden. Im Rahmen der durchgeführten Nutzerstudie wurden die Visualisierungen auf ihre Wirksamkeit mit 33 echten Nutzern und deren persönlichen Daten evaluiert. Die Untersuchungsergebnisse lassen darauf schließen, dass die Visualisierungen das Verständnis der Nutzer in Bezug auf ihre Daten und deren Relevanz für die Privatssphäre erhöhen können. Ferner zeigen sie, dass die Integration mentaler Modelle in den Design-Prozess von Visualisierungen sich positiv auf die Erstellung nutzertyp-spezifischer Visualisierungen auswirken und deren Akzeptanz durch die Zielgruppe verbessern kann.

Show publication details
Singer, Maxim; Kuijper, Arjan [1. Gutachten]; Wirtz, Andreas [2. Gutachten]

Vollautomatische Initialisierung von Coupled-Shape-Modellen mithilfe von Deep Neural Networks

2021

Darmstadt, TU, Bachelor Thesis, 2021

Die Verwendung von radiologischen Aufnahmen in der Zahnmedizin ist eine alltägliche Tätigkeit von Zahnärzten. Es geht dabei darum die Aufnahmen zu beurteilen und basierend darauf eine möglichst gute Behandlungsoption zu wählen. Zur Unterstützung und Automatisierung dieser Tätigkeit werden häufig Software-Lösungen eingesetzt. Die Benutzung von so genannten Coupled-Shape-Modellen, insbesondere in der Zahnersatz- und Zahnerhalt-Medizin, stellt die dafür benötigte technische Grundlage dar. Ein Problem von Coupled-Shape-Modellen ist, dass deren Initialisierung sehr spezifisch auf ein Modell zugeschnitten ist und ein großer Aufwand betrieben werden muss, um diese auf ein neues Modell anzupassen. In Rahmen dieser Arbeit wurde untersucht, wie Coupled-Shape-Modelle vollautomatisch und möglichst generisch initialisiert werden können. Dabei wurde die Segmentierung von GebissAufnahmen mit Hilfe von neuronalen Netzen U-Net und Mask R-CNN evaluiert. Je nach Bildtyp konnte eine Segmentierungsgenauigkeit - gemessen als Dice-Koeffizient - von über 95 % erreicht werden. Darauf aufbauend wurden zwei Methoden zur Bestimmung der Position und Skalierung des Coupled-Shape-Modells vorgestellt, diskutiert und in einem Coupled-Shape-Framework implementiert. Zur Verbesserung des Arbeitsflusses wurde eine Klassifizierung von dentalen Bildtypen evaluiert und ebenfalls in das Coupled-Shape-Framework integriert.