### List of scientific publications

Ergebnisse 1 - 161 von 161
Show publication details
Abb, Benjamin; Kuijper, Arjan [1. Review]; Gutbell, Ralf [2. Review]

#### 3D Mesh Generation Through Noised RGB-D Inputstream and Rule Based Denoising with Virtual City Model

2020

3D models are popular for planning in an urban context. The Levels of Detail (LoDs) can vary from cuboid shapes to highly detailed meshes. The acquisition and updating of those models is a cost intensive process requiring aerial footage and manual labor. This is why often only low detailed city models are available, which do not represent an up-to-date state. Updating a city model with RGB-D mesh generation can be a viable option, since depth sensing cameras have become cheap and machine learning techniques for predicting depth from a single color image have advanced. But depth values from those methods are very noisy. Although there are good options available for reconstructing a 3D mesh from a stream of color and depth images, this amount of noise represents a challenge. In this thesis a 3D mesh reconstruction method is presented that uses the existing virtual city model as a second data input to minimize the influence of noise. Therefore a virtual depth stream is created by rendering the urban model from the same perspective as the noised RGB-D stream. A set of rules merges both streams by leveraging their depth difference and normal deviation. The approach is implemented as an extension to the reconstruction algorithm of SurfelMeshing. The output is an updated model with more detailed building features. The evaluation is done in an artificial environment to test against ground truth with fixed noise levels. Quantitative results show that the approach is less prone to errors than using just the noised depth stream. Artifacts in the reconstruction can still arise especially with a very high noise level. The denoising capabilities show that salient features are kept while the overall output error is reduced.

Show publication details

#### A Characterization of Data Exchange between Visual Analytics Tools

2020

Information Visualisation

International Conference on Information Visualization (IV) <24, 2020, online>

Over the past years, the visualization of large andcomplex data sets brought up various Visual Analytics (VA)tools in order to solve domain-specific tasks. These VA toolsare typically implemented as individual software componentsin data-flow-oriented models, meaning that data is transferredfrom one component to the next. While most VA frameworksrely on a monolithic architecture with features for the integration of specialized analysis methods, we consider a loosecoupling of independent applications, where autonomous VAtools are used in predefined analysis sequences. To this end,we provide a characterization of the data exchange processamong individual VA tools in the form of a taxonomy. Thistaxonomy can be used as a checklist to identify characteristicsand improve the data flow of one’s own multi-tool VA setup.For this purpose, we conducted a systematic investigation of theindividual aspects of data exchange that are commonly foundacross different usage scenarios. We apply our taxonomy tothree existing multi-tool frameworks, the open-source libraryReVize, the toolchain editor AnyProc, and the visualizationand monitoring framework Plant@Hand3D.

Show publication details
Kuban, Katharina; Kuijper, Arjan [1. Review]; Schufrin, Marija [2. Review]

#### A Gamified Information Visualization for the Exploration of Home Network Traffic Data

2020

Internet users today are exposed to a variety of cyberthreats. Therefore, it is necessary to make even nonexpert users aware of the importance of cybersecurity. In this thesis, an approach to address this problem was developed based on the User Centered Design process. The development focused on visualizing the users home network data to improve security in private usage. Existing approaches are either focusing on visualizing network data for more experienced users or teaching cybersecurity with gamified solutions. Combining both, the visualization of the data was embedded in a game to motivate the user. A user study was conducted to identify the user requirements. It could be shown that the main reasons for not dealing with cybersecurity and network data are the user’s lack of motivation and the difficulty of the topic. While following information visualization and game design principles, a prototype was implemented based on the user requirements. The prototype was evaluated from the user perspective revealing that the game strengthens general awareness for the communication of devices in one’s home network and makes the topic network data more accessible. Additionally, it was found out that the quality of player experience design is a crucial factor to motivate the user in the context of the presented approach. It should therefore get higher attention in future steps.

Show publication details
Bohlender, Simon Peter; Kuijper, Arjan [1. Review]; Mukhopadhyay, Anirban [2. Review]

#### A Shape-Model based Loss for End-to-End Training of Neural Networks

2020

Traditional models for performing segmentation tasks on medical image datasets mostly rely on Convolutional Neural Networks (CNN). These models are trained with pixel-wise loss functions that aim for maximizing pixel accuracy of predicted segmentation maps. However, these losses do not have any knowledge about the global contour or unique shape characteristics. Despite the existence of special shape models that can capture global shape properties of certain objects, they are only used as pre- or post-processing steps but never included in CNN models themselves. At this point, our work comes into action. This thesis presents a novel method for segmenting medical images by incorporating an Active Shape Model (ASM) into the loss function of CNNs in order to add shape-knowledge to the training process. Therefore, a shape-constraining pipeline is introduced that takes initial estimates and generates shapes similar to ones from the training set. This pipeline is integrated into the loss function as an additional differentiable loss term that allows to jointly train shape-aware neural network models in an end-to-end fashion. Multiple versions of shape-loss functions are explored on a variety of datasets, and their strengths and weaknesses are discussed. Alongside this, a fully functional C++ implementation for training shape-loss CNN models is provided. The decision for C++ was made because of an already available ASM implementation in C++ and the lack of any usable Python version. In this work, TensorFlow’s C++ API is utilized, which is, in contrast to the widespread Python API, rarely applied and lacks proper documentation or any further resources. Hence, great efforts went into elaborating its correct usage for implementing custom losses as well as building and training custom networks. Additionally, a detailed guide on how to use this API to build TensorFlow models is provided.

Show publication details
Lengauer, Stefan; Komar, Alexander; Labrada, Arniel; Karl, Stephan; Trinkl, Elisabeth; Preiner, Reinhold; Bustos, Benjamin; Schreck, Tobias

#### A Sketch-aided Retrieval Approach for Incomplete 3D Objects

2020

Computers & Graphics

With the growing amount of digital collections of visual CH data being available across different repositories, it becomes increasingly important to provide archaeologists with means to find relations and cross-correspondences between different digital records. In principle, existing shape- and image-based similarity search methods can aid such domain analysis tasks. However, in practice, visual object data are given in different modalities, and often only in incomplete or fragmented state, posing a particular challenge for conventional similarity search approaches. In this paper we introduce a methodology and system for cross-modal visual search in CH object data that addresses these challenges. Specifically, we propose a new query modality based on 3D views enhanced by user sketches (3D+sketch). This allows for adding new context to the search, which is useful e.g., for searching based on incomplete query objects, or for testing hypotheses on existence of certain shapes in a collection. We present an appropriately designed workflow for constructing query views from incomplete 3D objects enhanced by a user sketch, based on shape completion and texture inpainting. Visual cues additionally help users compare retrieved objects with the query. The proposed approach extends on a previously presented retrieval system by introducing improved retrieval methods, an extended evaluation including retrieval in a larger and richer data collection, and enhanced interactive search weight specification. We demonstrate the feasibility and potential of our approach to support analysis of domain experts in Archaeology and the field of CH in general.

Show publication details

#### A System for Fast and Scalable Point Cloud Indexing Using Task Parallelism

2020

Smart Tools and Applications in computer Graphics - Eurographics Italian Chapter Conference

Eurographics Italian Chapter Conference - Smart Tools and Applications in computer Graphics (STAG) <2020, online>

We introduce a system for fast, scalable indexing of arbitrarily sized point clouds based on a task-parallel computation model.Points are sorted using Morton indices in order to efficiently distribute sets of related points onto multiple concurrent indexingtasks. To achieve a high degree of parallelism, a hybrid top-down, bottom-up processing strategy is used. Our system achievesa 2.3x to 9x speedup over existing point cloud indexing systems while retaining comparable visual quality of the resultingacceleration structures. It is also fully compatible with widely used data formats in the context of web-based point cloud visualization. We demonstrate the effectiveness of our system in two experiments, evaluating scalability and general performancewhile processing datasets of up to 52.5 billion points.

Show publication details

#### A Visualization Interface to Improve the Transparency of Collected Personal Data on the Internet

2020

VizSec 2020

IEEE Symposium on Visualization for Cyber Security (VizSec) <17, 2020, online>

Online services are used for all kinds of activities, like news, entertainment, publishing content or connecting with others. But information technology enables new threats to privacy by means of global mass surveillance, vast databases and fast distribution networks. Current news are full of misuses and data leakages. In most cases, users are powerless in such situations and develop an attitude of neglect for their online behaviour. On the other hand, the GDPR (General Data Protection Regulation) gives users the right to request a copy of all their personal data stored by a particular service, but the received data is hard to understand or analyze by the common internet user. This paper presents TransparencyVis - a web-based interface to support the visual and interactive exploration of data exports from different online services. With this approach, we aim at increasing the awareness of personal data stored by such online services and the effects of online behaviour. This design study provides an online accessible prototype and a best practice to unify data exports from different sources.

Show publication details
Neumann, Kai Alexander; Kuijper, Arjan [1. Review]; Domajnko, Matevz [2. Review]; Tausch, Reimar [3. Review]

#### Adaptive Camera View Clustering for Fast Incremental Image-based 3D Reconstruction

2020

Photogrammetry, more precisely image-based 3D reconstruction, is an established method for digitizing cultural heritage sites and artifacts. This method utilizes images from different perspectives to reconstruct the geometry and texture of an object. What images are necessary for a successful reconstruction depends on the size, shape, and complexity of the object. Therefore, an autonomous scanning system for 3D reconstruction requires some kind of feedback during acquisition. In this thesis, we present an evaluation of different state-of-the-art photogrammetry solutions to identify which of them is most capable of providing feedback that predicts the quality of the final 3D reconstruction during acquisition. For this, we focused on the open-source incremental reconstruction solutions COLMAP, Alicevision Meshroom and MVE. Additionally, we included the commercial solution Agisoft Metashape to evaluate how it compares against the open-source solutions. While we were able to identify some characteristic behaviors, the accuracy and runtime of all four reconstruction solutions vary based on the input dataset. Because of this, and the fact that all four solutions compute very similar results under the same conditions, our tests were not conclusive. Nevertheless, we chose COLMAP as the back-end for further use as it provided good results on the real dataset as well as an extensive command-line interface (CLI). Based on these results, we introduce an iterative image-based reconstruction pipeline that uses a cluster-based acceleration structure to deliver more robust and efficient 3D reconstructions. The photogrammetry solution used for reconstruction is exchangeable. In this pipeline, images that portray common parts of an object are assigned to clusters based on their camera frustums. Each cluster can be reconstructed separately. The pipeline was implemented as a c++ module and tested on the autonomous robotic scanner CultArm3D®. For this system, we embedded the pipeline in a feedback loop with a density-based Next-Best-View (NBV) algorithm to assist during autonomous acquisition.

Show publication details
Dang, Hien Quoc; Kuijper, Arjan [Referent]; Fürnkranz, Johannes [Korreferent]; Winner, Hermann [Korreferent]

#### Adaptive Personalization in Driver Assistance Systems

2020

Personalization is the task that aims at improving quality of products and services by adapting itself to the current user. In the context of automo-tive applications, personalization isnot only about how drivers sets up the position of their seat or their favorite radio channels.Going beyond that, personalization is also about the preference of driving styles and theindividual behaviors in every maneuver executions. One key challenge in personalizationis to be able to capture and understand the users from the historical data produced by theusers. The data are usually presented in form of time series and in some cases, those timeseries can be remarkably long. Capturing and learning from such data poses a challengefor machine learning models.To deal with this problem, this thesis presents an approach that makes uses of recurrentneural networks to capture the time series of behavioral data of drivers and predict theirslane change intentions. In comparison to previous works, our approach is capable of predicting not only driver’s intention as predefined discrete classes (i. e. left, right and lanekeeping) but also as continuous values of the time left until the drivers cross the lane markings. This provides additional information for advanced driver-assistance systems to decidewhen to warn drivers and when to intervene.There are two further aspects that need to be considered when develop-ing a personalizedassistance system: inter- and intra-personalization. The former refers to the differencesbetween different users whereas the later indicates the changes in preferences in one userover time (i. e. the differences in driving styles when driving to work versus when being ona city sightseeing tour). In the scope of this thesis, both problems of inter- and intra-personalization are addressed and tackled. Our approach exploits the correlation in drivingstyle between consecutively executed maneuvers to quickly derive the driver’s current preferences. The introduced networks architecture outperforms non-personalized approachesin predicting the preference of driver when turning left. To tackle inter-personalizationprob-lems, this thesis makes use of the Siamese architecture with long short-term memorynetworks for identifying drivers based on vehicle dynamic information. The evaluation,which is carried out on real-world data set collected from 32 test drivers, shows that thenetwork is able to identify unseen drivers. Further analysis on the trained network indicates that it identifies drivers by comparing their behaviors, especially the approaching andturning behaviors.

Show publication details
Chen, Cong; Kuijper, Arjan [1. Review]; Damer, Naser [2. Review]

#### Advanced Analyses of CrazyFaces Attacks on Face Identification Systems

2020

After 5 years in prison, the greedy criminal was released. He never gave up the idea of sinagain. But he didn’t want to spend another 5 years in prison. So he began to summarizethe lessons of his last arrest. Five years ago, he was arrested at a bank because surveillancecameras identified him. This was a bit of a surprise to him, because this clever criminal hadrepeatedly escaped the pursuit of surveillance cameras by changing his facial expressions.After investigating, he learned that the face recognition system in that bank is a differentone. Therefore, his previously trained facial expressions failed. So one new idea comesin his mind now, "can i just find one or more facial expressions that can disable moststate-of-the-art face recognition systems?". To known the end of this story, please read therest of this thesis.

Show publication details
Wirtz, Marc Christopher; Kohlhammer, Jörn [Betreuer]; Unbescheiden, Matthias [Advisor]

#### AI-based Anomaly Detection supported by Explainable AI

2020

Ziel dieser Arbeit ist es, aufzuzeigen, wie eine sinnvolle Verknüpfung von Anomalieerkennung mit Machine Learning und Explainable AI aussehen könnte. Hierfür werden die beiden Bereiche theoretisch aufgearbeitet, eine Kategorisierung von Explainable AI Ansätzen entwickelt und anhand der für diese Arbeit bereitgestellten Daten ein Modell konzipiert, welches je eine Methode aus den beiden Bereichen zum Einsatz bringt und erklärt, wie xAI Methoden Anomalieerkennung unterstützen können.

Show publication details
Kunkel, Julian; Jumah, Nabeeh; Novikova, Anastasiia; Ludwig, Thomas; Yashiro, Hisashi; Maruyama, Naoya; Wahib, Mohamed; Thuburn, John

#### AIMES: Advanced Computation and I/O Methods for Earth-System Simulations

2020

Software for Exascale Computing - SPPEXA 2016-2019
Lecture Notes in Computational Science and Engineering (LNCSE)
136

Dealing with extreme scale earth system models is challenging from the computer science perspective, as the required computing power and storage capacity are steadily increasing. Scientists perform runs with growing resolution or aggregate results from many similar smaller-scale runs with slightly different initial conditions (the so-called ensemble runs). In the fifth Coupled Model Intercomparison Project (CMIP5), the produced datasets require more than three Petabytes of storage and the compute and storage requirements are increasing significantly for CMIP6. Climate scientists across the globe are developing next-generation models based on improved numerical formulation leading to grids that are discretized in alternative forms such as an icosahedral (geodesic) grid. The developers of these models face similar problems in scaling, maintaining and optimizing code. Performance portability and the maintainability of code are key concerns of scientists as, compared to industry projects, model code is continuously revised and extended to incorporate further levels of detail. This leads to a rapidly growing code base that is rarely refactored. However, code modernization is important to maintain productivity of the scientist working with the code and for utilizing performance provided by modern and future architectures. The need for performance optimization is motivated by the evolution of the parallel architecture landscape from homogeneous flat machines to heterogeneous combinations of processors with deep memory hierarchy. Notably, the rise of many-core, throughput-oriented accelerators, such as GPUs, requires non-trivial code changes at minimum and, even worse, may necessitate a substantial rewrite of the existing codebase. At the same time, the code complexity increases the difficulty for computer scientists and vendors to understand and optimize the code for a given system. Storing the products of climate predictions requires a large storage and archival system which is expensive. Often, scientists restrict the number of scientific variables and write interval to keep the costs balanced. Compression algorithms can reduce the costs significantly but can also increase the scientific yield of simulation runs. In the AIMES project, we addressed the key issues of programmability, computational efficiency and I/O limitations that are common in next-generation icosahedral earth-system models. The project focused on the separation of concerns between domain scientist, computational scientists, and computer scientists. The key outcomes of the project described in this article are the design of a model-independent Domain-Specific Language (DSL) to formulate scientific codes that can then be mapped to architecture specific code and the integration of a compression library for lossy compression schemes that allow scientists to specify the acceptable level of loss in precision according to various metrics. Additional research covered the exploration of third-party DSL solutions and the development of joint benchmarks (mini-applications) that represent the icosahedral models. The resulting prototypes were run on several architectures at different data centers.

Show publication details
Strassheim, Konstantin; Rus, Silvia [Betreuer]; Kuijper, Arjan [Betreuer]

#### Ambient Respiratory Rate Detection Using Capacitive Sensors Inside Seats

2020

The purpose of this research is to develop an application, to measure the human respiratory rate using capacitive sensors while seated. If the respiratory rate could be filtered out, it would provide a way to monitor the health status in daily life, and can also be used to raise alerts or taking actions in case of measuring abnormal respiratory rate. The sensors could be integrated in daily life, like car seats, chairs and sofas but also could serve as medical instruments in hospitals to measure respiratory rate ambiently.

Show publication details
Huber, Jonah; Mukhopadhyay, Anirban [1. Review]; Kuijper, Arjan [2. Review]

#### An Extensible Platform for EMT Experiments

2020

Electromagnetic tracking (EMT) procedures become increasingly more relevant in the domain of image-guided surgery (IGS). Electromagnetic tracking (EMT) is a promising navigation tool for situations in which line-ofsight is blocked. This is due to the fact, that under such circumstances, usual approaches like optical tracking fall short. While there are some actual applications in medical environments today, unfortunately none of them are embedded into widely adopted surgical procedures. Drawbacks that arise with the utilization of electromagnetic navigation are still preventing it from being integrated in certain medical therapy. Therefore problems and limitations need to be challenged. To approach the drawbacks in general, the idea was to create an extensible graphical user interface that acts as a platform for any kind of experiments involving EMT. The software ultimately aims to offer a universal platform where the user can connect a selection of electromagnetic tracking devices and is able to interact with them via various visualization widgets for specific types of applications. Thereby time will be saved writing custom software to interface with tracker systems while providing support for certain general necessities occurring in the area of tracking. This thesis presents the first iteration of the software. This first iteration is completely build from the ground up with no previous reference point. After an overview is given for the chosen development software, frameworks and tracking hardware, the key features of the platform that were achieved are shown. In its current state, the software allows the user to connect to a commercially available tracking system and get a real-time sensor data. The user can choose to either visualize the data in a 3-dimensional scene or save a set amount of measurements to a text file. In addition to that, a detailed dive into the exact implementation of certain features is provided. Moreover, a number of tests will be introduced to evaluate the performance of the basic tracking procedure and its visual presentation. Finally, further development of such a platform should help accelerate the goal of optimizing electromagnetic tracking for various areas to eventually help overcoming current limitations of EMT and profit from all its promising possibilities.

Show publication details
Ewald, Tobias; Reif, Ulrich [Referent]; Hormann, Kai [Korreferent]

#### Analyse geometrischer univariater Subdivisionsalgorithmen

2020

Subdivisionsalgorithmen generieren Freiformgeometrien durch iteratives Verfeinern polygonaler Daten, bspw. Polygonzüge bei univariater Subdivision. Dabei kann die Frage "Konvergiert die Polygonzugfolge gegen eine Grenzkurve und wie glatt ist diese?" im klassischen Fall linearer Algorithmen mit einer systematischen Regularitätstheorie beantwortet werden. Für nichtlineare Verfahren im Euklidischen, die unvermeidliche Nachteile linearer Algorithmen umgehen, gibt es nur Einzeluntersuchungen oder numerische Experimente. Diese Arbeit führt die große Klasse der geometrischen Subdivisionsschemata (GLUED-Schema) ein, zeigt für sie eine universelle $C^{2,\\alpha}$-Regularitätstheorie und gibt erstmalig rigorose Glattheitsnachweise für prominente Beispiele an. Besagte Klasse erweitert sich im Nichtstationären auf die GLUGs-Schemata, für die eine Konvergenztheorie angegeben ist. Letztlich vereinheitlicht eine allgemeingültige Proximitätstheorie für beliebige Algorithmen und beliebige $C^k$-Glattheit, genannt PAS-Theorie, die GLUED-, GLUGs- und lineare Theorien.

Show publication details

#### Analysis of Schedule and Layout Tuning for Sparse Matrices With Compound Entries on GPUs

2020

Computer Graphics Forum

Large sparse matrices with compound entries, i.e. complex and quaternionic matrices as well as matrices with dense blocks, are a core component of many algorithms in geometry processing, physically based animation and other areas of computer graphics. We generalize several matrix layouts and apply joint schedule and layout autotuning to improve the performance of the sparse matrix-vector product on massively parallel graphics processing units. Compared to schedule tuning without layout tuning, we achieve speedups of up to 5.5×. In comparison to cuSPARSE, we achieve speedups of up to 4.7×.

Show publication details
Grebe, Jonas Henry; Kuijper, Arjan [1. Gutachten]; Terhörst, Philipp [2. Gutachten]

#### Anomaly-based Face Search

2020

Biometric face identification refers to the use of face images for the automatic identification of individuals. Due to the high performance achieved by current face search algorithms, these algorithms are useful tools, e.g. in criminal investigations. Based on the facial description of a witness, the number of suspects can be significantly reduced. However, while modern face image retrieval approaches either require an accurate verbal description or an example image of the suspect’s face, eyewitness testimonies can seldom provide this level of detail. Moreover, while eyewitness’ recall is one of the most convincing pieces of evidence, it is also one of the most unreliable. Hence, exploiting the more reliable, but vague memories about distinctive facial features directly, such as obvious tattoos, scars or birthmarks, should be considered to filter potential suspects in a first step. This might reduce the risk of wrongful convictions caused by retroactively inferred details in the witness’ recall for subsequent steps. Therefore, this thesis proposes an anomaly-based face search solution that aims at enabling a reduction of the search space solely based on locations of anomalous facial features. We developed an unsupervised image anomaly detection approach based on a cascaded image completion network that allows to roughly localize anomalous regions in face images. (1) This completion model is assumed to fill in deleted regions with probable values conditioned on all the remaining parts of the face image. (2) The reconstruction errors of this model were used as an anomaly signal to create a grid of potential anomaly locations in a given face image. (3) These grids, in the form of a thresholded matrix, were then subsequently used to search for the most relevant images. We evaluated the respective retrieval model on a preprocessed subset of 17.855 images of the VGGFace2 dataset. The three main contributions of this work are (1) a cascaded face image completion approach, (2) an unsupervised inpainting-based anomaly localization approach, and (3) a query-by-anomaly face image retrieval approach. The face inpainting achieved promising results when compared to other recent completion approaches since we didn’t leverage any adversarial component in order to simplify the entire training procedure. These inpaintings enabled to roughly localize anomalies in face images. The proposed retrieval model achieved a 60% hit rate at a penetration rate of about 20% over a gallery of 17.855 images. Despite the limitations of the proposed searching approach, the results revealed the potential benefits of using the more reliable anomaly information to reduce the search space, instead of entirely relying on the elicitation of detailed perpetrator descriptions, either in textual or in visual form.

Show publication details
Bergmann, Tim Alexander; Kuijper, Arjan [1. Gutachten]; Noll, Matthias [2. Gutachten]

#### AR-Visualisierung von Echtzeitbildgebung für ultraschallgestützte Leberbiopsien

2020

In dieser Arbeit wird ein Augmented Reality-System zur Anzeige von Ultraschallbildern direkt am Patienten vorgestellt. Die Überlagerung wird mit der Hilfe von optisch durchsichtigen Head­mounted Displays durchgeführt. Die lagegerichtete Darstellung der Ultraschallbilder basiert auf einem externen optischen Trackingsystem, dem NDI Polaris Vicra. Um die korrekte Überlagerung zu gewährleisten, wird das Sichtfeld eines Trägers mittels angepasster Single Point Active Alignment Method bestimmt. Die Lage der Ultraschallbilder relativ zu den Tracking-Markierungen der Ultra­schallsonde wird mit einer angepassten Pivot-Kalibrierung ermittelt. Zum objektiven Testen des Systems wurde ein Träger-Dummy verwendet, der das Sehen eines Trägers durch Kameras simuliert. Die Lage von Tracking-Markierungen im Sichtfeld des Träger-Dummies konnte mit einem RMSE von 1,1480 mm bestimmt werden. Bei den Tests der Überlagerung der Ultraschallbilder über den darin repräsentierten Strukturen erreicht das System einen Dice-Koeffizienten von 88,33 %. Zur besseren Skalierung der Berechnungsdauer mit der Anzahl der verwendeten Geräte wurden Matrixoperatoren für die verwendeten Transformationsmatrizen optimiert. Die Berechnungen werden im Schnitt mehr als dreimal so schnell durchgeführt wie die allgemeine Implementierung der Operatoren. Das System versetzt behandelnde Ärzte in die Lage, Ultraschallbilder lagegerichtet über den darin repräsentierten Strukturen zu betrachten. Die Anzeige der Bilder auf einem externen Monitor wird dadurch überflüssig.

Show publication details
Preiner, Reinhold; Schmidt, Johanna; Krösl, Katharina; Schreck, Tobias; Mistelbauer, Gabriel

#### Augmenting Node-Link Diagrams with Topographic Attribute Maps

2020

EuroVis 2020. Eurographics / IEEE VGTC Conference on Visualization 2020

Eurographics / IEEE VGTC Conference on Visualization (EuroVis) <22, 2020, online>

We propose a novel visualization technique for graphs that are attributed with scalar data. In many scenarios, these attributes (e.g., birth date in a family network) provide ambient context information for the graph structure, whose consideration is important for different visual graph analysis tasks. Graph attributes are usually conveyed using different visual representations (e.g., color, size, shape) or by reordering the graph structure according to the attribute domain (e.g., timelines). While visual encodings allow graphs to be arranged in a readable layout, assessing contextual information such as the relative similarities of attributes across the graph is often cumbersome. In contrast, attribute-based graph reordering serves the comparison task of attributes, but typically strongly impairs the readability of the structural information given by the graph’s topology. In this work, we augment force-directed node-link diagrams with a continuous ambient representation of the attribute context. This way, we provide a consistent overview of the graph’s topological structure as well as its attributes, supporting a wide range of graph-related analysis tasks. We resort to an intuitive height field metaphor, illustrated by a topographic map rendering using contour lines and suitable color maps. Contour lines visually connect nodes of similar attribute values, and depict their relative arrangement within the global context. Moreover, our contextual representation supports visualizing attribute value ranges associated with graph nodes (e.g., lifespans in a family network) as trajectories routed through this height field. We discuss how user interaction with both the structural and the contextual information fosters exploratory graph analysis tasks. The effectiveness and versatility of our technique is confirmed in a user study and case studies from various application domains.

Show publication details

#### Automated 3D Mass Digitization for the GLAM Sector

2020

Archiving 2020 online. Final Program and Proceedings

Archiving <2020, Online>

Archiving Conference

The European Cultural Heritage Strategy for the 21st century has led to an increased demand for fast, efficient and faithful 3D digitization technologies for cultural heritage artefacts. Yet, unlike the digital acquisition of cultural goods in 2D which is widely used and automated today, 3D digitization often still requires significant manual intervention, time and money. To overcome this, the authors have developed CultLab3D, the world’s first fully automatic 3D mass digitization technology for collections of three-dimensional objects. 3D scanning robots such as the CultArm3D-P are specifically designed to automate the entire 3D digitization process thus allowing to capture and archive objects on a large-scale and produce highly accurate photo-realistic representations.

Show publication details

#### Automated Cephalometric Landmark Localization using a Coupled Shape Model

2020

Proceedings of the 2020 Annual Meeting of the German Society of Biomedical Engineering

Jahrestagung der Deutschen Gesellschaft für Biomedizinische Technik im VDE (BMT) <54, 2020, online>

Current Directions in Biomedical Engineering

Cephalometric analysis is an important method in orthodontics for the diagnosis and treatment of patients. It is performed manually in clinical practice, therefore automation of this time consuming task would be of great assistance. In order to provide dentists with such tools, a robust and accurate identification of the necessary landmarks is required. However, poor image quality of lateral cephalograms like low contrast or noise make this task difficult. In this paper, an approach for automatic landmark localization is presented and used to find 19 landmarks in lateral cephalometric images. An initial predicting of the individual landmark locations is done by using a 2-D coupled shape model to utilize the spatial relation between landmarks and other anatomical structures. These predictions are refined with a Hough Forest to determine the final landmark location. The approach achieves competitive performance with a successful detection rate of 70.24% on 250 images for the clinically relevant 2mm accuracy range.

Show publication details
Haescher, Marian; Chodan, Wencke; Höpfner, Florian; Bieber, Gerald; Aehnelt, Mario; Srinivasan, Karthik; Alt Murphy, Margit

#### Automated Fall Risk Assessment of Elderly Using Wearable Devices

2020

Journal of Rehabilitation and Assistive Technologies Engineering (RATE)

Introduction: Falls cause major expenses in the healthcare sector. We investigate the ability of supporting a fall risk assessment by introducing algorithms for automated assessments of standardized fall risk-related tests via wearable devices. Methods: In a study, 13 participants conducted the standardized 6-Minutes Walk Test, the Timed-Up-and-Go Test, the 30-Second Sit-to-Stand Test, and the 4-Stage Balance Test repeatedly, producing 226 tests in total. Automated algorithms computed by wearable devices, as well as a visual analysis of the recorded data streams, were compared to the observational results conducted by physiotherapists. Results: There was a high congruence between automated assessments and the ground truth for all four test types (ranging from 78.15% to 96.55%), with deviations ranging all well within one standard deviation of the ground truth. Fall risk (assessed by questionnaire) correlated with the individual tests. Conclusions: The automated fall risk assessment using wearable devices and algorithms matches the validity of the ground truth, thus providing a resourceful alternative to the effortful observational assessment, while minimizing the risk of human error. No single test can predict overall fall risk; instead, a much more complex model with additional input parameters (e.g., fall history, medication etc.) is needed.

Show publication details

#### Automatic Procedural Model Generation for 3D Object Variation

2020

The Visual Computer

3D objects are used for numerous applications. In many cases not only single objects but also variations of objects are needed. Procedural models can be represented in many different forms, but generally excel in content generation. Therefore this representation is well suited for variation generation of 3D objects. However, the creation of a procedural model can be time-consuming on its own. We propose an automatic generation of a procedural model from a single exemplary 3D object. The procedural model consists of a sequence of parameterizable procedures and represents the object construction process. Changing the parameters of the procedures changes the surface of the 3D object. By linking the surface of the procedural model to the original object surface, we can transfer the changes and enable the possibility of generating variations of the original 3D object. The user can adapt the derived procedural model to easily and intuitively generate variations of the original object. We allow the user to define variation parameters within the procedures to guide a process of generating random variations. We evaluate our approach by computing procedural models for various object types, and we generate variations of all objects using the automatically generated procedural model.

Show publication details
Mertz, Tobias; Guthe, Stefan [1. Gutachten]; Kuijper, Arjan [2. Gutachten]

#### Automatic View Planning for 3D Reconstruction of Objects with Thin Features

2020

View planning describes the process of planning view points, from which to record an object or environment for digitization. This thesis examines the applicability of view planning to the 3D reconstruction of insect specimens from extended depth of field images and depth maps generated with a focus stacking method. Insect specimens contain very thin features, such as legs and antennae, while the depth maps, generated during the focus stacking, contain large levels of uncertainty. Since focus stacking is usually not used for 3D reconstruction, there are no state-of-the-art view planning systems, which deal with the unique challenges of this data. Within this thesis, a view planning system with two components is designed to deal with the uncertainty explicitly. The first component utilizes volumetric view planning methods from well established research along with a novel sensor model, to represent the synthetic camera, generated from the focus stack. The second component is a novel 2D feature tracking module, which is designed to capture small details, which can not be recorded within a volumetric representation. The evaluation of the system shows that the application of view planning can still significantly reduce the time required for scene exploration and provide similar amounts of detail as an unplanned approach. Some future improvements are suggested, which may enable the system to capture even more detail.

Show publication details

#### AutoML Techniques for Medical Image Segmentation

2020

In computer vision tasks such as image classification and segmentation convolutional neural networks are becoming a de-facto state of the art. More and more complex models are becoming available to solve those challenging tasks. As those complex neural networks are prone to overfitting they require large annotated datasets to be trained on. In medical imaging, such large annotated datasets are not common. This is especially true for segmentation where medical experts need to label each pixel in a time-consuming manner. A commonly used approach to tackle this shortage of data is the application of data augmentation. To keep the generalization abilities of the network data augmentation tries to sample additional data from the data distribution. The current state of the art to accomplish this is to manually design a sequence of image transformations which are randomly applied to the training data. This manual design requires domain knowledge and results in suboptimal choices, caused by its complexity and model dependence. Therefore the demand for an automatic, task-specific, and data-driven augmentation strategy arises. This allows for better generalization abilities of the network as well as important insight into the data itself. In this work, we propose an efficient optimization framework to automatically design an augmentation strategy based on the model and data directly. The augmentation strategy is represented as a non-commutative sequence of image transformations, defined as operator, probability, and magnitude tuples. The search strategy utilizes an autoencoder network to relax the discrete search problem to a continuous one. This is accomplished by training a value estimator to predict the performance of a sequence’s latent space representation and utilizing gradient ascent. This performance prediction is based on a new measure that describes the generalization ability of the target network directly. We apply this automation to a challenging medical image segmentation problem and show the benefits for the network’s performance.

Show publication details

#### Automobilbau: Qualitätsprüfung mit Augmented Reality

2020

VDI-Z Zeitschrift für integrierte Produktionstechnik

Augmented-Reality-Technologien des Fraunhofer-Instituts für Graphische Datenverarbeitung IGD vereinen reale und digitale Produktionsumgebung und lassen so auf den ersten Blick Abweichungen zwischen Ist und Soll erkennen.

Show publication details
Kügler, David; Uecker, Marc; Kuijper, Arjan; Mukhopadhyay, Anirban

#### AutoSNAP: Automatically Learning Neural Architectures for Instrument Pose Estimation

2020

Medical Image Computing and Computer Assisted Intervention - MICCAI 2020

International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI) <23, 2020, Online>

Lecture Notes in Computer Science (LNCS)
12263

Despite recent successes, the advances in Deep Learning have not yet been fully translated to Computer Assisted Intervention (CAI) problems such as pose estimation of surgical instruments. Currently, neural architectures for classification and segmentation tasks are adopted ignoring significant discrepancies between CAI and these tasks. We propose an automatic framework (AutoSNAP) for instrument pose estimation problems, which discovers and learns architectures for neural networks. We introduce 1) an efficient testing environment for pose estimation, 2) a powerful architecture representation based on novel Symbolic Neural Architecture Patterns (SNAPs), and 3) an optimization of the architecture using an efficient search scheme. Using AutoSNAP, we discover an improved architecture (SNAPNet) which outperforms both the hand-engineered i3PosNet and the state-of-the-art architecture search method DARTS.

Show publication details
Fuchs, Moritz; Kuijper, Arjan [1. Review]; Mukhopadhyay, Anirban [2. Review]

#### Bayesian Deep Learning for Medical Image Analysis and Diagnosis

2020

Despite being the de-facto standard for medical image segmentation, researchers have identified shortcomings of frequentist U-Nets such as overconfidence and poor outof- distribution generalization. Although their Bayesian counterpart has already been proposed, often these methods rely on the well-known Monte-Carlo Drop Out (MCDO) approximation. We move beyond the MCDO approximation and introduce a novel multi-headed Bayesian U-Net. The proposed approach combines the global exploration capabilities of deep ensembles with the out-of-distribution robustness of Variational Inference. An efficient training strategy, along with an expressive yet general design, ensures superior approximation of the true Bayesian posterior within a reasonable compute requirement. Further we thoroughly analyze different properties of our model and give insights on other prior and regularization techniques. We evaluate our approach on the publicly available BRATS2018 dataset.

Show publication details

#### Beyond Identity: What Information Is Stored in Biometric Face Templates ?

2020

IJCB 2020. IEEE/IARP International Joint Conference on Biometrics

IEEE/IARP International Joint Conference on Biometrics (IJCB) <2020, online>

Deeply-learned face representations enable the success of current face recognition systems. Despite the ability of these representations to encode the identity of an individual, recent works have shown that more information is stored within, such as demographics, image characteristics, and social traits. This threatens the user's privacy, since for many applications these templates are expected to be solely used for recognition purposes. Knowing the encoded information in face templates helps to develop bias-mitigating and privacy-preserving face recognition technologies. This work aims to support the development of these two branches by analysing face templates regarding 113 attributes. Experiments were conducted on two publicly available face embeddings. For evaluating the predictability of the attributes, we trained a massive attribute classifier that is additionally able to accurately state its prediction confidence. This allows us to make more sophisticated statements about the attribute predictability. The results demonstrate that up to 74 attributes can be accurately predicted from face templates. Especially non-permanent attributes, such as age, hairstyles, haircolors, beards, and various accessories, found to be easily-predictable. Since face recognition systems aim to be robust against these variations, future research might build on this work to develop more understandable privacy preserving solutions and build robust and fair face templates.

Show publication details
Wenzl, Nico; Sachs, Moritz Karl [Betreuer]; Unbescheiden, Matthias [Betreuer]

#### Biased Decision Making in Venture Capital

2020

Im Rahmen einer Literaturrecherche wird betrachtet, welche Einflüsse und Verzerrungen beim Investitionsentscheidungsprozess von Venture Capitalists existieren. Motiviert durch praktische Relevanz und Vorteile für Investoren und Unternehmen werden 36 Artikel betrachtet, die zusammen 13 verschiedene Einflussfaktoren auf die Entscheidung untersuchen. Der Investitions-prozess besteht aus den vier Schritten Deal Origination, Screening, Evaluation und Structuring. Im Screening-Schritt beeinflussen die Passion des Unternehmers, der Gender Bias und die sozialen Verbindungen des Venture Capitalists die Entscheidung. Im Evaluation-Schritt wird der Einfluss von sozialen Verbindungen und Glück- oder Pechsträhnen untersucht, wohingegen im Structuring-Schritt der Einfluss von Zuversicht und Kontrolle, sowie der Gender Bias und Continuation Bias untersucht werden. Prozessübergreifend werden zusätzlich noch der Similarity Bias, Location Bias, Overconfidence Bias, die Verfügbarkeitsheuristik, der Einfluss von Erfahrung, sowie Short- beziehungsweise Long-termism betrachtet. Abschließend wird der Einsatz von Entscheidungsmodellen zur Milderung der Einflüsse und Verbesserung des Entscheidungsprozesses und der Einsatz von Entscheidungsmodellen in der Praxis diskutiert.

Show publication details
Iffland, Dominik; Kuijper, Arjan [1. Gutachten]; Efremov, Anton [2. Gutachten]

#### Bildsegmentierung und Erkennung von Farben auf Buchcovern

2020

Die automatische Erkennung von Farben auf Bildern ist im Allgemeinen schwierig, da die menschliche Wahrnehmung von Farbe sehr individuell und deshalb nicht universal berechenbar ist. Die einzelnen Farbwerte werden in den meisten Fällen in Gruppen wie Rot, Grün oder Gelb eingeteilt, wobei sich die Elemente innerhalb einer Gruppe oft unterscheiden. Speziell in der Buchbranche werden die aussagekräftigsten Farbgruppen für jedes Buchcover angegeben, da diese Farbinformationen in verschiedenen Bereichen, wie beispielsweise dem Marketing einen großen Mehrwert besitzen und für diverse Werbemaßnahmen eingesetzt werden. Das Ziel dieser Arbeit ist die Erforschung und Bewertung verschiedener Algorithmen zur Segmentierung und Farberkennung mit anschließender Extraktion der stärksten Farben in einem Bild. Dazu werden im Theorieteil verschiedene Methoden zur Farberkennung wie k-means vorgestellt und diskutiert. Die zur Segmentierung genutzten Algorithmen, wie diverse Kantenerkennungen oder Threshold Verfahren, werden ebenfalls diskutiert und ausgewertet. Anschließend wird die Farberkennung mit einer in Vorder- und Hintergrund segmentierten Variante verglichen. Die Einteilung der Bilder in die Segmente Vorder- und Hintergrund imitiert die menschliche Wahrnehmung von Bildern und erlaubt es diese Bereiche separat zu analysieren. Diese Segmentierung optimiert mit angepasster Gewichtung des Vorder- und Hintergrunds die Ergebnisse der Farberkennung. Die gemessenen Beobachtungen zeigen, dass eine Kombination von Farberkennung und gewichteter Segmentierung zu den profitabelsten Ergebnissen führt. Die Segmentierungsalgorithmen werden mithilfe verschiedener Datensets und Gütekriterien beurteilt. Eine anschließende Nutzerstudie soll zeigen, ob die Ergebnisse des gewählten Algorithmus zu einem der menschlichen Wahrnehmung entsprechenden Ergebnis führen. Die Farberkennung wird dabei auf zuvor ausgewählten Bildern angewendet und von verschiedenen Nutzern bezüglich der Qualität bewertet. Aufbauend auf den gewonnenen Ergebnissen der Segmentierung mit entsprechender Gewichtung der einzelnen Farben kann das resultierende Produkt in der Praxis und im speziellem Bereich der Buchbranche eingesetzt werden um die Qualität der Farberkennung deutlich zu steigern. Im praktischen Teil der Arbeit wird deshalb ein Programm entwickelt, das eine automatische Extraktion der kräftigsten Farben auf einem Bild durchführt.

Show publication details
Brömme, Arslan [Ed.]; Busch, Christoph [Ed.] [et al.]

#### BIOSIG 2020

2020

Conference on Biometrics and Electronic Signatures (BIOSIG) <19, 2020, Online>

GI-Edition - Lecture Notes in Informatics (LNI)
P-306
• 978-3-88579-700-5
Show publication details
Vaniet, Emmanuelle; Wesarg, Stefan

#### Blick in den Körper

2020

Spektrum der Wissenschaft

Die Entdeckung der Röntgenstrahlen läutete die klinische Bildgebung ein - und ermöglichte das äußerst leistungsfähige Verfahren der Computertomografie.

Show publication details
Venkatesh, Sushma; Zhang, Haoyu; Ramachandra, Raghavendra; Raja, Kiran; Damer, Naser; Busch, Christoph

#### Can GAN Generated Morphs Threaten Face Recognition Systems Equally as Landmark Based Morphs ? - Vulnerability and Detection

2020

IWBF 2020. Proceedings

International Workshop on Biometrics and Forensics (IWBF) <8, 2020, online>

The primary objective of face morphing is to com-bine face images of different data subjects (e.g. an malicious actor and an accomplice) to generate a face image that can be equally verified for both contributing data subjects. In this paper, we propose a new framework for generating face morphs using a newer Generative Adversarial Network (GAN) - StyleGAN. In contrast to earlier works, we generate realistic morphs of both high-quality and high resolution of 1024 × 1024 pixels. With the newly created morphing dataset of 2500 morphed face images, we pose a critical question in this work. (i) Can GAN generated morphs threaten Face Recognition Systems (FRS) equally as Landmark based morphs? Seeking an answer, we benchmark the vulnerability of a Commercial-Off-The-Shelf FRS (COTS) and a deep learning-based FRS (ArcFace). This work also benchmarks the detection approaches for both GAN generated morphs against the landmark based morphs using established Morphing Attack Detection (MAD) schemes.

Show publication details

#### Capability-based Scheduling of Scientific Workflows in the Cloud

2020

Proceedings of the 9th International Conference on Data Science, Technology and Applications

International Conference on Data Science, Technology and Applications (DATA) <9, 2020>

We present a distributed task scheduling algorithm and a software architecture for a system executing scientific workflows in the Cloud. The main challenges we address are (i) capability-based scheduling, which means that individual workflow tasks may require specific capabilities from highly heterogeneous compute machines in the Cloud, (ii) a dynamic environment where resources can be added and removed on demand, (iii) scalability in terms of scientific workflows consisting of hundreds of thousands of tasks, and (iv) fault tolerance because in the Cloud, faults can happen at any time. Our software architecture consists of loosely coupled components communicating with each other through an event bus and a shared database. Workflow graphs are converted to process chains that can be scheduled independently. Our scheduling algorithm collects distinct required capability sets for the process chains, asks the agents which of these sets they can manage, and then assigns process chains accordingly. We present the results of four experiments we conducted to evaluate if our approach meets the aforementioned challenges. We finish the paper with a discussion, conclusions, and future research opportunities. An implementation of our algorithm and software architecture is publicly available with the open-source workflow management system “Steep”.

Show publication details

#### Compact Models for Periocular Verification Through Knowledge Distillation

2020

BIOSIG 2020

Conference on Biometrics and Electronic Signatures (BIOSIG) <19, 2020, Online>

GI-Edition - Lecture Notes in Informatics (LNI)
P-306

Despite the wide use of deep neural network for periocular verification, achieving smaller deep learning models with high performance that can be deployed on low computational powered devices remains a challenge. In term of computation cost, we present in this paper a lightweight deep learning model with only 1.1m of trainable parameters, DenseNet-20, based on DenseNet architecture. Further, we present an approach to enhance the verification performance of DenseNet-20 via knowledge distillation. With the experiments on VISPI dataset captured with two different smartphones, iPhone and Nokia, we show that introducing knowledge distillation to DenseNet-20 training phase outperforms the same model trained without knowledge distillation where the Equal Error Rate (EER) reduces from 8.36% to 4.56% EER on iPhone data, from 5.33% to 4.64% EER on Nokia data, and from 20.98% to 15.54% EER on cross-smartphone data.

Show publication details
Stober, Gunter; Baumgarten, Kathrin; McCormack, John P.; Brown, Peter; Czarnecki, Jerry

#### Comparative Study between Ground-Based Observations and NAVGEM-HA Analysis Data in the Mesosphere and Lower Thermosphere Region

2020

Atmospheric Chemistry and Physics (ACP)

Recent studies have shown that day-to-day variability of the migrating semidiurnal solar (SW2) tide within the mesosphere and lower thermosphere (MLT) is a key driver of anomalies in the thermosphere–ionosphere system. Here, we study the variability in both the amplitude and phase of SW2 using meteor radar wind and lidar temperature observations at altitudes of 75–110 km as well as wind and temperature output from the Navy Global Environmental Model – High Altitude (NAVGEM-HA), a high-altitude meteorological analysis system. Application of a new adaptive spectral filter technique to both local radar wind observations and global NAVGEM-HA analyses offers an important cross-validation of both data sets and makes it possible to distinguish between migrating and non-migrating tidal components, which is difficult using local measurements alone. Comparisons of NAVGEM-HA, meteor radar and lidar observations over a 12-month period show that the meteorological analyses consistently reproduce the seasonal as well as day-to-day variability in mean winds, mean temperatures and SW2 features from the ground-based observations. This study also examines in detail the day-to-day variability in SW2 during two sudden stratospheric warming, events that have been implicated in producing ionospheric anomalies. During this period, both meteor radar and NAVGEM-HA winds show a significant phase shift and amplitude modulation, but no signs of coupling to the lunar tide as previous studies have suggested. Overall, these findings demonstrate the benefit of combining global high-altitude meteorological analyses with ground-based observations of the MLT region to better understand the tidal variability in the atmosphere.

Show publication details

#### Comparison-Level Mitigation of Ethnic Bias in Face Recognition

2020

IWBF 2020. Proceedings

International Workshop on Biometrics and Forensics (IWBF) <8, 2020, online>

Current face recognition systems achieve high performance on several benchmark tests. Despite this progress,recent works showed that these systems are strongly biasedagainst demographic sub-groups. Previous works introducedapproaches that aim at learning less biased representations.However, applying these approaches in real applications requiresa complete replacement of the templates in the database. Thisreplacement procedure further requires that a face image ofeach enrolled individual is stored as well. In this work, wepropose the first bias-mitigating solution that works on thecomparison-level of a biometric system. We propose a fairnessdriven neural network classifier for the comparison of twobiometric templates to replace the systems similarity function.This fair classifier is trained with a novel penalization termin the loss function to introduce the criteria of group andindividual fairness to the decision process. This penalization termforces the score distributions of different ethnicities to be similar,leading to a reduction of the intra-ethnic performance differences.Experiments were conducted on two publicly available datasetsand evaluated the performance of four different ethnicities. Theresults showed that for both fairness criteria, our proposedapproach is able to significantly reduce the ethnic bias, whileit preserves a high recognition ability. Our model, build onindividual fairness, achieves bias reduction rate between 15.35%and 52.67%. In contrast to previous work, our solution is easy tointegrate into existing systems by simply replacing the systemssimilarity functions with our fair template comparison approach.

Show publication details

#### Data Augmentation for Time Series: Traditional vs Generative Models on Capacitive Proximity Time Series

2020

Proceedings of the 13th ACM International Conference on PErvasive Technologies Related to Assistive Environments

ACM International Conference on PErvasive Technologies Related to Assistive Environments (PETRA) <13, 2020, Corfu, Greece>

ACM International Conference Proceedings Series (ICPS)

Large labeled quantities and diversities of training data are often needed for supervised, data-based modelling. Data distribution should cover a rich representation to support the generalizability of the trained end-to-end inference model. However, this is often hindered by limited labeled data and the expensive data collection process, especially for human activity recognition tasks. Extensive manual labeling is required. Data augmentation is thus a widely used regularization method for deep learning, especially applied on image data to increase the classification accuracy. But it is less researched for time series. In this paper, we investigate the data augmentation task on continuous capacitive time series with the example on exercise recognition. We show that the traditional data augmentation can enrich the source distribution and thus make the trained inference model more generalized. This further increases the recognition performance for unseen target data around 21.4 percentage points compared to inference model without data augmentation. The generative models such as variational autoencoder or conditional variational autoencoder can further reduce the variance on the target data.

Show publication details
Süßmilch, Marius; Kuijper, Arjan [1. Review]; Kohlhammer, Jörn [2. Review]

#### De-disguise Faces for Accurate Disguised Faces Recognition

2020

In recent years, face recognition has become an important and reliable application of deep learning used in many industries on a global scale, reaching from smartphone user identification to border security. Despite recent advances in the design of loss functions working with angular margins, disguise and impersonation still pose a challenge to face recognition systems. Even unintentional changes in a subject’s appearance can significantly reduce the accuracy of correctly classifying an individual as genuine or imposter. Although methods like ArcFace increase the detection rate of imposters, they do not attempt to visually remove disguise which will be the main focus of this work. Providing a de-disguised picture enhances the human ability to correctly identify a subject as genuine or imposter but can also increase the discriminative power of the state of the art classification models. Throughout this work, the method of de-disguising disguised faces will be derived from well-known concepts in the field of deep learning and evaluated in comparison to classic and cutting edge methods to better understand the promises and problems of this approach.

Show publication details
Kraft, Dimitri; Srinivasan, Karthik; Bieber, Gerald

#### Deep Learning Based Fall Detection Algorithms for Embedded Systems, Smartwatches, and IoT Devices Using Accelerometers

2020

Technologies

A fall of an elderly person often leads to serious injuries or even death. Many falls occur in the home environment and remain unrecognized. Therefore, a reliable fall detection is absolutely necessary for a fast help. Wrist-worn accelerometer based fall detection systems are developed, but the accuracy and precision are not standardized, comparable, or sometimes even known. In this work, we present an overview about existing public databases with sensor based fall datasets and harmonize existing wrist-worn datasets for a broader and robust evaluation. Furthermore, we are analyzing the current possible recognition rate of fall detection using deep learning algorithms for mobile and embedded systems. The presented results and databases can be used for further research and optimizations in order to increase the recognition rate to enhance the independent life of the elderly. Furthermore, we give an outlook for a convenient application and wrist device.

Show publication details

#### Deep Learning Multi-layer Fusion for an Accurate Iris Presentation Attack Detection

2020

FUSION 2020

International Conference on Information Fusion (FUSION) <23, 2020, Online>

Iris presentation attack detection (PAD) algorithms are developed to address the vulnerability of iris recognition systems to presentation attacks. Taking into account that the deep features successfully improved computer vision performance in various fields including iris recognition, it is natural to use features extracted from deep neural networks for iris PAD. Each layer in a deep learning network carries features of different level of abstraction. The features extracted from the first layer to the higher layers become more complex and more abstract. This might point our complementary information in these features that can collaborate towards an accurate PAD decision. Therefore, we propose an iris PAD solution based on multi-layer fusion. The information extracted from the last several convolutional layers are fused on two levels, feature-level and score-level. We demonstrated experiments on both, off-theshelf pre-trained network and network trained from scratch. An extensive experiment also explores the complementary between different layer combinations of deep features. Our experimental results show that feature-level based multi-layer fusion method performs better than the best single layer feature extractor in most cases. In addition, our fusion results achieve similar or better results than the state-of-the-art algorithms on the Notre Dame and IIITD-WVU databases of the Iris Liveness Detection Competition 2017 (LivDet-Iris 2017).

Show publication details
Dong, Jiangxin; Roth, Stefan; Schiele, Bernt

#### Deep Wiener Deconvolution: Wiener Meets Deep Learning for Image Deblurring

2020

Advances in Neural Information Processing Systems

Annual Conference on Neural Information Processing Systems (NeurIPS) <34, 2020, Online>

We present a simple and effective approach for non-blind image deblurring, combining classical techniques and deep learning. In contrast to existing methods that deblur the image directly in the standard image space, we propose to perform an explicit deconvolution process in a feature space by integrating a classical Wiener deconvolution framework with learned deep features. A multi-scale feature refinement module then predicts the deblurred image from the deconvolved deep features, progressively recovering detail and small-scale structures. The proposed model is trained in an end-to-end manner and evaluated on scenarios with both simulated and real-world image blur. Our extensive experimental results show that the proposed deep Wiener deconvolution network facilitates deblurred results with visibly fewer artifacts. Moreover, our approach quantitatively outperforms state-of-the-art non-blind image deblurring methods by a wide margin.

Show publication details
Drozdowski, Pawel; Rathgeb, Christian; Dantcheva, Antitza; Damer, Naser; Busch, Christoph

#### Demographic Bias in Biometrics: A Survey on an Emerging Challenge

2020

IEEE Transactions on Technology and Society

Systems incorporating biometric technologies have become ubiquitous in personal, commercial, and governmental identity management applications. Both cooperative (e.g. access control) and non-cooperative (e.g. surveillance and forensics) systems have benefited from biometrics. Such systems rely on the uniqueness of certain biological or behavioural characteristics of human beings, which enable for individuals to be reliably recognised using automated algorithms. Recently, however, there has been a wave of public and academic concerns regarding the existence of systemic bias in automated decision systems (including biometrics). Most prominently, face recognition algorithms have often been labelled as “racist” or “biased” by the media, non-governmental organisations, and researchers alike. The main contributions of this article are: (1) an overview of the topic of algorithmic bias in the context of biometrics, (2) a comprehensive survey of the existing literature on biometric bias estimation and mitigation, (3) a discussion of the pertinent technical and social matters, and (4) an outline of the remaining challenges and future work items, both from technological and social points of view.

Show publication details

#### Demographic Bias in Presentation Attack Detection of Iris Recognition Systems

2020

28th European Signal ProcessingConference (EUSIPCO 2020). Proceedings

European Signal Processing Conference (EUSIPCO) <28, 2020, online>

With the widespread use of biometric systems, the demographic bias problem raises more attention. Although many studies addressed bias issues in biometric verification, there are no works that analyze the bias in presentation attack detection (PAD) decisions. Hence, we investigate and analyze the demographic bias in iris PAD algorithms in this paper. To enable a clear discussion, we adapt the notions of differential performance and differential outcome to the PAD problem. We study the bias in iris PAD using three baselines (hand-crafted, transfer-learning, and training from scratch) using the NDCLD- 2013 [18] database. The experimental results point out that female users will be significantly less protected by the PAD, in comparison to males.

Show publication details
Wang, Yu; Yu, Weidong; Liu, Xiuqing; Wang, Chunle; Kuijper, Arjan; Guthe, Stefan

#### Demonstration and Analysis of an Extended Adaptive General Four-Component Decomposition

2020

IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing

The overestimation of volume scattering is an essentialshortcoming of the model-based polarimetric syntheticaperture radar (PolSAR) target decomposition method. It islikely to affect the measurement accuracy and result in mixedambiguity of scattering mechanism. In this paper, an extendedadaptive four-component decomposition method (ExAG4UThs)is proposed. First, the orientation angle compensation (OAC)is applied to the coherency matrix and artificial areas areextracted as the basis for selecting the decomposition method.Second, for the decomposition of artificial areas, one of the twocomplex unitary transformation matrices of the coherency matrixis selected according to the wave anisotropy (Aw). In addition, thebranch condition that is used as a criterion for the hierarchicalimplementation decomposition is the ratio of the correlationcoefficient (Rcc). Finally, the selected unitary transformationmatrix and discriminative threshold are used to determine thestructure of the selected volume scattering models, which aremore effectively to adapt to various scattering mechanisms. Inthis paper, the performance of the proposed method is evaluatedon GaoFen-3 full PolSAR data sets for various time periods andregions. The experimental results demonstrate that the proposedmethod can effectively represent the scattering characteristics ofthe ambiguous regions and the oriented building areas can bewell discriminated as dihedral or odd-bounce structures.

Show publication details
Bijan, Romal; Rus, Silvia [1. Review]; Kuijper, Arjan [2. Review]

#### Designing a Capacitive Sensing System – a Survey

2020

Through the years, sensors took a very important part in our lives. They help us to implement goals or simplify many things. We use them in many different ways and many things would not be as easy as they are without them. Especially in relation to Smart Environments and Human-Computer-Interactions, sensors take over several important and essential tasks. Therefor, we want to show the utility and importance of sensing systems in relation to these application areas. In this thesis, the focus is to find a general process chain on how to build sensing systems in general, and then focusing on capacitive sensing systems. This is done by reviewing literature and related work about general design approaches, systems using other sensors, which we will introduce as well, and then focusing on systems using capacitive sensors with a special focus on capacitive sensing on flexible surfaces. Among others, we elaborate several key aspects, which we use as a guideline to chose the most suitable design. Furthermore, a benchmark is given, which underlines our key aspects and can be used as well to help us to achieve our aim. Moreover, based on the guidelines, designs and techniques we introduce in relation to their application area, we will be able to point out similarities between the presented systems and compare the systems based on these factors and the characteristics we expect from our design. Finally, after comparing them and show their advantages and limitations, we will be able to recommend a design and justify it trough our learned lessons.

Show publication details
Dyroff, Luisa; Kuijper, Arjan [1. Review]; Rus, Silvia [2. Review]

#### Designing Flexible Capacitive Electrodes based on Performance Measurement while Deformation

2020

In recent years, capacitive sensing has become more and more popular due to its versatile deployment possibilities such as in wearables or in smart living applications. For these purposes, the sensing electrodes often need to be flexible and integrated into textiles. Yet, there is not much research regarding the design of such electrodes and in particular their behavior while being deformed, although flexible sensors are frequently exposed to deformation. Therefore, in this thesis, flexible capacitive sensors and their design will be explored, focusing on the materials, shape and size of the electrodes. Further, it will be investigated how deformation of flexible electrodes impacts the sensing performance. For this aim, an extensive literature review on different electrode characteristics and their influence on sensing performance has been conducted. Besides, a measurement series with capacitive textile electrodes under deformation and also applied pressure has been performed. Based on the results of the literature review and the measurement series, a recommendation for designing flexible capacitive electrodes will be provided.

Show publication details

#### Designing Smart Home Controls for Elderly

2020

Proceedings of the 13th ACM International Conference on PErvasive Technologies Related to Assistive Environments

ACM International Conference on PErvasive Technologies Related to Assistive Environments (PETRA) <13, 2020, Corfu, Greece>

ACM International Conference Proceedings Series (ICPS)

Technology is evolving by the day and with it the devices to control it. Sophisticated systems, like Smart Homes, are currently controlled in most cases via a smartphone app. While this may be acceptable for younger and middle-aged people, elders, however, have trouble keeping up with new devices and might not want to use a smartphone. Most modern-day control schemes like touch screens and menus are regarded as too complicated. However, Smart Homes provide many opportunities to reduce the every-day burden on elderly and people with special needs. Providing elderly people easy access to advanced and helpful technology via familiar interface types immensely improves their quality of life.We propose a Smart Home control designed especially for use by elderly. Our contribution ranges from evaluating existing systems to designing and building the Smart Home control for elderly based on their special requirements. Moreover, we involve elderly in the design process and evaluate the proposed prototype in a qualitative study with 10 elderly users. The results conclude that being presented with the scenario to already own the required Smart Home technology, the participants were quick to accept the cube as user friendlier when compared to smartphone controls or touchscreen controls in general.

Show publication details
Krispel, Ulrich; Fellner, Dieter W.; Ullrich, Torsten

#### Distance Measurements of CAD Models in Boundary Representation

2020

Transactions on Computational Science XXXVI
Lecture Notes in Computer Science (LNCS), Transactions on Computational Science
12060, 12060

The need to analyze and visualize distances between objects arises in many use cases. Although the problem to calculate the distance between two polygonal objects may sound simple, real-world scenarios with large models will always be challenging, but optimization techniques – such as space partitioning – can reduce the complexity of the average case significantly. Our contribution to this problem is a publicly available benchmark to compare distance calculation algorithms. To illustrate the usage, we investigated and evaluated a grid-based distance measurement algorithm.

Show publication details
Reiß-Wöltche, Dominik Bernd; Kuijper, Arjan [1. Gutachten]; Bielinski, Lukas [2. Gutachten]

#### Effiziente Softwareentwicklung in kleinen Teams am Beispiel der Entwicklung einer Android App für WLAN-Onboarding mithilfe von Bluetooth Beacons

2020

Zunehmende Digitalisierung und Vernetzung sind mitunter Grund für den steigenden Bedarf von individuellen Softwarelösungen. Eine Herausforderung, die damit einhergeht, ist genügend Softwareentwickler zu finden. Viele Softwareentwicklungsteams sind klein oder unterbesetzt und müssen wegen dieser Resscourcenknappheit einen Kompromiss zwischen Qualitätsicherungsmaßnahmen und Entwicklung der Funktionalität finden - oft zum Nachteil der Qualität. Das Ziel dieser Arbeit ist es, einen Softwareentwicklungsprozess zu entwerfen, der für kleine Teams optimiert ist und eine effiziente Entwicklung qualitativ hochwertiger Software ermöglicht. Dazu beschäftigt sich die Arbeit mit den Fragen: • Wie sind Softwareentwicklungsprozesse grundlegend gestaltet? • Welche Anforderungen aus Team- und Projektkontext beeinflussen die Gestaltung eines Softwareentwicklungsprozesses? • Was bedeutet Effizienz in der Softwareentwicklung? Zur Beantwortung dieser Fragen wurde recherchiert, welches die fundamentalen Phasen von Softwareentwicklungsprozessen sind, analysiert, welche Aspekte bei Softwareprojekten am aufwändigsten sind und betrachtet, welche Herausforderungen bei Projekten in kleinen Entwicklungsteams auftreten. Die Resultate wurden genutzt, um die für effiziente Entwicklung maßgebenden Phasen eines Prozesses hervorzuheben und zu optimieren. Das Ergebnis ist ein modular aufgebauter Prozess, der zum einen an den Wechsel und die Veränderung des Entwicklerteams während des gesamten Lebenszyklus eines Produktes, zum anderen an spezifische Projektanforderungen angepasst werden kann. Dazu wird eine Prozessvorlage zu Beginn eines Projektes vom Entwicklerteam mithilfe einer definierten Methodik vervollständigt. Ausgehend von Teamfluktuation und Projektumfang wird mit der Methodik die Relevanz und der Ausprägungsgrad von Prozessaktivitäten festgelegt. Um die definierte Methodik und den entworfenen Prozesses zu evaluieren, wurden diese zur Entwicklung einer mobilen Applikation genutzt. Es wurde bewertet, inwieweit Anforderungen des Team- und Projektkontextes, sowie weitere Kriterien aus der Problemstellung durch den definierten Prozess umgesetzt, und welche Auswirkungen auf die Entwicklungsperformanz festgestellt werden konnten.

Show publication details
Becker, Hagen; Kuijper, Arjan [1. Gutachten]; Tazari, Mohammad-Reza [2. Gutachten]

#### Einführung von Serious-Gaming Techniken in die Digitale Physiotherapie der Zukunft

2020

Im Rahmen dieser Arbeit wurde das Spiel PDDanceCity, welches der Bewegungsförderung dient aber auch kognitive Fähigkeiten trainieren soll, in mehreren Punkten erweitert. Das Spiel befand sich zu Beginn der Arbeit noch im Status eines Proof of Conecpt und ist vor allem für ältere Menschen konzeptioniert worden, in dem sich der Spieler auf einer Tanzfläche physisch bewegen muss, um seine Spielfigur durch ein Labyrinth ans Ziel zu steuern. Die Erweiterungen beinhalten zum einen die Möglichkeit das Spiel in der modernen Physiotherapie einzusetzen. Durch das in der Arbeit umgesetzte Profilverwaltungssystem ist es möglich, dass Physiotherapeuten die Entwicklung eines Patienten beobachten, begleiten und gegebenenfalls die Therapie anpassen können. Weitere Punkte waren die Verbesserung und Automatisierung der Steuereinheit des Spieles sowie die Erarbeitung eines Algorithmus für die Generierung der einzelnen Spielkarten, basierend auf den Einstellungen der Profile der Spieler. Zu Beginn musste die Tanzmatte mit der man das Spiel steuert für jeden Spieler neu eingestellt werden und die Generierung einer Spielkarte war zufällig und bezog sich nicht auf Spielerprofile. Durch diese Arbeit entstand ein Algorithmus, welcher die Spielkarten individuell basierend auf den Einstellungen des jeweiligen Spielers generierte. Auch wurde die Kommunikation der Tanzmatte mit dem Spiel verbessert, sodass zukünftig die Kalibrierung der Tanzmatte für jeden einzelnen Spieler entfällt. Außerdem ist es nun durch Gesichtserkennung möglich sich in sein Spielerprofil einzuloggen. Dies soll die Akzeptanz von älteren Menschen verbessern, da sie durch diese Technologie nicht mit Maus und Tastatur agieren müssen und einfacher und schneller das Spiel starten können. Weiterhin wurde eine Studie in einer Einrichtung für ältere Menschen durchgeführt, um Zusammenhänge zwischen der Fitness und der Spielweise eines Probanden zu untersuchen. Der Fitnesszustand jedes Spielers wurde mittels eines unabhängigen Fitnesstestes ermittelt. Nach dem Spielen von PDDanceCity wurden mit verschiedenen Maschinellen Lernen-Algorithmen die Bewegungsdaten der Probanden ausgewertet. Dadurch können künftig Rückschlüsse auf die Fitness der Probanden abhängig von ihrer Spielweise geschlossen werden.

Show publication details
Sahu, Manish; Strömsdörfer, Ronja; Mukhopadhyay, Anirban; Zachow, Stefan

#### Endo-Sim2Real: Consistency Learning-Based Domain Adaptation for Instrument Segmentation

2020

Medical Image Computing and Computer Assisted Intervention - MICCAI 2020

International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI) <23, 2020, Online>

Lecture Notes in Computer Science (LNCS)
12263

Surgical tool segmentation in endoscopic videos is an important component of computer assisted interventions systems. Recent success of image-based solutions using fully-supervised deep learning approaches can be attributed to the collection of big labeled datasets. However, the annotation of a big dataset of real videos can be prohibitively expensive and time consuming. Computer simulations could alleviate the manual labeling problem, however, models trained on simulated data do not generalize to real data. This work proposes a consistency-based framework for joint learning of simulated and real (unlabeled) endoscopic data to bridge this performance generalization issue. Empirical results on two data sets (15 videos of the Cholec80 and EndoVis’15 dataset) highlight the effectiveness of the proposed Endo-Sim2Real method for instrument segmentation. We compare the segmentation of the proposed approach with state-of-the-art solutions and show that our method improves segmentation both in terms of quality and quantity.

Show publication details
Kraft, Dimitri; Bader, Rainer; Bieber, Gerald

#### Enhancing Vibroarthrography by using Sensor Fusion

2020

Proceedings of the 9th International Conference on Sensor Network

International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP) <15, 2020, Valetta, Malta>

Natural and artificial joints of a human body are emitting vibration and sound during the movement. The sound and vibration pattern of a joint is characteristic and changes due to damage, uneven tread wear, injuries, or other influences. Hence, the vibration and sound analysis enables an estimation of the joint condition. This kind of analysis, vibroarthrography (VAG), allows the analysis of diseases like arthritis or osteoporosis and might determine trauma, inflammation, or misalignment. The classification of the vibration and sound data is very challenging and needs a comprehensive annotated data base. Current existing data bases are very limited and insufficient for deep learning or artificial intelligent approaches. In this paper, we describe a new concept of the design of a vibroarthrography system using a sensor network. We discuss the possible improvements and we give an outlook for the future work and application fields of VAG.

Show publication details
Bersch, Jessica; Wiedling, Hans-Peter [Referent]; Trapp, Ute [Korreferentin]; Tausch, Reimar [Erster Betreuer]; Domajnko, Matevz [Zweiter Betreuer]

#### Erweiterter 2D Farbmanagement Workflow für 3D Digitalisierung

2020

Konsistente und der Realität entsprechende Farbwiedergabe ist bei Digitalisierung oftmals ein wichtiger Faktor. Insbesondere bei der Erfassung von Kulturerbe spielt dies eine große Rolle. Für den 3D Bereich gibt es jedoch bislang keine veröffentlichten Untersuchungen bzgl. der Farbqualität im Vergleich zu 2D Aufnahmen oder spezielle Empfehlungen für einen vollständigen Farbmanagement Workflow. Das Ziel der vorliegenden Arbeit war daher, zu untersuchen, welche Faktoren sich auf Farbe und Farbwahrnehmung auswirken und wie sie dies tun. Insbesondere wurde ermittelt, wie sich Farbmanagement im 3D Bereich im Vergleich zum 2D Bereich auswirkt und wie ein optimales Ergebnis durch Übertragung und Erweiterung eines 2D Workflows erreicht werden kann. Betrachtet wurden die drei Kriterien Farbabstand (Delta E), Rauschen und Farbneutralität. Dazu wurde anhand der Technical Guidelines for Digitizing Cultural Heritage Materials ein 2D Workflow entwickelt. Dieser wurde auf eine digitale 3D Abbildung (Rekonstruktion) eines flachen Objekts übertragen und entsprechend erweitert. Die Ergebnisse bei verschiedenen Einstellungen des Tools Agisoft Metashape für die Rekonstruktion, sowie bei Anwendungsvarianten von ICC-Profilen und Maskierung, wurden mit der 2D Abbildung und untereinander verglichen. Die Untersuchungen zeigen, dass der Farbmanagement Workflow allgemein eine sichtbare und messbare Verbesserung bewirkt. Die 3D Rekonstruktion weist gegenüber dem 2D Vergleichsbild einen deutlich schlechteren Farbabstand, jedoch bessere Farbneutralität und Rauschwerte auf. Optimale Farbwiedergabe wird durch den Blending Mode Mosaic oder Max Intensity bei der Rekonstruktion und die Anwendung eines ICC-Profils auf den Textur- Atlas erreicht. Zudem sollte keine weitere Color Calibration durch Agisoft durchgeführt werden. Der Lichtabfall zeigt einen starken Einfluss, daher sollte dieser durch z.B. Depth of Field Masking ausgeglichen werden. Als problematisch haben sich auch Reflexionen und der Weißabgleich herausgestellt. Letzterer muss für die gesamte Aufnahme fest eingestellt sein. Die Reflexionen sollten mit einem Polfilter entfernt werden.

Show publication details

#### Evaluierung verschiedener Detektoren als Basis für Single Objekt Tracking in Multi Objekt Fisch Szenen mittels Convolutional Neural Networks

2020

Stralsund, Hochschule, Bachelor Thesis, 2020

Ziel dieser Bachelorarbeit ist es einen Überblick über Tracking verfahren zu geben, die neuronale Netze verwenden und aus diesen eine Architektur auszuwählen und diese exemplarisch zu implementieren. Dazu wird ein Ansatz ausgewählt, der sich für das Austauschen von den Detektoren anbietet, da ein weiteres Ziel dieser Bachelorarbeit ist herauszufinden ob es einen Unterschied in der Performanz der implementierten Architektur gibt, wenn Faster-RCNN [Gir15] oder Yolov3 [RF18] als Detektor verwendet werden. Weiterhin ist es das Ziel dieser Bachelorarbeit das Tracking von Fischen anhand des hier implementierten Systems zu testen. Dies ergibt sich aus dem hier gegebenem Dataset.

Show publication details

#### ExerTrack - Towards Smart Surfaces to Track Exercises

2020

Technologies

The concept of the quantified self has gained popularity in recent years with the hype of miniaturized gadgets to monitor vital fitness levels. Smartwatches or smartphone apps and other fitness trackers are overwhelming the market. Most aerobic exercises such as walking, running, or cycling can be accurately recognized using wearable devices. However whole-body exercises such as push-ups, bridges, and sit-ups are performed on the ground and thus cannot be precisely recognized by wearing only one accelerometer. Thus, a floor-based approach is preferred for recognizing whole-body activities. Computer vision techniques on image data also report high recognition accuracy; however, the presence of a camera tends to raise privacy issues in public areas. Therefore, we focus on combining the advantages of ubiquitous proximity-sensing with non-optical sensors to preserve privacy in public areas and maintain low computation cost with a sparse sensor implementation. Our solution is the ExerTrack, an off-the-shelf sports mat equipped with eight sparsely distributed capacitive proximity sensors to recognize eight whole-body fitness exercises with a user-independent recognition accuracy of 93.5% and a user-dependent recognition accuracy of 95.1% based on a test study with 9 participants each performing 2 full sessions. We adopt a template-based approach to count repetitions and reach a user-independent counting accuracy of 93.6 %. The final model can run on a Raspberry Pi 3 in real time. This work includes data-processing of our proposed system and model selection to improve the recognition accuracy and data augmentation technique to regularize the network.

Show publication details
Schurig, Martin Ralf; Hergenröther, Elke [Referent]; Frömmer, Björn [Korreferent]; Tausch, Reimar [Erster Betreuer]; Domajnko, Matevz [Zweiter Betreuer]

#### Extended 2D Image Quality Assessment for Photogrammetric 3D Digitization

2020

With recent technical progress the digitization of cultural heritage has become increasingly practicable and important. Many museums capture their artifacts to document and preserve their current state or to have a digital copy in case the original is damaged, destroyed or being restored. The majority of this digitization today is achieved by photographing or flatbed scanning of artifacts, thus producing a 2D image suited to capture 2D documents but limited to only a specific view angle for a 3D object. Therefore, in recent years, the application of new 3D scanners that can completely capture an object’s surface from all sides have become more popular. For both, 2D and 3D digitization methods, the goal is to digitally represent the original artifact in the most realistic and authentic way. Therefore, in 2D digitization, guidelines have been developed to estimate and assure the quality of the captured data. One widely used standard is the ISO norm 19264 which considers a large number of image quality criteria, such as sharpness and color accuracy. However, currently no comparable guidelines exist for the 3D digitization of artifacts. This thesis bridges part of the gap between 2D and 3D. It focuses on increasing the achievable resolution of the reconstructed texture and geometry. For this purpose, the quality characteristics from ISO 19264, which describe the level of detail of a 2D image, are extended to 2.5D so that they describe the complete depth of field of the image. Afterward, the effect of different camera-, image- and reconstruction-settings on the 3D results are investigated. For this purpose, the obtained point cloud is investigated both visually and based on the VDI/VDE 2634 guideline. A clear correlation between the quality of the camera parameters and the achievable reconstruction result can be shown. Thus, best practices for the settings can be derived, which allow for reconstruction details in the texture and geometry, which are not visible to the naked eye. The best practices describe, among other things, that the used image sections must be limited by the depth of field, how to calculate the texture size of the reconstruction based on the 2D sampling rate and that it is better to use a smaller depth of field with better quality then a larger one with slightly worse quality. At the end of the thesis, the current limitations of the 3D quality evaluation methods are discussed and an outlook is given describing how these can be even further improved in the future to enable a better evaluation of the 3D results.

Show publication details
Gan, Guan Teck; Scherer, Florian [Supervisor]; Kuijper, Arjan [Supervisor]; Winner, Hermann [Supervisor]

#### Extension of a Camera-Based Stationary Measurement Concept to Record the Position of the Motorcycle Rider Using Machine Learning

2020

In the present master thesis, a stationary measurement concept is extended to measure the riding position of motorcyclists during cornering. The focus is the backside detection of the rider’s pose using a camera with the help of deep learning. Based on this, the lean angle is calculated depending on the position of the motorcycle in space. Various deep learning models for 2D human pose estimation are examined. Through the models Mask R-CNN and HRNet, the pose of a motorcyclist is computed with little modification. The rider is represented by 15 body joints with special focus on the spine. The performance of both models is compared, and the best model is selected. The corresponding training data is recorded on a closed test area. These are labeled with a specially developed annotation tool. In combination with a license plate detector and the IPPE algorithm (Infinitesimal Plane-based Pose Estimation), which calculates the position of the motorcycle in space, the lean angle of the rider is determined. Furthermore, information such as the rider’s lateral position on the motorcycle, lean-in and lean-out positions is detected. A comparison with reference data recorded on a closed test area is evaluated to review the quality of the predictions created by the model. Based on further external data sources, it is examined whether the system is deployable in real road traffic and which limitations the detection will face. With the predictions of the adapted model, the rider’s lean angle and seating position can be clearly determined under certain conditions. The license plate detector does not provide reliable predictions in every scenario. Finally, possible improvements regarding training data generation, rider detection and license plate detection are mentioned and discussed.

Show publication details
Scherhag, Ulrich; Fellner, Dieter W. [1. Gutachten]; Busch, Christoph [2. Gutachten]; Veldhuis, Raymond [3. Gutachten]

#### Face Morphing and Morphing Attack Detection

2020

In modern society, biometrics is gaining more and more importance, driven by the increase in recognition performance of the systems. In some areas, such as automatic border controls, there is no alternative to the application of biometric systems. Despite all the advantages of biometric systems, the vulnerability of these still poses a problem. Facial recognition systems for example offer various attack points, like faces printed on paper or silicone masks. Besides the long known and well researched presentation attacks there is also the danger of the so-called morphing attack. The research field of morphing attacks is quite young, which is why it has only been investigated to a limited extent so far. Publications proposing algorithms for the detection of morphing attacks often lack uniform databases and evaluation methods, which leads to a restricted comparability of the previously published work. Thus, the focus of this thesis is the comprehensive analysis of different features and classifiers in their suitability as algorithms for the detection of morphing attacks. In this context, evaluations are performed with uniform metrics on a realistic morphing database, allowing the simulation of various realistic scenarios. If only the suspected morph is available, a HOG feature extraction in combination with an SVM is able to detect morphs with a D-EER ranging from 13.25% to 24.05%. If a trusted live capture image is available in addition, for example from a border gate, the deep ArcFace features in combination with an SVM can detect morphs with a D-EER ranging from 2.71% to 7.17%.

Show publication details

#### Face Presentation Attack Detection in Ultraviolet Spectrum via Local and Global Features

2020

BIOSIG 2020

Conference on Biometrics and Electronic Signatures (BIOSIG) <19, 2020, Online>

GI-Edition - Lecture Notes in Informatics (LNI)
P-306

The security of the commonly used face recognition algorithms is often doubted, as they appear vulnerable to so-called presentation attacks. While there are a number of detection methods that are using different light spectra to detect these attacks this is the first work to explore skin properties using the ultraviolet spectrum. Our multi-sensor approach consists of learning features that appear in the comparison of two images, one in the visible and one in the ultraviolet spectrum. We use brightness and keypoints as features for training, experimenting with different learning strategies. We present the results of our evaluation on our novel Face UV PAD database. The results of our method are evaluated in an leave-one-out comparison, where we achieved an APCER/BPCER of 0%/0.2%. The results obtained indicate that UV images in presentation attack detection include useful information that are not easy to overcome.

Show publication details

#### Face Quality Estimation and Its Correlation to Demographic and Non-Demographic Bias in Face Recognition

2020

IJCB 2020. IEEE/IARP International Joint Conference on Biometrics

IEEE/IARP International Joint Conference on Biometrics (IJCB) <2020, online>

Face quality assessment aims at estimating the utility of a face image for the purpose of recognition. It is a key factor to achieve high face recognition performances. Currently, the high performance of these face recognition systems come with the cost of a strong bias against demographic and non-demographic sub-groups. Recent work has shown that face quality assessment algorithms should adapt to the deployed face recognition system, in order to achieve highly accurate and robust quality estimations. However, this could lead to a bias transfer towards the face quality assessment leading to discriminatory effects e.g. during enrolment. In this work, we present an in-depth analysis of the correlation between bias in face recognition and face quality assessment. Experiments were conducted on two publicly available datasets captured under controlled and uncontrolled circumstances with two popular face embed-dings. We evaluated four state-of-the-art solutions for face quality assessment towards biases to pose, ethnicity, and age. The experiments showed that the face quality assessment solutions assign significantly lower quality values towards subgroups affected by the recognition bias demonstrating that these approaches are biased as well. This raises ethical questions towards fairness and discrimination which future works have to address.

Show publication details
Kohlhammer, Jörn; Angelini, Marco; Bryan, Chris; Romero-Gómez, Rosa; McKenna, Sean; Prigent, Nicholas

#### Foreword

2020

VizSec 2020

IEEE Symposium on Visualization for Cyber Security (VizSec) <17, 2020, online>

Welcome message from the proceedings "IEEE Symposium on Visualization for Cyber Security (VizSec)".

Show publication details
Gove, Robert; Bryan, Chris; Arendt, Dustin; McKenna, Sean; Kohlhammer, Jörn; Prigent, Nicholas; Angelini, Marco; Najafi, Parnian; Paul, Celest Lyn; Sopan, Awalin

#### Foreword

2020

VizSec 2019

IEEE Symposium on Visualization for Cyber Security (VizSec) <16, 2019>

Presents the introductory welcome message from the conference proceedings. May include the conference officers' congratulations to all involved with the conference event and publication of the proceedings record.

Show publication details
Kutlu, Hasan; Weinmann, Andreas [Referee]; Ritz, Martin [Co-Referee]

#### Fully Automatic Mechanical Scan Range Extension of a Lens-Shifted Structured Light System

2020

Cultural heritage are precious goods which need to be preserved for coming generations. Due to many reasons, e.g., wars or natural decay, those objects are in danger of destruction. In order to prevent them from being lost forever, those objects are digitized as 3D models to be accessible for further generations of mankind, the Fraunhofer Institute for computer graphics research offers a fully automatic 3D digitization system called the CultLab3D. There is already a fully functional system for big objects. However, it is more difficult to scan small objects like coins or rings. Those small objects are often referred to as 2.5D objects because they often got engravings and inscriptions on their surface, which cannot even be felt with ones fingers. Scanning such fine detailed objects needs a system that can measure such details. This is accomplished by the MesoScannerV2, an extension of the CultLab3D. It is designed for the digitization of these 2.5D objects without missing details. The MesoScannerV2 is a structured light system which uses a special variation of the phase shift method in order to improve the accuracy of the digitized 3D model of the object. The structured light-based MesoScannerV2 reaches an advanced depth and lateral resolution due to its specialty, the extension of state-of-the-art fringe patterns by a mechanical lens-shifted surface encoding method. Due to bad data acquisition and due to possible uncertainties of numerical algorithms noise is generated which directly influences the digitized 3D models. Therefore, this thesis aims to reduce the generated noise to get cleaner 3D models. Furthermore, the MesoScannerV2 needs to be future-proof, which requires an automation of the scan process of many objects at the same time. The integration of an automation procedure to the MesoScannerV2 is another topic discussed in this thesis. We show that methods are found to reduce the generated noise significantly in particular, we provide a corresponding evaluation. Further, possible solutions to automate the scan process could be found.

Show publication details

#### Function as a Service for the Storage of Large Geospatial Data

2020

Applications grow over time. While they are usually small at the beginning, more and more features are added over the years. At some point, this leads to a heavy monolithic system. On the other hand, there is a trend towards deploying applications in smaller units. The most recent stage of this development is Function as a Service (FaaS). It uses isolated, short-running and stateless functions. This reduces the complexity of individual functions compared to the entire monolithic application. Furthermore, a framework can scale the functions up and down as needed. To run an existing monolithic application as FaaS, it has to be adjusted. This thesis presents a strategy for the migration process. It analyzes the processing flow in the monolithic application and defines criteria for a division into functions. The suitability of the process is demonstrated based on an existing application. For this purpose two functional concepts with different focuses are developed. The first one preserves the backwards compatibility to the monolithic application and thus allows a flexible change between a monolithic operation and a FaaS execution. The second concept focuses on a fine division of functions. Here the compatibility to the monolithic application is lost, but the implementation becomes more flexible and can be extended more easily. To execute the designed functions, an improved scaling metric is presented. It is based on the number of outstanding function requests and integrates into the underlying FaaS-Framework. The evaluation shows that the presented functional concepts are suitable for processing real world data. Both concepts lead to an speed up in processing compared to the monolithic application. However, this performance gain is accompanied by an increased resource consumption, so that the use of a FaaS-based solution must be weighed up depending on the situation.

Show publication details
Boutros, Fadi; Damer, Naser; Raja, Kiran; Ramachandra, Raghavendra; Kirchbuchner, Florian; Kuijper, Arjan

#### Fusing Iris and Periocular Region for User Verification in Head Mounted Displays

2020

FUSION 2020

International Conference on Information Fusion (FUSION) <23, 2020, Online>

The growing popularity of Virtual Reality and Augmented Reality (VR/AR) devices in many applications also demands authentication of users. As the devices inherently capture the eye image while capturing the user interaction, the authentication can be devised using the iris and periocular recognition. While both iris and periocular data being non-ideal unlike the data captured from standard biometric sensors, the authentication performance is expected to be lower. In this work, we present and evaluate a fusion framework for improving the biometric authentication performance. Specifically, we employ score-level fusion for two independent biometric systems of iris and periocular region to avoid expensive feature-level fusion. With a detailed evaluation of three different score-level fusion after the score normalization on a dataset of 12579 images, we report the performance gain in authentication using score-level fusion for iris and periocular recognition.

Show publication details
Kazeminia, Salome; Baur, Christoph; Kuijper, Arjan; Ginneken, Bram van; Navab, Nassir; Albarqouni, Shadi; Mukhopadhyay, Anirban

#### GANs for Medical Image Analysis

2020

Artificial Intelligence in Medicine

Generative adversarial networks (GANs) and their extensions have carved open many exciting ways to tackle well known and challenging medical image analysis problems such as medical image de-noising, reconstruction, segmentation, data simulation, detection or classification. Furthermore, their ability to synthesize images at unprecedented levels of realism also gives hope that the chronic scarcity of labeled data in the medical field can be resolved with the help of these generative models. In this review paper, a broad overview of recent literature on GANs for medical applications is given, the shortcomings and opportunities of the proposed methods are thoroughly discussed, and potential future work is elaborated. We review the most relevant papers published until the submission date. For quick access, essential details such as the underlying method, datasets, and performance are tabulated. An interactive visualization that categorizes all papers to keep the review alive is available at http://livingreview.in.tum.de/GANs_for_Medical_Applications/.

Show publication details
Andújar, Carlos [Conference Co-Chair] [et al.]; Fellner, Dieter W. [Proceedings Production Ed.]

#### GCH 2020

2020

Eurographics Workshop on Graphics and Cultural Heritage (GCH) <18, 2020, online>

This year the scientific program consists of a keynote lesson, two full papers sessions, two short paper sessions and one poster session. In the keynote session, Dr. Ivan Sipiran will illustrate the connection between computer graphics and artificial intelligence techniques in the field of cultural heritage. Technical sessions deal with traditional topics in our area, as 3D Geometry and Modelling, VR/AR Applications, Image Processing, 3D Data Management and Visualization or Applications for Museums. Finally, we have also invited a set of experts to deal with two specific topics of interest in the current situation: how the pandemic has affected research in our discipline, and how can our discipline help in recovering cultural tourism after the crisis passes.

• 978-3-03868-110-6
Show publication details
Stockhause, Simon; Ritz, Harald [Referent]; Bormann, Pascal [Korreferent]

#### Generierung und Ordnung von Events in verteilten Systemen mit asynchroner Kommunikation

2020

Giessen, Technische Hochschule Mittelhessen, Bachelor Thesis, 2020

Der Trend der serviceorientierten Architekturen schafft das Bedürfnis, die Komplexität von verteilten Systemen fassen zu können. Viele bestehende Werkzeuge nutzen Logs und Metriken, um Schlussfolgerungen aus der Anwendung ziehen zu können. Allerdings bieten diese nur eingeschränkt die Möglichkeit, kausal zusammenhänge Events zu erfassen. In dieser Arbeit werden Konzepte zur Darstellung von Events und deren Ordnung in verteilten Systemen präsentiert. Diese werden in praxisnahen Anwendungen eingesetzt. Es wird gezeigt, inwiefern die erarbeiteten Konzepte die spezifizierten Anforderungen und Ziele erfüllen. Um die Eventgenerierung und ihre anschließende Ordnung zu gewährleisten, wird ein Datenmodell beschrieben. Es werden zwei Prototypen zur Kontextpropagierung vorgestellt. Zudem werden Visualisierungsansätze präsentiert, die die erhobenen Daten in ansprechender Form darstellen können. Die implementierte Kontextpropagierung bieten Erfahrungswerte, die für zukünftige Arbeit genutzt werden kann. Die Visualisierungsformen der Frame Galerie und des dreidimensionalen Flamengraphs bieten neue Perspektiven zur Darstellung von Tracingdaten.

Show publication details
Wälde, Simone; Kuijper, Arjan [1. Review]; Ritz, Martin [2. Review]

#### Geometry Classification through Feature Extraction and Pattern Recognition in 3D Space

2020

In dieser Masterarbeit wird der Versuch unternommen, ähnliche Wappendarstellungen auf 3D-Modellen von Scherben abzugleichen. Teil eines initialen Workflows ist die Reliefextraktion, für die ein Ansatz von Zatzarinni et al.[30] verwendet wird. Um Informationen der Objektoberfläche zu extrahieren, wird eine Local Binary Pattern Variante von Thompson et al.[24] implementiert. Die resultierenden Merkmalsdeskriptoren werden dann unter Verwendung einer Abstandsmetrik verglichen. Am Ende führt der vorgeschlagene Ansatz nicht zu guten Ergebnissen, aber die aufgetretenen Herausforderungen sind dokumentiert und zukünftige Lösungen werden diskutiert.

Show publication details

#### GeoRocket: A scalable and cloud-based data store for big geospatial files

2020

SoftwareX

We present GeoRocket, a software for the management of very large geospatial datasets in the cloud. GeoRocket employs a novel way to handle arbitrarily large datasets by splitting them into chunks that are processed individually. The software has a modern reactive architecture and makes use of existing services including Elasticsearch and storage back ends such as MongoDB or Amazon S3. GeoRocket is schema-agnostic and supports a wide range of heterogeneous geospatial file formats. It is also format-preserving and does not alter imported data in any way. The main benefits of GeoRocket are its performance, scalability, and usability, which make it suitable for a number of scientific and commercial use cases dealing with very high data volumes, complex datasets, and high velocity (Big Data). GeoRocket also provides many opportunities for further research in the area of geospatial data management.

Show publication details
Ceneda, Davide; Andrienko, Natalia; Andrienko, Gennady; Gschwandtner, Theresia; Miksch, Silvia; Piccolotto, Nikolaus; Schreck, Tobias; Streit, Marc; Suschnigg, Josef; Tominski, Christian

#### Guide Me in Analysis: A Framework for Guidance Designers

2020

Computer Graphics Forum

Guidance is an emerging topic in the field of visual analytics. Guidance can support users in pursuing their analytical goals more efficiently and help in making the analysis successful. However, it is not clear how guidance approaches should be designed and what specific factors should be considered for effective support. In this paper, we approach this problem from the perspective of guidance designers. We present a framework comprising requirements and a set of specific phases designers should go through when designing guidance for visual analytics. We relate this process with a set of quality criteria we aim to support with our framework, that are necessary for obtaining a suitable and effective guidance solution. To demonstrate the practical usability of our methodology, we apply our framework to the design of guidance in three analysis scenarios and a design walk-through session. Moreover, we list the emerging challenges and report how the framework can be used to design guidance solutions that mitigate these issues.

Show publication details
Zhang, Alex; Chen, Kan; Johan, Henry; Erdt, Marius

#### High Performance Texture Streaming and Rendering of Large Textured 3D Cities

2020

2020 International Conference on Cyberworlds. Proceedings

International Conference on Cyberworlds (CW) <19, 2020, online>

We introduce a novel, high performing, bandwidth-aware texture streaming system for progressive texturing of buildings in large 3D cities, with optional texture pre-processing. We seek to maintain high and consistent texture streaming performance across different city datasets, and to address the high memory binding latency in hardware virtual textures. We adopt the sparse partially-resident image to cache mesh textures at runtime and propose to allocate memory persistently, based on mesh visibility weightings and estimated GPU bandwidth. We also retain high quality rendering by minimizing texture pop-ins when transitioning between texture mipmaps. We evaluate our texture streaming system on large city datasets, including a tile-based dataset with 56K large atlases and a dataset containing 5.7M individual textures. Results indicate fast and robust streaming and rendering performance with minimal pop-in artifacts suitable for real-time rendering of large 3D cities.

Show publication details
Liu, Yisi; Lan, Zirui; Tschoerner, Benedikt; Virdi, Satinder Singh; Cui, Jian; Li, Fan; Sourina, Olga; Zhang, Daniel; Chai, David; Müller-Wittig, Wolfgang K.

#### Human Factors Assessment in VR-based Firefighting Training in Maritime: A Pilot Study

2020

2020 International Conference on Cyberworlds. Proceedings

International Conference on Cyberworlds (CW) <19, 2020, online>

Virtual Reality (VR) has been used for training aircraft pilots, maritime seafarers, operators, etc as it provides an immersive environment with realistic lifelike quality. We developed and implemented a VR-based Liquefied Natural Gas (LNG) firefighting simulation system with head-mounted displays (HMD) and novel human factors evaluation that could train and assess both technical and non-technical skills in the firefighting scenarios. The proposed human factors evaluation is based on a competence model and the non-technical skills such as situation awareness, vigilance, and decision making of seafarers could be assessed. An experiment was carried out with 6 trainees and 2 trainers using the implemented LNG firefighting simulation system. The results show that that the maritime trainees felt the VR scene was realistic to them, evoked similar emotions (such as fear, stress) during the demanding events as in the real world and made them attentive during the experience.

Show publication details
Liu, Yisi; Trapsilawati, Fitri; Lan, Zirui; Sourina, Olga; Johan, Henry; Li, Fan; Chen, Chun-Hsien; Müller-Wittig, Wolfgang K.

#### Human Factors Evaluation of ATC Operational Procedures in Relation to Use of 3D Display

2020

Advances in Human Factors of Transportation

International Conference on Human Factors in Transportation <2019, Washington D.C., USA>

Advances in Intelligent Systems and Computing (AISC)
964

In this paper, Holding Stack Management (HSM), Continuous Climb Operations (CCO), Continuous Descent Operations (CDO), and Trajectory Based Operations (TBO) procedures are assessed in relation to the use of an additional 3D display. Two display seetings are compared, namely 2D+3D and 2D only. Twelve Air Traffic Control Officers (ATCOs) took part in the experiment. Traditional questionnaires such as NASA TLX, TRUST, etc. were given at the end of each 30-minute trial for each display setting. Electroencephalogram (EEG) was recorded during the experiments to continuously monitor the changes of the brain states of the ATCOs. The results of the data analyses show that by using 2D+3D display setting, more positive emotions, but higher stress and workload levels were experienced by ATCOs in TBO, CCO and CDO procedures than in 2D setting. In HSM, reduced stress and significantly lower cognitive workload were experienced by ATCOs when they were using 2D+3D setting.

Show publication details
Kügler, David; Sehring, Jannik Matthias; Stefanov, Andrei; Stenin, Igor; Kristin, Julia; Klenzner, Thomas; Schipper, Jörg; Mukhopadhyay, Anirban

#### i3PosNet: Instrument Pose Estimation from X-ray in Temporal Bone Surgery

2020

International Journal of Computer Assisted Radiology and Surgery

International Conference on Information Processing in Computer-Assisted Interventions (IPCAI) <11, 2020, Munich, Germany>

PURPOSE:Accurate estimation of the position and orientation (pose) of surgical instruments is crucial for delicate minimally invasive temporal bone surgery. Current techniques lack in accuracy and/or line-of-sight constraints (conventional tracking systems) or expose the patient to prohibitive ionizing radiation (intra-operative CT). A possible solution is to capture the instrument with a c-arm at irregular intervals and recover the pose from the image. METHODS:i3PosNet infers the position and orientation of instruments from images using a pose estimation network. Said framework considers localized patches and outputs pseudo-landmarks. The pose is reconstructed from pseudo-landmarks by geometric considerations. RESULTS:We show i3PosNet reaches errors [Formula: see text] mm. It outperforms conventional image registration-based approaches reducing average and maximum errors by at least two thirds. i3PosNet trained on synthetic images generalizes to real X-rays without any further adaptation. CONCLUSION:The translation of deep learning-based methods to surgical applications is difficult, because large representative datasets for training and testing are not available. This work empirically shows sub-millimeter pose estimation trained solely based on synthetic training data.

Show publication details
Kloiber, Simon; Settgast, Volker; Schinko, Christoph; Weinzerl, Martin; Fritz, Johannes; Schreck, Tobias; Preiner, Reinhold

#### Immersive Analysis of User Motion in VR Applications

2020

The Visual Computer

With the rise of virtual reality experiences for applications in entertainment, industry, science and medicine, the evaluation of human motion in immersive environments is becoming more important. By analysing the motion of virtual reality users, design choices and training progress in the virtual environment can be understood and improved. Since the motion is captured in a virtual environment, performing the analysis in the same environment provides a valuable context and guidance for the analysis.We have created a visual analysis system that is designed for immersive visualisation and exploration of human motion data. By combining suitable data mining algorithms with immersive visualisation techniques, we facilitate the reasoning and understanding of the underlying motion. We apply and evaluate this novel approach on a relevant VR application domain to identify and interpret motion patterns in a meaningful way.

Show publication details
Zhou, Wei; Hao, Xingxing; Wang, Kaidi; Zhang, Zhenyang; Yu, Yongxiang; Su, Haonan; Li, Kang; Cao, Xin; Kuijper, Arjan

#### Improved Estimation of Motion Blur Parameters for Restoration from a Single Image

2020

PLOS ONE

This paper presents an improved method to estimate the blur parameters of motion deblurring algorithm for single image restoration based on the point spread function (PSF) in frequency spectrum. We then introduce a modification to the Radon transform in the blur angleestimation scheme with our proposed difference value vs angle curve. Subsequently, theauto-correlation matrix is employed to estimate the blur angle by measuring the distancebetween the conjugated-correlated troughs. Finally, we evaluate the accuracy, robustnessand time efficiency of our proposed method with the existing algorithms on the public benchmarks and the natural real motion blurred images. The experimental results demonstratethat the proposed PSF estimation scheme not only could obtain a higher accuracy for theblur angle and blur length, but also demonstrate stronger robustness and higher time efficiency under different circumstances.

Show publication details
Knauthe, Volker; Ballweg, Kathrin; Wunderlich, Marcel; Landesberger, Tatiana von; Guthe, Stefan

#### Influence of Container Resolutions on the Layout Stability of Squarified and Slice-And-Dice Treemaps

2020

EuroVis 2020. Eurographics / IEEE VGTC Conference on Visualization 2020. Short Papers

Eurographics / IEEE VGTC Conference on Visualization (EuroVis) <22, 2020, online>

In this paper, we analyze the layout stability for the squarify and slice-and-dice treemap layout algorithms when changingthe visualization containers resolution. We also explore how rescaling a finished layout to another resolution compares toa recalculated layout, i.e. fixed layout versus changing layout. For our evaluation, we examine a real world use-case anduse a total of 240000 random data treemap visualizations. Rescaling slice-and-dice or squarify layouts affects the aspectratios. Recalculating slice-and-dice layouts is equivalent to rescaling since the layout is not affected by changing the containerresolution. Recalculating squarify layouts, on the other hand, yields stable aspect ratios but results in potentially huge layoutchanges. Finally, we provide guidelines for using rescaling, recalculation and the choice of algorithm.

Show publication details

#### Information Visualization Interface on Home Router Traffic Data for Laypersons

2020

Proceedings of the Working Conference on Advanced Visual Interfaces AVI 2020

International Conference on Advanced Visual Interfaces (AVI) <2020, online>

With the aim to increase the awareness of the everyday internet user for the own home network traffic, we present two interactive visualization interfaces for visual exploration of home router traffic records. Thereby we differentiate between users with a present intrinsic motivation for the topic and those with absent intrinsic motivation. Therefore, gamification in the first interface is used to maintain motivation of the first type of user, while the storytelling concept based on the hero's journey in the second interface aims at increasing the perceived incentives for the second user group.

Show publication details
Liu, Yisi; Lan, Zirui; Cui, Jian; Sourina, Olga; Müller-Wittig, Wolfgang K.

#### Inter-subject Transfer Learning for EEG-based Mental Fatigue Recognition

2020

Mental fatigue is one of the major factors leading to human errors. To avoid failures caused by mental fatigue, researchers are working on ways to detect/monitor fatigue using different types of signals. Electroencephalography (EEG) signal is one of the most popular methods to recognize mental fatigue since it directly measures the neurophysiological activities in the brain. Current EEG-based fatigue recognition algorithms are usually subject-specific, which means a classifier needs to be trained per subject. However, as fatigue may need a relatively long period to induce, collecting training data from each new user could be time-consuming and troublesome. Calibration-free methods are desired but also challenging since significant variability of physiological signals exists among different subjects. In this paper, we proposed algorithms using inter-subject transfer learning for EEG-based mental fatigue recognition, which did not need a calibration. To explore the influence of the number of EEG channels on the algorithms’ accuracy, we also compared the cases of using one channel only and multiple channels. Random forest was applied to choose the channel that has the most distinguishable features. A public EEG fatigue dataset recorded during driving was used to validate the algorithms. EEG data from 11 subjects were selected from the dataset and leave-one-subject-out cross-validation was employed. The channel from the occipital lobe is selected when only one channel is desired. The proposed transfer learning-based algorithms using Maximum Independence Domain Adaptation (MIDA) achieved an accuracy of 73.01% with all thirty channels, and using Transfer Component Analysis (TCA) achieved 68.00% with the one selected channel.

Show publication details
Chegini, Mohammad; Bernard, Jürgen; Cui, Jian; Chegini, Fatemeh; Sourin, Alexei; Andrews, Keith; Schreck, Tobias

#### Interactive Visual Labelling versus Active Learning: an Experimental Comparison

2020

Frontiers of Information Technology & Electronic Engineering

Methods from supervised machine learning allow the classification of new data automatically and are tremendously helpful for data analysis. The quality of supervised maching learning depends not only on the type of algorithm used, but also on the quality of the labelled dataset used to train the classifier. Labelling instances in a training dataset is often done manually relying on selections and annotations by expert analysts, and is often a tedious and time-consuming process. Active learning algorithms can automatically determine a subset of data instances for which labels would provide useful input to the learning process. Interactive visual labelling techniques are a promising alternative, providing effective visual overviews from which an analyst can simultaneously explore data records and select items to a label. By putting the analyst in the loop, higher accuracy can be achieved in the resulting classifier. While initial results of interactive visual labelling techniques are promising in the sense that user labelling can improve supervised learning, many aspects of these techniques are still largely unexplored. This paper presents a study conducted using the mVis tool to compare three interactive visualisations, similarity map, scatterplot matrix (SPLOM), and parallel coordinates, with each other and with active learning for the purpose of labelling a multivariate dataset. The results show that all three interactive visual labelling techniques surpass active learning algorithms in terms of classifier accuracy, and that users subjectively prefer the similarity map over SPLOM and parallel coordinates for labelling. Users also employ different labelling strategies depending on the visualisation used.

Show publication details
Zielinski, Michael; Urban, Bodo [Erstgutachter]; Lukas, Uwe von [Zweitgutachter]; Nonnemann, Lars [Betreuer]

#### Interaktionsbasierter Datenaustausch zwischen unabhängigen Visual Analytics Werkzeugen

2020

Rostock, Universität, Master Thesis, 2020

Die Arbeit befasst sich mit der Heterogenität von Standards beim Informationsaustausch zwischen Visual Analytics (VA) Werkzeugen. Im Zuge dessen, wurde ein Modell entwickelt, welches Transformationsmöglichkeiten zwischen mehreren VA Werkzeugen anbieten soll. Das entwickelte Modell definiert den Datenaustausch zwischen Werkzeugen in Bezug auf Ursprung, Modifikation und Auswirkung vollständig. Dadurch könnten Transformationen mit möglichst geringem Aufwand umgesetzt werden. Die konzipierte Lösung wurde als Erweiterung im Rahmen eines existierenden Editors zur Konfiguration und Ausführung von analytischen Toolchains namens AnyProc implementiert. Die vorgestellten Methoden werden Anhand eines gegebenen Anwendungsszenarios zur Untersuchung von Aktivitätsdaten mit den Visual Analytics Tools Health@Hand und Microsoft Excel prototypisch demonstriert.

Show publication details
Boutros, Fadi; Damer, Naser; Raja, Kiran; Ramachandra, Raghavendra; Kirchbuchner, Florian; Kuijper, Arjan

#### Iris and Periocular Biometrics for Head Mounted Displays: Segmentation, Recognition, and Synthetic Data Generation

2020

Image and Vision Computing

Augmented and virtual reality deployment is finding increasing use in novel applications. Some of these emerging and foreseen applications allow the users to access sensitive information and functionalities. Head Mounted Displays (HMD) are used to enable such applications and they typically include eye facing cameras to facilitate advanced user interaction. Such integrated cameras capture iris and partial periocular region during the interaction. This work investigates the possibility of using the captured ocular images from integrated cameras from HMD devices for biometric verification, taking into account the expected limited computational power of such devices. Such an approach can allow user to be verified in a manner that does not require any special and explicit user action. In addition to our comprehensive analyses, we present a light weight, yet accurate, segmentation solution for the ocular region captured from HMD devices. Further, we benchmark a number of well-established iris and periocular verification methods along with an in-depth analysis on the impact of iris sample selection and its effect on iris recognition performance for HMD devices. To the end, we also propose and validate an identity-preserving synthetic ocular image generation mechanism that can be used for large scale data generation for training purposes or attack generation purposes. We establish the realistic image quality of generated images with high fidelity and identity preserving capabilities through benchmarking them for iris and periocular verification.

Show publication details
Das, Priyanka; McGrath, Joseph; Fang, Zhaoyuan; Boyd, Aidan; Jang, Ganghee; Mohammadi, Amir; Purnapatra, Sandip; Yambay, David; Marcel, Sébastien; Trokielewicz, Mateusz; Maciejewicz, Piotr; Bowyer, Kevin W.; Czajka, Adam; Schuckers, Stephanie; Tapia, Juan; Gonzalez, Sebastian; Fang, Meiling; Damer, Naser; Boutros, Fadi; Kuijper, Arjan; Sharma, Renu; Chen, Cunjian; Ross, Arun A.

#### Iris Liveness Detection Competition (LivDet-Iris) – The 2020 Edition

2020

IJCB 2020. IEEE/IARP International Joint Conference on Biometrics

IEEE/IARP International Joint Conference on Biometrics (IJCB) <2020, online>

Launched in 2013, LivDet-Iris is an international competition series open to academia and industry with the aim to assess and report advances in iris Presentation Attack Detection (PAD). This paper presents results from the fourth competition of the series: LivDet-Iris 2020. This year's competition introduced several novel elements: (a) incorporated new types of attacks (samples displayed on a screen, cadaver eyes and prosthetic eyes), (b) initiated LivDet-Iris as an on-going effort, with a testing protocol available now to everyone via the Biometrics Evaluation and Testing (BEAT)* open-source platform to facilitate reproducibility and benchmarking of new algorithms continuously, and (c) performance comparison of the submitted entries with three baseline methods (offered by the University of Notre Dame and Michigan State University), and three open-source iris PAD methods available in the public domain. The best performing entry to the competition reported a weighted average APCER of 59.10% and a BPCER of 0.46% over all five attack types. This paper serves as the latest evaluation of iris PAD on a large spectrum of presentation attack instruments.

Show publication details
Siegmund, Dirk; Sacco, Luís Rüger; Kuijper, Arjan

#### Issue Based OCR Error Prediction in Video Streams

2020

SPA 2020. Signal Processing: Algorithms, Architectures, Arrangements, and Applications. Conference Proceedings

Signal Processing Algorithms, Architectures, Arrangements, and Applications Conference (SPA) <24, 2020, online>

This paper increases the reliability of Optical Character Recognition (OCR) systems in natural scene by proposing a novel Image Quality Assessment (IQA) system. We propose to increase reliability based on the principle that OCR accuracy is a function of the quality of the input image. Detected text boxes are analyzed regarding their OCR score and different quality issues, such as blur, light and reflection effects. The novelty of our approach is to model IQA as a classification task, where one class represents high quality elements and each of the other classes represent a specific quality issue. We demonstrate how this methodology allows the training of IQA systems for complex quality metrics, even when no data labeled with the desired metric is available. Furthermore, a single IQA system outputs the quality score as well as the quality issues for a given image. We built on publicly available databases to generate 60k text boxes for each class and obtain 97,1% classification accuracy on a test set of 24k images. We conclude that the learnt quality metric is a valid indicator of common OCR errors by evaluating on the ICDAR 2003 Robust Word Recognition dataset.

Show publication details
Fellner, Dieter W. [Hg.]; Sihn, Wilfried [Advisor]

#### Jahresbericht 2019

2020

Show publication details
Bauer, Markus; Noll, Matthias [1. Gutachten]; Kuijper, Arjan [2. Gutachten]

#### KI basierte automatische Lymphknotendetektion für Ultraschall

2020

Mit der steigenden Geschwindigkeit im Umgang mit großen Datenmengen kam es in den letzten Jahren zu großen Fortschritten in der Stärke von lernenden Systemen. Diese Arbeit zeigt ein mittelgroßes Modell, welches auch mit weniger Daten gute Ergebnisse produziert. Dabei gebe ich Erklärungen für die einzelnen Teile der Architektur und ihrer Auswirkung auf das Lernen, sowie Vergleiche mit aktuellen Modellen. Ich zeige anhand verschiedener Metriken die Lernkraft der Modelle auf 2-dimensionalen Ultraschallbildern. Anschließend zeige ich, wie gut diese Modelle für 3-dimensionale Daten segmentieren. Dabei wird ersichtig, dass die Diversität von Daten wichtig für die Generalisierung der statistischen Modelle ist und wie trügerisch gut die Metriken sein können, wenn nur auf einem limitiereten Datensatz gelernt wird.

Show publication details
Hsu, Wei-Hung; Kuijper, Arjan [1. Review]; Boutros, Fadi [2. Review]

#### Learned Data Augmentation for Model Optimization

2020

This thesis explores the effectiveness of shape variation in the context of occlusion type augmentations by utilizing a shape generation policy, called Shapeshifting, for the construction of different occlusion shapes. In this approach, a reference point is randomly initialized within the image and several polygon vertices are then randomly placed around the reference point for the construction of the occlusion area. Further improvements, by applying the proposed approach multiple times in unstructured and structured ways, in what is referred to as K-Shapeshifting and Structured Shapeshifting, were also explored. This thesis also explores a segmentation-based occlusion approach, called Semantic Occlusion. The proposed approach constructs an occlusion mask using the regions inferred through an unsupervised semantic segmentation approach. The occlusion mask is constructed by selecting an arbitary region from the inferred segmentation. This approach was extended by further evaluating the performance of occluding multiple regions from the semantic segmentation. The proposed approaches were evaluated using the widelyused augmentation policy of random crop alongside with random flip as baseline. On the benchmark dataset CIFAR10, ResNet18 with Shapeshifting achieved an accuracy of 0.9574, an improvement over the baseline accuracy of 0.9528. On SVHN, the multi-component variant of Shapeshifting, K-Shapeshifting, achieved an accuracy of 0.9731, an improvement over the baseline accuracy of 0.9631. On STL10, the same policy, K-Shapeshifting, achieved 0.9729 on STL10 over the baseline accuracy of 0.9704. The proposed approach, Shapeshifting, that uses polygon generation algorithm for the construction of the occlusion mask achieved competitive performances. While it is shown that adding more polygon vertices for the construction of the occlusion polygon contributes to a significant improvment in performance in the proposed setting, the experiment results also provide empirical evidence that the improvement in performance is mainly attributed to the underlying increase in occlusion ratio. As such, it is concluded with empirical evidence that shape variation of the occlusion mask does not provide significant contribution to the model accuracy. The proposed Semantic Occlusion approach, which uses a single region of the inferred segmentation for the construction of the occlusion mask achieved 0.9444 on CIFAR10, 0.9649 on SVHN and 0.9637 on STL10. Improvements were achieved in the extension, K-Semantic Occlusion, with 0.9502 on CIFAR10, 0.9678 on SVHN and 0.967 on STL10. The proposed segmentation-based approach only achieved improvements over the baseline on SVHN and did not outperform the previous approach, Shapeshifting.

Show publication details
Bortolato, Blaz; Ivanovska, Marija; Rot, Peter; Križaj, Janez; Terhörst, Philipp; Damer, Naser; Peer, Peter; Struc, Vitomir

#### Learning Privacy-Enhancing Face Representations through Feature Disentanglement

2020

15th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2020). Proceedings

International Conference on Automatic Face and Gesture Recognition (FG) <15, 2020, Buenos Aires, Argentina>

Convolutional Neural Networks (CNNs) are today the de-facto standard for extracting compact and discriminative face representations (templates) from images in automatic face recognition systems. Due to the characteristics of CNN models, the generated representations typically encode a multitude of information ranging from identity to soft-biometric attributes, such as age, gender or ethnicity. However, since these representations were computed for the purpose of identity recognition only, the soft-biometric information contained in the templates represents a serious privacy risk. To mitigate this problem, we present in this paper a privacy-enhancing approach capable of suppressing potentially sensitive soft-biometric information in face representations without significantly compromising identity information. Specifically, we introduce a Privacy-Enhancing Face-Representation learning Network (PFRNet) that disentangles identity from attribute information in face representations and consequently allows to efficiently suppress soft-biometrics in face templates. We demonstrate the feasibility of PFRNet on the problem of gender suppression and show through rigorous experiments on the CelebA, Labeled Faces in the Wild (LFW) and Adience datasets that the proposed disentanglement-based approach is highly effective and improves significantly on the existing state-of-the-art.

Show publication details
Krumb, Henry John; Hofmann, Sofie; Kügler, David; Ghazy, Ahmed; Dorweiler, Bernhard; Bredemann, Judith; Schmitt, Robert; Sakas, Georgios; Mukhopadhyay, Anirban

#### Leveraging spatial uncertainty for online error compensation in EMT

2020

International Journal of Computer Assisted Radiology and Surgery

PURPOSE: Electromagnetic tracking (EMT) can potentially complement fluoroscopic navigation, reducing radiation exposure in a hybrid setting. Due to the susceptibility to external distortions, systematic error in EMT needs to be compensated algorithmically. Compensation algorithms for EMT in guidewire procedures are only practical in an online setting. METHODS: We collect positional data and train a symmetric artificial neural network (ANN) architecture for compensating navigation error. The results are evaluated in both online and offline scenarios and are compared to polynomial fits. We assess spatial uncertainty of the compensation proposed by the ANN. Simulations based on real data show how this uncertainty measure can be utilized to improve accuracy and limit radiation exposure in hybrid navigation. RESULTS: ANNs compensate unseen distortions by more than 70%, outperforming polynomial regression. Working on known distortions, ANNs outperform polynomials as well. We empirically demonstrate a linear relationship between tracking accuracy and model uncertainty. The effectiveness of hybrid tracking is shown in a simulation experiment. CONCLUSION: ANNs are suitable for EMT error compensation and can generalize across unseen distortions. Model uncertainty needs to be assessed when spatial error compensation algorithms are developed, so that training data collection can be optimized. Finally, we find that error compensation in EMT reduces the need for X-ray images in hybrid navigation.

Show publication details

#### Live Soll-Ist-Abgleich mit Augmented Reality

2020

wt Werkstattstechnik online

Automatisierte, Computer-Vision-gestützte Systeme objektivieren die Qualitätskontrolle in der Automobilproduktion. Augmented-Reality-Technologien des Fraunhofer-Instituts für Graphische Datenverarbeitung IGD vereinen reale und digitale Produktionsumgebung und lassen so auf den ersten Blick Abweichungen zwischen Ist und Soll erkennen.

Show publication details
Rohr, Maurice; Kuijper, Arjan [1. Review]; Adamy, Jürgen [2. Review]; Noll, Matthias [3. Review]

#### Lymph Node Navigation in the Head and Neck Area Using 3D Ultrasound Images

2020

The aftercare of head and neck carcinoma patients is laborious because the physician needs to locate each lymph node in 2D ultrasound for analysis. 3D ultrasound context information can be utilized to navigate in the head neck area automatically. We review methods for using context information in medical imaging in the literature, discuss the characteristics of ultrasound and the head-neck area and present four distinct approaches for locating small structures in ultrasound images. The focus is placed on two approaches: locating small structures by using bright and dark regions in the images and using PatchMatch, a correspondence algorithm for image patches, for constructing a nearest neighbour field that is used to estimated an image transform between follow-up images. Experimentally we find that the PatchMatch approach yields promising results and using bright and dark regions does not work robustly on all image configurations.

Show publication details

#### Measurement Based AR for Geometric Validation Within Automotive Engineering and Construction Processes

2020

Virtual, Augmented and Mixed Reality. Industrial and Everyday Life Applications

International Conference Virtual Augmented and Mixed Reality (VAMR) <12, 2020, Copenhagen, Denmark>

Lecture Notes in Computer Science (LNCS)
12191

We look at the final stages of the automobile design process, during which the geometric validation process for a design, in particular for the vehicle front end, is examined. A concept is presented showing how this process can be improved using augmented reality. Since the application poses high accuracy requirements the augmented reality also needs to be highly accurate and of measurable quality. We present a Measurement Based AR approach to overlaying 3D information onto images, which extends the existing process and is particularly suited to the application in question. We also discuss how the accuracy of this new approach can be validated using computer vision methods employed under the appropriate conditions. The results of an initial study are presented, where the overlay accuracy is expressed in image pixels as well as millimeters followed by a discussion on how this validation can be improved to meet the requirements posed by the application.

Show publication details
Cheng, Wentao; Lin, Weisi [Supervisor]; Goesele, Michael [Co-supervisor]; Kuijper, Arjan [Co-supervisor]

#### Methods for Large-Scale Image-Based Localization Using Structure-from-Motion Point Clouds

2020

Singapore, Nanyang Technological Univ., Diss., 2020

Image-based localization, i.e. estimating the camera pose of an image, is a fundamental task in many 3D computer vision applications. For instance, visual navigation for self-driving cars and robots, mixed reality and augmented reality all rely on this essential task. Due to easy availability and richness of information, large-scale 3D point clouds reconstructed from images via Structure-from-Motion (SfM) techniques have received broad attention in the area of image-based localization. Therein, the 6-DOF camera pose can be computed from 2D-3D matches established between a query image and an SfM point cloud. During the last decade, to handle large-scale SfM point clouds, many image-based localization methods have been proposed, in which significant improvements have been achieved in many aspects. Yet, it remains difficult but meaningful to build a system, which (i) robustly handles the prohibitively expensive memory consumption brought by large-scale SfM point clouds, (ii) well resolves the match disambiguation problem, i.e. distinguishing correct matches from wrong ones, which is even more challenging in urban scenes or under binary feature representation and (iii) achieves high localization accuracy so that the system can be safely applied in low false tolerance applications such as autonomous driving. In this thesis, we propose three methods that tackle these challenging problems to make a further step towards such an ultimate system. First of all, we aim to solve the memory consumption problem by means of simplifying a large-scale SfM point cloud to a small but highly informative subset. To this end, we propose a data-driven SfM point cloud simplification framework, which allows us to automatically predict a suitable parameter setting by exploiting the intrinsic visibility information. In addition, we introduce a weight function into the standard greedy SfM point cloud simplification algorithm, so that more essential 3D points can be well preserved. We experimentally evaluate the proposed framework on real-world large-scale datasets, and show the robustness of parameter prediction. The simplified SfM point clouds generated by our framework achieve better localization performance, which demonstrates the benefit of our framework for image-based localization in devices with limited memory resources. Second, we investigate the match disambiguation problem in large-scale SfM point clouds depicting urban environments. Due to feature space density and massive repetitive structures, this problem becomes challenging if solely depending on feature appearances. As such, we present a two-stage outlier filtering framework that leverages both the visibility and geometry information of SfM point clouds. We first propose a visibility-based outlier filter, which is based on the bipartite visibility graph, to filter outliers on a coarse level. By deriving a data-driven geometrical constraint for urban environments, we present a geometry-based outlier filter to generate a set of fine-grained matches. The proposed framework only relies on the intrinsic information of an SfM point cloud. It is thus widely applicable to be embedded into existing image-based localization approaches. Our framework is able to handle matches of very large outlier ratio and outperforms state-of-the-art image-based localization methods in terms of effectiveness. Last, we aim to build a general-purpose image-based localization system that simultaneously solves the memory consumption, match disambiguation and localization accuracy problems. We adopt a binary feature representation and propose a corresponding match disambiguation method by adequately utilizing the intrinsic feature, visibility and geometry information. The core idea is that we divide the challenging disambiguation task into two different tasks before deriving an auxiliary camera pose for final disambiguation. One task focuses on preserving potentially correct matches, while another focuses on obtaining high quality matches to facilitate subsequent more powerful disambiguation. Moreover, our system improves the localization accuracy by introducing a quality-aware spatial reconfiguration method and a principal focal length enhanced pose estimation method. Our experimental study confirms that the proposed system achieves superior localization accuracy using significantly smaller memory resources comparing with state-of-the-art methods.

Show publication details

#### Micro Stripes Analyses for Iris Presentation Attack Detection

2020

IJCB 2020. IEEE/IARP International Joint Conference on Biometrics

IEEE/IARP International Joint Conference on Biometrics (IJCB) <2020, online>

Iris recognition systems are vulnerable to the presentation attacks, such as textured contact lenses or printed images. In this paper, we propose a lightweight framework to detect iris presentation attacks by extracting multiple micro-stripes of expanded normalized iris textures. In this procedure, a standard iris segmentation is modified. For our Presentation Attack Detection (PAD) network to better model the classification problem, the segmented area is processed to provide lower dimensional input segments and a higher number of learning samples. Our proposed Micro Stripes Analyses (MSA) solution samples the segmented areas as individual stripes. Then, the majority vote makes the final classification decision of those micro-stripes. Experiments are demonstrated on five databases, where two databases (IIITD-WVU and Notre Dame) are from the LivDet-2017 Iris competition. An in-depth experimental evaluation of this framework reveals a superior performance compared with state-of-the-art (SoTA) algorithms. Moreover, our solution minimizes the confusion between textured (attack) and soft (bona fide) contact lens presentations.

Show publication details
Syeda-Mahmood, Tanveer [Ed.]; Oyarzun Laura, Cristina [Ed.]; Wesarg, Stefan [Ed.]; Erdt, Marius [Ed.]

#### Multimodal Learning for Clinical Decision Support and Clinical Image-Based Procedures. Proceedings

2020

International Workshop on Clinical Image-based Procedures (CLIP) <9, 2020, Online>

Image Processing, Computer Vision, Pattern Recognition, and Graphics (LNIP), Lecture Notes in Computer Science (LNCS), 12445
12445

On October 4, 2020, the 9th International Workshop on Clinical Image-based Procedures: From Planning to Intervention (CLIP 2020), was held in conjunction with the 23rd International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI 2020). Due to the COVID-19 pandemic, the workshop was held as an online-only meeting to contribute to slowing down the spread of the virus. Despite the challenges involved, we have continued to build on what we have successfully practiced over the past eight years: providing a platform for the dissemination of clinically tested, state-of-the-art methods for image-based planning, monitoring, and evaluation of medical procedures. A major focus of CLIP 2020 was on the creation of holistic patient models to better understand the need of the individual patient and thus provide better diagnoses and therapies. In this context, it is becoming increasingly important to not only base decisions on image data alone, but to combine these with non-image data, such as ‘omics’ data, electronic medical records, electroencephalograms, and others. This approach offers exciting opportunities to research. CLIP provides a platform to present and discuss these developments and work, centered on specific clinical applications already in use and evaluated by clinical users. In 2020, CLIP accepted nine original manuscripts from all over the world for oral presentation at the online event. Each of the manuscripts underwent a single-blind peer review by two members of the Program Committee, all of them prestigious experts in the field of medical image analysis and clinical translations of technology. We would like to thank our Program Committee for its invaluable contributions and continuous support of CLIP over the years. It is not always easy to find the time to support our workshop given full schedules and challenges due to the ongoing pandemic, and we are very grateful to all our members because CLIP 2020 would not have been possible without them. We would also like to thank all the authors for their high-quality contributions this year as well as their efforts to make CLIP 2020 a success. Finally, we would like to thank all MICCAI 2020 organizers for supporting the organization of CLIP 2020.

• 978-3-030-60945-0
Show publication details
Rasheed, Muhammad Irtaza Bin; Kuijper, Arjan [1. Prüfer]; Burkhardt, Dirk [2. Prüfer]

#### Name Disambiguation

2020

Name ambiguity is a challenge and critical problem in many applications, such as scienti_c literature management, trend analysis etc. The main reason of this is due to di_erent name abbreviations, identical names, name misspellings in publications and bibliographies. An author may have multiple names and multiple authors may have the same name. So when we look for a particular name, many documents containing that person's name may be returned or missed because of the author's di_erent style of writing their name. This can produce name ambiguity which a_ects the performance of document retrieval, web search, database integration, and may result improper classi_cation of authors. Previously, many clustering based algorithm have been proposed, but the problem still remains largely unsolved for both research and industry communities, specially with the fast growth of information available. The aim of this thesis is the implementation of a universal name disambiguation approach that considers almost any existing property to identify authors. After an author of a paper is identi_ed, the normalized name writing form on the paper is used to re_ne the author model and even give an overview about the di_erent writing forms of the author's name. This can be achieved by _rst examine the research on Human-Computer Interaction speci_cally with focus on (Visual) Trend Analysis. Furthermore, a research on di_erent name disambiguation techniques. After that, building a concept and implementing a generalized method to identify author name and a_liation disambiguation while evaluating di_erent properties.

Show publication details
Frank, Sebastian; Kuijper, Arjan

#### NannyCaps - Monitoring Child Conditions and Activity in Automotive Applications Using Capacitive Proximity Sensing

2020

HCI International 2020 – Late Breaking Papers
Lecture Notes in Computer Science (LNCS)
12429

Children have to be transported safely. Securing children in a child seat is indicated. Due to structure and restraint systems, children are secured in case of an accident. Children require our attention to keep them healthy and at good mood. Nonetheless, attention must be payed to driving, too. This discrepancy leads to unattended children. Furthermore, responsible must decide to leave their children alone in the vehicle in case of emergencies. This may lead to heat strokes. Aside of limiting effects of an accident, it would be helpful to assist ambulance after an emergency and to detect injuries even without accident. Besides safety features, preserving good mood of children is an exquisite comfort feature. This can be achieved without privacy issues as they would occur using camera-based systems. The proposed solution, NannyCaps, is designed to contribute to safety and comfort. An invisible array of capacitive proximity sensors enables head position recognition, sleep state recognition, heart rate recognition and occupancy recognition. The system is included into the child seat, only. In this paper, we present the design and implementation of Nanny- Caps. By conducting ten test runs under real world conditions, more than 600km of data is collected. Using this data, NannyCaps is trained and evaluated. Reasonable results are shown in evaluation. Thus, following the development of NannyCaps will likely improve the situation for children in transportation systems.

Show publication details

#### NetCapVis: Web-based Progressive Visual Analytics for Network Packet Captures

2020

VizSec 2019

IEEE Symposium on Visualization for Cyber Security (VizSec) <16, 2019>

Network traffic log data is a key data source for forensic analysis of cybersecurity incidents. Packet Captures (PCAPs) are the raw information directly gathered from the network device. As the bandwidth and connections to other hosts rise, this data becomes very large quickly. Malware analysts and administrators are using this data frequently for their analysis. However, the currently most used tool Wireshark is displaying the data as a table, making it difficult to get an overview and focus on the significant parts. Also, the process of loading large files into Wireshark takes time and has to be repeated each time the file is closed. We believe that this problem poses an optimal setting for a client-server infrastructure with a progressive visual analytics approach. The processing can be outsourced to the server while the client is progressively updated. In this paper we present NetCapVis, an web-based progressive visual analytics system where the user can upload PCAP files, set initial filters to reduce the data before uploading and then instantly interact with the data while the rest is progressively loaded into the visualizations.

Show publication details

#### Neue Normen für biometrische Datenaustauschformate

2020

Datenschutz & Datensicherheit

Dieser Artikel gibt einen Überblick über die neuen, erweiterbaren biometrischen Datenaustauschformate in der Normenreihe ISO/IEC 39794. Diese könnten in ein paar Jahren, nach der erforderlichen Vorbereitungszeit, für biometrische Referenzdaten in langlebigen maschinenlesbaren Reisedokumenten eingesetzt werden.

Show publication details

#### OLBVH: Octree Linear Bounding Volume Hierarchy for Volumetric Meshes

2020

The Visual Computer

We present a novel bounding volume hierarchy for GPU-accelerated direct volume rendering (DVR) as well as volumetric mesh slicing and inside-outside intersection testing. Our novel octree-based data structure is laid out linearly in memory using space filling Morton curves. As our new data structure results in tightly fitting bounding volumes, boundary markers can be associated with nodes in the hierarchy. These markers can be used to speed up all three use cases that we examine. In addition, our data structure is memory-efficient, reducing memory consumption by up to 75%. Tree depth and memory consumption can be controlled using a parameterized heuristic during construction. This allows for significantly shorter construction times compared to the state of the art. For GPU-accelerated DVR, we achieve performance gain of 8.4×–13×. For 3D printing, we present an efficient conservative slicing method that results in a 3×–25× speedup when using our data structure. Furthermore, we improve volumetric mesh intersection testing speed by 5×–52×.

Show publication details
Boutros, Fadi; Damer, Naser; Raja, Kiran; Ramachandra, Raghavendra; Kirchbuchner, Florian; Kuijper, Arjan

#### On Benchmarking Iris Recognition within a Head-mounted Display for AR/VR Applications

2020

IJCB 2020. IEEE/IARP International Joint Conference on Biometrics

IEEE/IARP International Joint Conference on Biometrics (IJCB) <2020, online>

Augmented and virtual reality is being deployed in different fields of applications. Such applications might involve accessing or processing critical and sensitive information, which requires strict and continuous access control. Given that Head-Mounted Displays (HMD) developed for such applications commonly contains internal cameras for gaze tracking purposes, we evaluate the suitability of such setup for verifying the users through iris recognition. In this work, we first evaluate a set of iris recognition algorithms suitable for HMD devices by investigating three well-established handcrafted feature extraction approaches, and to complement it, we also present the analysis using four deep learning models. While taking into consideration the minimalistic hardware requirements of stand-alone HMD, we employ and adapt a recently developed miniature segmentation model (EyeMMS) for segmenting the iris. Further, to account for non-ideal and non-collaborative capture of iris, we define a new iris quality metric that we termed as Iris Mask Ratio (IMR) to quantify the iris recognition performance. Motivated by the performance of iris recognition, we also propose the continuous authentication of users in a non-collaborative capture setting in HMD. Through the experiments on a publicly available OpenEDS dataset, we show that performance with EER = 5% can be achieved using deep learning methods in a general setting, along with high accuracy for continuous user authentication.

Show publication details
Henniger, Olaf; Fu, Biying; Chen, Cong

#### On the Assessment of Face Image Quality Based on Handcrafted Features

2020

BIOSIG 2020

Conference on Biometrics and Electronic Signatures (BIOSIG) <19, 2020, Online>

GI-Edition - Lecture Notes in Informatics (LNI)

This paper studies the assessment of the quality of face images, predicting the utility of face images for automated recognition. The utility of frontal face images from a publicly available dataset was assessed by comparing them with each other using commercial off-the-shelf face recognition systems. Multiple face image features delineating face symmetry and characteristics of the capture process were analysed to find features predictive of utility. The selected features were used to build system-specific and generic random forest classifiers.

Show publication details

#### Open Problems in 3D Model and Data Management

2020

Proceedings of the 15th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications

International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP) <15, 2020, Valetta, Malta>

In interdisciplinary, cooperative projects that involve different representations of 3D models (such as CAD data and simulation data), a version problem can occur: different representations and parts have to be merged to form a holistic view of all relevant aspects. The individual partial models may be exported by and modified in different software environments. These modifications are a recurring activity and may be carried out again and again during the progress of the project. This position paper investigates the version problem; furthermore, this contribution is intended to stimulate discussion on how the problem can be solved.

Show publication details
Tamellini, Lorenzo; Chiumenti, Michele; Altenhofen, Christian; Attene, Marco; Barrowclough, Oliver; Livesu, Marco; Marini, Federico; Martinelli, Massimiliano; Skytt, Vibeke

#### Parametric Shape Optimization for Combined Additive–Subtractive Manufacturing

2020

JOM: The Journal of The Minerals, Metals & Materials Society

In industrial practice, additive manufacturing (AM) processes are often followed by post-processing operations such as heat treatment, subtractive machining, milling, etc., to achieve the desired surface quality and dimensional accuracy. Hence, a given part must be 3D-printed with extra material to enable this finishing phase. This combined additive/subtractive technique can be optimized to reduce manufacturing costs by saving printing time and reducing material and energy usage. In this work, a numerical methodology based on parametric shape optimization is proposed for optimizing the thickness of the extra material, allowing for minimal machining operations while ensuring the finishing requirements. Moreover, the proposed approach is complemented by a novel algorithm for generating inner structures to reduce the part distortion and its weight. The computational effort induced by classical constrained optimization methods is alleviated by replacing both the objective and constraint functions by their sparse grid surrogates. Numerical results showcase the effectiveness of the proposed approach.

Show publication details

#### PAVED: Pareto Front Visualization for Engineering Design

2020

EuroVis 2020. Eurographics / IEEE VGTC Conference on Visualization 2020

Eurographics / IEEE VGTC Conference on Visualization (EuroVis) <22, 2020, online>

Design problems in engineering typically involve a large solution space and several potentially conflicting criteria. Selecting a compromise solution is often supported by optimization algorithms that compute hundreds of Pareto-optimal solutions, thus informing a decision by the engineer. However, the complexity of evaluating and comparing alternatives increases with the number of criteria that need to be considered at the same time. We present a design study on Pareto front visualization to support engineers in applying their expertise and subjective preferences for selection of the most-preferred solution. We provide a characterization of data and tasks from the parametric design of electric motors. The requirements identified were the basis for our development of PAVED, an interactive parallel coordinates visualization for exploration of multi-criteria alternatives. We reflect on our user-centered design process that included iterative refinement with real data in close collaboration with a domain expert as well as a summative evaluation in the field. The results suggest a high usability of our visualization as part of a real-world engineering design workflow. Our lessons learned can serve as guidance to future visualization developers targeting multi-criteria optimization problems in engineering design or alternative domains

Show publication details
Terhörst, Philipp; Riehl, Kevin; Damer, Naser; Rot, Peter; Bortolato, Blaz; Kirchbuchner, Florian; Struc, Vitomir; Kuijper, Arjan

#### PE-MIU: A Training-Free Privacy-Enhancing Face Recognition Approach Based on Minimum Information Units

2020

IEEE Access

Research on soft-biometrics showed that privacy-sensitive information can be deduced from biometric data. Utilizing biometric templates only, information about a persons gender, age, ethnicity, sexual orientation, and health state can be deduced. For many applications, these templates are expected to be used for recognition purposes only. Thus, extracting this information raises major privacy issues. Previous work proposed two kinds of learning-based solutions for this problem. The first ones provide strong privacy-enhancements, but limited to pre-defined attributes. The second ones achieve more comprehensive but weaker privacy-improvements. In this work, we propose a Privacy-Enhancing face recognition approach based on Minimum Information Units (PE-MIU). PE-MIU, as we demonstrate in this work, is a privacy-enhancement approach for face recognition templates that achieves strong privacy-improvements and is not limited to pre-defined attributes. We exploit the structural differences between face recognition and facial attribute estimation by creating templates in a mixed representation of minimal information units. These representations contain pattern of privacy-sensitive attributes in a highly randomized form. Therefore, the estimation of these attributes becomes hard for function creep attacks. During verification, these units of a probe template are assigned to the units of a reference template by solving an optimal best-matching problem. This allows our approach to maintain a high recognition ability. The experiments are conducted on three publicly available datasets and with five state-of-the-art approaches. Moreover, we conduct the experiments simulating an attacker that knows and adapts to the systems privacy mechanism. The experiments demonstrate that PE-MIU is able to suppress privacy-sensitive information to a significantly higher degree than previous work in all investigated scenarios. At the same time, our solution is able to achieve a verification performance close to that of the unmodified recognition system. Unlike previous works, our approach offers a strong and comprehensive privacy-enhancement without the need of training.

Show publication details
Großmann, Tom; Kuijper, Arjan [Betreuer]

#### Perceptual Hashing

2020

Das Prinzip des Perceptual Hashings dient schon seit geraumer Zeit als Grundlage für das Erkennen ähnlicher Bilder. Perceptual Hash Functions generieren einen Bildfingerabdruck aus einem Eingabebild. Sie orientieren sich an der menschlichen Wahrnehmung, sodass für Bilder, die als ähnlich erkannt werden, auch ähnliche Hashwerte erzeugt werden. Auch wenn durchaus viele verschiedene Perceptual Hash Functions existieren, so wurden diese bisher vorrangig anhand detailreicher Bilder getestet. Das Ziel dieser Arbeit ist es nun, zu untersuchen, inwiefern Perceptual Hash Functions dafür geeignet sind, auch Social Media Icons von anderen Bildern zu unterscheiden. Dazu werden der Average Hash, Block Hash, Difference Hash und DCT Hash implementiert und dahingehend untersucht. Die Untersuchungen führen jedoch zu keinem eindeutigen Ergebnis, das es erlauben würde, allgemeine Rückschlüsse auf Perceptual Hash Funtions zu ziehen. Die Ergebnisse zeigen, dass der Block Hash, Difference Hash und DCT Hash nicht für die Unterscheidung zwischen Social Media Icons und anderen Bildern geeignet sind. Der Average Hash dagegen schafft es bis zu 86 Prozent der Social Media Icons in einem Datensatz, bei einer False Acceptance Rate von zwei Prozent, zu identifizieren.

Show publication details

#### Performing Realistic Workout Activity Recognition on Consumer Smartphones

2020

Technologies

Smartphones have become an essential part of our lives. Especially its computing power and its current specifications make a modern smartphone a powerful device for human activity recognition tasks. Equipped with various integrated sensors, a modern smartphone can be leveraged for lots of smart applications. We already investigated the possibility of using an unmodified commercial smartphone to recognize eight strength-based exercises. App-based workouts have become popular in the last few years. The advantage of using a mobile device is that you can practice anywhere at anytime. In our previous work, we proved the possibility of turning a commercial smartphone into an active sonar device to leverage the echo reflected from exercising movement close to the device. By conducting a test study with 14 participants, we showed the first results for cross person evaluation and the generalization ability of our inference models on disjoint participants. In this work, we extended another model to further improve the model generalizability and provided a thorough comparison of our proposed system to other existing state-of-the-art approaches. Finally, a concept of counting the repetitions is also provided in this study as a parallel task to classification.

Show publication details
Boutros, Fadi; Damer, Naser; Raja, Kiran; Ramachandra, Raghavendra; Kirchbuchner, Florian; Kuijper, Arjan

#### Periocular Biometrics in Head-Mounted Displays: A Sample Selection Approach for Better Recognition

2020

IWBF 2020. Proceedings

International Workshop on Biometrics and Forensics (IWBF) <8, 2020, online>

Virtual and augmented reality technologies are increasingly used in a wide range of applications. Such technologies employ a Head Mounted Display (HMD) that typicallyincludes an eye-facing camera and is used for eye tracking.As some of these applications require accessing or transmittinghighly sensitive private information, a trusted verification ofthe operator’s identity is needed. We investigate the use ofHMD-setup to perform verification of operator using periocularregion captured from inbuilt camera. However, the uncontrollednature of the periocular capture within the HMD results inimages with a high variation in relative eye location and eyeopening due to varied interactions. Therefore, we propose a newnormalization scheme to align the ocular images and then, a newreference sample selection protocol to achieve higher verificationaccuracy. The applicability of our proposed scheme is exemplifiedusing two handcrafted feature extraction methods and two deeplearning strategies.We conclude by stating the feasibility of sucha verification approach despite the uncontrolled nature of thecaptured ocular images, especially when proper alignment andsample selection strategy is employed.

• 978-1-7281-6232-4
Show publication details
Nottebaum, Moritz; Kuijper, Arjan [1. Review]; Rus, Silvia [2. Review]

#### Person Re-identification in a Car Seat

2020

In this thesis, I enhanced a car seat with 16 capacity sensors, which collect data from the person sitting on it, which is then used to train a machine learning algorithm to re-identify the person from a group of other already trained persons. In practice, the car seat recognizes the person when he/she sits on the car seat and greets the person with their own name, enabling various customisations in the car unique to the user, like seat configurations, to be applied. Many researchers have done similar things with car seats or seats in general, though focusing on other topics like posture classification. Other interesting use cases of capacitive sensor enhanced seats involved measuring the emotions or focusing on general activity recognition. One major challenge in capacitive sensor research is the inconstancy of the received data, as they are not only affected by objects or persons near to it, but also by changing effects like humidity and temperature. My goal was to make the re-identification robust and use a learning algorithm which can quickly learn the patterns of new persons and is able to achieve satisfiable results even after getting only few training instances to learn from. Another important property was to have a learning algorithm which can operate independent and fast to be even applicable in cars. Both points were achieved by using a shallow convolutional neural network which learns an embedding and is trained with triplet loss, resulting in a computationally cheap inference. In Evaluation, results showed that neural networks are definitely not always the best choice, even though the computation time difference is insignificant. Without enough training data, they often lack in generalisation over the training data. Therefore an ensemble-learning approach with majority voting proved to be the best choice for this setup.

Show publication details
Pflanzer, Yannick; Domajnko, Matevz; Ritz, Martin

#### Phenomenological Roughness Extraction for Physically Based Rendering

2020

Many modern rendering systems use physically based shading techniques, one of which bases its surface property description on the "metalness" and "roughness" of a material. In this work a method for the automatic extraction of the surface roughness parameter from images of an object is developed. The extraction employs polarized light to filter out reflections and uses the difference in reflectivity visible in the images compared to an unpolarized lighting environment to deduce a surface roughness value.

Show publication details
Grimm, Niklas; Damer, Naser [1. Gutachten]; Kuijper, Arjan [2. Gutachten]

#### Poseninvariante Handerkennung mit generativer Bildkorrektur

2020

Auf Händen basierende biometrische Verfahren sind in weiten Bevölkerungsschichten akzeptiert und können kontaktlos angewendet werden. Bei diesen kontaktlosen Authentifizierungsverfahren sind variierende Handposen eines der größten Probleme. Diese Arbeit erforscht, ob es möglich ist, aus Bildern mit variierenden Handposen solche zu synthetisieren, die einer normalisierten Handpose entsprechen. Auf der Grundlage dieser normalisierten Handpose wäre dann ein besserer Vergleich mit dem Referenzbild möglich. Das Fehlen eines großen Datensatzes und die variierende Skalierung bei kontaktlos akquirierten Handbildern bringt viele Herausforderungen. Von diesen Herausforderungen motiviert beschreibt diese Arbeit mehrere Experimente mit einer begrenzten Menge an Trainingsdaten und variierenden Skalierungen der Eingabebilder, echt wirkende Bilder mit normalisierten Handposen, zu synthetisieren. Die synthetisierten Bilder werden in Verifikationsverfahren mit den Orginalbildern verglichen. Am Ende zeigten die Experimente, dass es nicht möglich ist, mit diesem Versuchsaufbau echt wirkende Handbilder mit normalisierten Posen zu synthetisieren.

Show publication details

#### Post-comparison Mitigation of Demographic Bias in Face Recognition Using Fair Score Normalization

2020

Pattern Recognition Letters

Current face recognition systems achieve high progress on several benchmark tests. Despite this progress, recent works showed that these systems are strongly biased against demographic sub-groups. Consequently, an easily integrable solution is needed to reduce the discriminatory effect of these biased systems. Previous work mainly focused on learning less biased face representations, which comes at the cost of a strongly degraded overall recognition performance. In this work, we propose a novel unsupervised fair score normalization approach that is specifically designed to reduce the effect of bias in face recognition and subsequently lead to a significant overall performance boost. Our hypothesis is built on the notation of individual fairness by designing a normalization approach that leads to treating “similar” individuals “similarly”. Experiments were conducted on three publicly available datasets captured under controlled and in-the-wild circumstances. Results demonstrate that our solution reduces demographic biases, e.g. by up to 82.7% in the case when gender is considered. Moreover, it mitigates the bias more consistently than existing works. In contrast to previous works, our fair normalization approach enhances the overall performance by up to 53.2% at false match rate of 10−3 and up to 82.9% at a false match rate of 10−5. Additionally, it is easily integrable into existing recognition systems and not limited to face biometrics.

Show publication details
Morrissey, John P.; Totoo, Prabhat; Hanley, Kevin J.; Papanicolopulos, Stefanos-Aldo; Ooi, Jin Y.; Gonzalez, Ivan Cores; Raffin, Bruno; Mostajabodaveh, Seyedmorteza; Gierlinger, Thomas

#### Post-processing and Visualization of large-scale DEM Simulation Data with the open-source VELaSSCo Platform

2020

Simulation

Regardless of its origin, in the near future the challenge will not be how to generate data, but rather how to manage big and highly distributed data to make it more easily handled and more accessible by users on their personal devices. VELaSSCo (Visualization for Extremely Large-Scale Scientific Computing) is a platform developed to provide new visual analysis methods for large-scale simulations serving the petabyte era. The platform adopts Big Data tools/architectures to enable in-situ processing for analytics of engineering and scientific data and hardware-accelerated interactive visualization. In large-scale simulations, the domain is partitioned across several thousand nodes, and the data (mesh and results) are stored on those nodes in a distributed manner. The VELaSSCo platform accesses this distributed information, processes the raw data, and returns the results to the users for local visualization by their specific visualization clients and tools. The global goal of VELaSSCo is to provide Big Data tools for the engineering and scientific community, in order to better manipulate simulations with billions of distributed records. The ability to easily handle large amounts of data will also enable larger, higher resolution simulations, which will allow the scientific and engineering communities to garner new knowledge from simulations previously considered too large to handle. This paper shows, by means of selected Discrete Element Method (DEM) simulation use cases, that the VELaSSCo platform facilitates distributed post-processing and visualization of large engineering datasets.

Show publication details
Fauser, Johannes; Fellner, Dieter W. [Referent]; Kuijper, Arjan [Korreferent]; Essert, Caroline [Korreferentin]

#### Preoperative Surgical Planning

2020

Since several decades, minimally-invasive surgery has continuously improved both clinical workflow and outcome. Such procedures minimize patient trauma, decrease hospital stay or reduce risk of infection. Next generation robot-assisted interventions promise to further improve on these advantages while at the same time opening the way to new surgical applications. Temporal Bone Surgery and Endovascular Aortic Repair are two examples for such currently researched approaches, where manual insertion of instruments, subject to a clinician's experience and daily performance, could be replaced by a robotic procedure. In the first, a flexible robot would drill a nonlinear canal through the mastoid, allowing a surgeon access to the temporal bone's apex, a target often unreachable without damaging critical risk structures. For the second example, robotically driven guidewires could significantly reduce the radiation exposure from fluoroscopy, that is exposed to patients and surgeons during navigation through the aorta. These robot-assisted surgeries require preoperative planning consisting of segmentation of risk structures and computation of nonlinear trajectories for the instruments. While surgeons could so far rely on preoperative images and a mental 3D model of the anatomy, these new procedures will make computational assistance inevitable due to the added complexity from image processing and motion planning. The automation of tiresome and manually laborious tasks is therefore crucial for successful clinical implementation. This thesis addresses these issues and presents a preoperative pipeline based on CT images that automates segmentation and trajectory planning. Major contributions include an automatic shape regularized segmentation approach for coherent anatomy extraction as well as an exhaustive trajectory planning step on locally optimized Bézier Splines. It also introduces thorough in silico experiments that perform functional evaluation on real and synthetically enlarged datasets. The benefits of the approach are shown on an in house dataset of otobasis CT scans as well as on two publicly available datasets containing aorta and heart.

Show publication details
González, Camila; Kuijper, Arjan [1. Gutachten]; Mukhopadhyay, Anirban [2. Gutachten]

#### Preventing Catastrophic Forgetting in Deep Learning Classifiers

2020

Deep neural networks suffer from the problem of catastrophic forgetting. When a model is trained sequentially with batches of data coming from different domains, it adapts too strongly to properties present on the last batch. This causes a catastrophic fall in performance for data similar to that in the initial batches of training. Regularization-based methods are a popular way to reduce the degree of forgetting, as they have an array of desirable properties. However, they perform poorly when no information about the data origin is present at inference time. We propose a way to improve the performance of such methods which comprises introducing insularoty noise in unimportant parameters so that the model grows robust against them changing. Additionally, we present a way to bypass the need for sourcing information. We propose using an oracle to decide which of the previously seen domains a new instance belongs to. The oracle’s prediction is then used to select the model state. In this work, we introduce three such oracles. Two of these select the model which is most confident for the instance. The first, the cross-entropy oracle, chooses the model with least cross-entropy between the prediction and the one-hot form of the prediction. The second, the MC dropout oracle, chooses the model with lowest standard deviation between predictions resulting from performing an array of forward passes while applying dropout. Finally, the domain identification oracle extracts information about the data distribution for each task using the training data. At inference time, it assesses which task the instance is likeliest to belong to, and applies the corresponding model. For all of our three different datasets, at least one oracle performs better than all regularization-based methods. Furthermore, we show that the oracles can be combined with a sparsification-based approach that significantly reduces the memory requirements.

Show publication details
Frank, Sebastian; Kuijper, Arjan

#### Privacy by Design: Analysis of Capacitive Proximity Sensing as System of Choice for Driver Vehicle Interfaces

2020

HCI International 2020 – Late Breaking Papers
Lecture Notes in Computer Science (LNCS)
12429

Data collection is beneficial. Therefore, automotive manufacturers start including data collection services. At the same time, manufacturers install cameras for human machine interfaces in vehicles. But those systems may disclose further information than needed for gesture recognition. Thus, they may cause privacy issues. The law (GDPR) enforces privacy by default and design. Research often states that capacitive proximity sensing is better to serve privacy by design than cameras. Furthermore, it is unclear if customers value privacy preserving features. Nonetheless, manufacturers value the customer’s voice. Therefore, several vehicular human machine interface systems, with camera or capacitive proximity sensing, are analyzed. Especially concerning gesture recognition, capacitive proximity sensing systems provide similar features like camera-based systems. The analysis is based on the GDPR privacy definition. Due to the analysis, it is revealed that capacitive proximity sensing systems have less privacy concerns causing features. Subsequently, three hypotheses are formulated to capture the customer’s voice. Due to analysis results, it is questionable if gesture recognition systems, which utilize cameras, are compliant with privacy by design. Especially since well-known systems like capacitive proximity sensing are available. A survey concerning the hypotheses will give further insights in future work.

Show publication details
Frank, Sebastian; Kuijper, Arjan

#### Privacy by Design: Survey on Capacitive Proximity Sensing as System of Choice for Driver Vehicle Interfaces

2020

Proceedings CSCS 2020

ACM Computer Science in Cars Symposium (CSCS) <2020, online>

Physiological properties are recorded everywhere with cameras. They are also used to identify people in public spaces. Vehicle manufacturers also use camera systems in cars. For example, those cameras are used in cars to identify people, measure attention and recognize gestures. Especially the recording of facial images can cause privacy concerns following the GDPR. It is therefore questionable whether the recording of facial car user images corresponds to the paradigm privacy-by-design required by the GDPR. Nonetheless, the car user may not have privacy concerns towards the usage of cameras in vehicles. If customers have privacy concerns, vehicle manufacturers should switch to other systems. One of those systems could be capacitive proximity sensing. But capacitive proximity sensing could cause privacy concerns, too. To assess the privacy concerns of car users, a study is conducted. More than 250 participants are recruited. They are asked to rate their privacy concerns when a camera is used in driver assistance systems. Furthermore, they are asked the same questions concerning capacitive proximity sensing. Additionally, they can choose their preferred system, capacitive proximity sensing or camera. The exploratory study emerged due to three hypotheses of a previous paper. These hypotheses, concerning the user’s perception of privacy towards cameras, are tested for the sample. Using the test results, the hypotheses are refined. Based on the analysis of the sample, people have concerns towards cameras in vehicles and prefer capacitive proximity sensing as system of choice.

Show publication details

#### Privacy Evaluation Protocols for the Evaluation of Soft-Biometric Privacy-Enhancing Technologies

2020

BIOSIG 2020

Conference on Biometrics and Electronic Signatures (BIOSIG) <19, 2020, Online>

GI-Edition - Lecture Notes in Informatics (LNI)
P-306

Biometric data includes privacy-sensitive information, such as soft-biometrics. Soft-biometric privacy enhancing technologies aim at limiting the possibility of deducing such information. Previous works proposed several solutions to this problem using several different evaluation processes, metrics, and attack scenarios. The absence of a standardized evaluation protocol makes a meaningful comparison of these solutions difficult. In this work, we propose privacy evaluation protocols (PEPs) for privacy-enhancing technologies (PETs) dealing with soft-biometric privacy. Our framework evaluates PETs in the most critical scenario of an attacker that knows and adapts to the systems privacy-mechanism. Moreover, our PEPs differentiate between PET of learning-based or training-free nature. To ensure that our protocol meets the highest standards in both cases, it is based on Kerckhoffs‘s principle of cryptography.

Show publication details
Liu, Yisi; Lan, Zirui; Cui, Jian; Krishnan, Gopala; Sourina, Olga; Konovessis, Dimitrios; Ang, Hock Eng; Müller-Wittig, Wolfgang K.

#### Psychophysiological Evaluation of Seafarers to Improve Training in Maritime Virtual Simulator

2020

Over the years, safety in maritime industries has been reinforced by many state-of-the-art technologies. However, the accident rate hasn’t dropped significantly with the advanced technology onboard. The main cause of this phenomenon is human errors which drive researchers to study human factors in the maritime domain. One of the key factors that contribute to human performance is their mental states such as cognitive workload and stress. In this paper, we propose and implement an Electroencephalogram (EEG)-based psychophysiological evaluation system to be used in maritime virtual simulators for monitoring, training and assessing the seafarers. The system includes an EEG processing part, visualization part, and an evaluation part. By using the processing part of the system, different brain states including cognitive workload and stress can be identified from the raw EEG data recorded during maritime exercises in the simulator. By using the visualization part, the identified brain states, raw EEG signals, and videos recorded during the maritime exercises can be synchronized and displayed together. By using the evaluation part of the system, an indicative recommendation on “pass”, “retrain”, or “fail” of the seafarers’ performance can be obtained based on the EEG-based cognitive workload and stress recognition. Detailed analysis of the demanding events in the maritime tasks is provided by the system for each seafarer that could be used to improve their training. A case study is presented using the proposed system. EEG data from 4 pilots were recorded when they were performing maritime tasks in the simulator. The data are processed and evaluated. The results show that one pilot gets a “pass” recommendation, one pilot gets a “retrain” recommendation, and the other two get “fail” results regarding their performance in the simulator.

Show publication details

#### Random Forests for Automatic Paranasal Sinus and Nasal Cavity Detection in CT Images

2020

Sinusitis and other pathologies of the nasal cavity and paranasal sinuses are common diseases. Physicians need segmentations of the structures for diagnosis, operation planning, learning models and more. The current practice is manual or sometimes semi-automatic segmentation of the computed tomography (CT) scans. This time-consuming, tedious task is not practical for clinical routine. An automatic segmentation of the paranasal sinuses and nasal cavity in CT images is therefore desirable, but extremely hard to accomplish because of the high variance and complexity of the structures. An often applied approach for difficult segmentation tasks is to use bounding boxes (BBs) of the structures as regions of interest (ROIs) to limit the search space for the segmentation to a small area in the image. This thesis therefore proposes a method for an automatic detection of the paranasal sinuses and nasal cavity in CT images that predicts axis-aligned bounding boxs (AABBs) of the structures. The BBs can then be used as ROIs in a possible subsequent segmentation. The relative locations of the nasal cavity and paranasal sinuses are strongly regularized by the human anatomy. It therefore makes sense to use this knowledge for the localization of the structures of interest. Thus, a random forest (RF) is proposed, which performs a simultaneous regression of the BBs of all structures of interest. Thereby it implicitly learns the relative locations of the structures directly from the training data. Intensity based features that capture spatial relations are applied. For the detection of the paranasal sinuses the forest is trained on 71 and tested on 29 CT scans. A subset of 30 images of the dataset is furthermore used for the simultaneous detection of the paranasal sinuses, nasal conchae and nasal septum. The dataset contains images with many different resolutions and sizes, often pathologies are present. With a mean Intersection over Union (IoU) of 0.27 for detection of the paranasal sinuses and 0.21 for all structures at once, the results are still improvable. Though, in some cases good detection results are achieved. Furthermore, the results show that the proposed method is able to learn the relative positions of the nasal cavity and paranasal sinuses from the training data. This offers interesting potential for the detection of the nasal cavity and paranasal sinuses.

Show publication details
Jansen, Nils; Kuijper, Arjan [1. Review]; Siegmund, Dirk [2. Review]

#### Rapid Depth from Multi-view Images

2020

Show publication details

#### Rapid Interactive Structural Analysis

2020

2020 NAFEMS DACH Regionalkonferenz

NAFEMS DACH Regionalkonferenz <5, 2020, Online>

Auf dem Weg zu einem hocheffizienten Produktentwicklungsprozess, der aus Design, Simulation, Analyse und Iterationen besteht, liegen noch einige ungenutzte Potenziale. Bei der Analyse des Prozesses wird häufig die Integration verschiedener Software-Werkzeuge entlang der Prozesskette als eine der Schwachstellen identifiziert. Hierbei sind Interoperabilität und die Standardisierung von Austauschformaten insbesondere für die Kombination von Software verschiedener Hersteller von zentraler Bedeutung. Jedoch hat die Dauer der Simulation ebenfalls einen maßgeblichen Einfluss auf die Effizienz, um schlussendlich Produkte schneller auf den Markt zu bringen oder eine höhere Qualität zu erzielen. Wenn die Ergebnisse von Simulationen praktisch direkt nach ihrem Start zur Verfügung stehen würden, so könnten einzelne Iterationsschleifen drastisch verkürzt und dasmit eine Vielzahl von Design-Variationen exploriert werden. Auch die rechnergestützte Formoptimierung, bei der Hunderte von Simulationsrechnungen automatisiert durchgeführt werden, würde von solch kurzen Simulationszeiten stark profitieren. Im Projekt Rapid Interactive Structural Analysis wurde eine schnelle, interaktive Simulationslösung mit direkter Visualisierung auf Basis finiter Elemente entwickelt. Durch Nutzung der zur Verfügung stehen-den, immensen Rechenpower von Graphikkarten (GPUs) können Simulationen, wie beispielsweise strukturmechanische Analysen, signifikant beschleunigt werden. Der Lösungsansatz basiert auf massiv-parallelen Algorithmen und beschleunigt dadurch linear-elastische Struktursimulationen um einen Fak-tor von bis zu 80. Mit dieser schnellen Simulationstechnologie werden neuartige Anwendungen möglich, wie beispielsweise die direkte Identifikation von Korrelationen zwischen geometrischen Änderungen und Spannungsverteilung oder die signifikante Beschleunigung von Form- oder Topologieoptimierungen. Die Genauigkeit und Geschwindigkeit des vorgestellten Ansatzes zu herkömmlichen Simulationen auf Basis der Finite-Elemente-Methode (FEM) wird verglichen.

Show publication details
Fina, Kenten; Kuijper, Arjan [1. Review]; Urban, Philipp [2. Review]; Dennstädt, Marco [3. Review]

#### Real-time Rendering of CSG-operations on High Resolution Data for Preview of 3D-Prints

2020

In this thesis various optimizations for the ray-marching algorithm are introduced to efficiently render CSG-operations on high resolution meshes. By using a 2- pass render method and CSG-node memory method, speed-ups of factor 2 to 3 can be achieved in contrast to standard ray marching. We implement a oct-tree based data structure to compress the high resolution SDF (signed distance function) as well as color data. For raw data at a resolution 1024^3, our compressed data requires on average 1.69% of the raw data. Lastly we compare our performance against the openCSG implementations of the well-known Goldfeather and SCS algorithm.

Show publication details
Klemt, Marcel; Kuijper, Arjan [1. Review]; Boutros, Fadi [2. Review]

#### Reducing Deep Face Recognition Model Size by Knowledge Distillation

2020

Current face recognition models have benefited from the recent advanced development of deep learning techniques achieving very high verification performances. However, most of the recent works pay less attention to the computational efficiency of these models. Hence, deploying such models on low computational powered mobile devices is challenging. Nevertheless, recent studies have also shown an increasing demand for mobile user identity authentication using biometrics modalities i.e. face, fingerprint, iris, etc. As a consequence, large well-performing face recognition models have to become smaller to be deployable on mobile devices. This thesis proposes a solution to enhance the verification performance of small face recognition models via knowledge distillation. Conventional knowledge distillation transfers knowledge from a large teacher network to a small student network by mimicking the classification layer. In addition to that, this thesis adapts the knowledge distillation method to be applicable to the feature level which the used teacher ArcFace tries to optimize. The verification results of this thesis prove that knowledge distillation can enhance the performance of a small face recognition model compared to the same model trained without knowledge distillation. Applying conventional knowledge distillation to a ResNet- 56 model increased the accuracy from 99.267% to 99.3% on LFW and from 93.767% to 93.867% on AgeDB. This accuracy of the ResNet-56 student is only 0.117% below the accuracy of its twelve times larger ResNet-18 teacher on LFW and even higher on AgeDB by 0.067%. Moreover, when matching the objective function of ArcFace with knowledge distillation, the performance of a ResNet-56 model could be further increased to 99.367% on LFW. This implies it exceeded the accuracy of the same face recognition model trained without knowledge distillation by a margin of 0.1%. At the same time, it decreased the FMR on LFW compared to the model trained without knowledge distillation.

Show publication details
Fauser, Johannes; Bohlender, Simon Peter; Stenin, Igor; Kristin, Julia; Klenzner, Thomas; Schipper, Jörg; Mukhopadhyay, Anirban

#### Retrospective in Silico Evaluation of Optimized Preoperative Planning for Temporal Bone Surgery

2020

International Journal of Computer Assisted Radiology and Surgery

Purpose: Robot-assisted surgery at the temporal bone utilizing a flexible drilling unit would allow safer access to clinical targets such as the cochlea or the internal auditory canal by navigating along nonlinear trajectories. One key sub-step for clinical realization of such a procedure is automated preoperative surgical planning that incorporates both segmentation of risk structures and optimized trajectory planning. Methods: We automatically segment risk structures using 3D U-Nets with probabilistic active shape models. For nonlinear trajectory planning, we adapt bidirectional rapidly exploring random trees on Bézier Splines followed by sequential convex optimization. Functional evaluation, assessing segmentation quality based on the subsequent trajectory planning step, shows the suitability of our novel segmentation approach for this two-step preoperative pipeline. Results: Based on 24 data sets of the temporal bone, we perform a functional evaluation of preoperative surgical planning. Our experiments show that the automated segmentation provides safe and coherent surface models that can be used in collision detection during motion planning. The source code of the algorithms will be made publicly available. Conclusion: Optimized trajectory planning based on shape regularized segmentation leads to safe access canals for temporal bone surgery. Functional evaluation shows the promising results for both 3D U-Net and Bézier Spline trajectories.

Show publication details
Ziesing, Alexander Steffen; Kuijper, Arjan [1. Gutachten]; Adamy, Jürgen [2. Gutachten]

#### Rulebook-based Trajectory Planning of an Autonomous Agent for Urban Traffic Scenarios

2020

Ziel dieser Arbeit ist die Umsetzung einer regelbasierten Trajektorienplanung in der Simulationsumgebung CARLA. Dazu soll ein autonomes Fahrzeug mit Sensorik ausgestattet werden, um verschiedene Szenarien simulieren zu können. Dabei sollen Trajektorien aufgespannt werden, die durch eine regelbasierte Entscheidungsfindung bewertet werden. Die Bewertung wird anhand von Kostenfunktionen der Regeln bestimmt. Anschließend wird das modellierte Regelwerk in den Szenarien angewendet und die Ergebnisse diskutiert.

Show publication details

#### Scalable Processing of Massive Geodata in the Cloud: Generating a Level-of-Detail Structure Optimized for Web visualization

2020

Full paper Proceedings of the 23rd AGILE Conference on Geographic Information Science

Conference on Geographic Information Science (AGILE) <23, 2020, Chania, Crete, Creece>

We present a cloud-based approach to transform arbitrarily large terrain data to a hierarchical level-of-detail structure that is optimized for web visualization. Our approach is based on a divide-andconquer strategy. The input data is split into tiles that are distributed to individual workers in the cloud. These workers apply a Delaunay triangulation with a maximum number of points and a maximum geometric error. They merge the results and triangulate them again to generate less detailed tiles. The process repeats until a hierarchical tree of different levels of detail has been created. This tree can be used to stream the data to the web browser. We have implemented this approach in the frameworks Apache Spark and GeoTrellis. Our paper includes an evaluation of our approach and the implementation. We focus on scalability and runtime but also investigate bottlenecks, possible reasons for them, as well as options for mitigation. The results of our evaluation show that our approach and implementation are scalable and that we are able to process massive terrain data.

Show publication details
Schoosleitner, Michael; Ullrich, Torsten

#### Scene Understanding and 3D Imagination: A Comparison between Machine Learning and Human Cognition

2020

Proceedings of the 15th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications

International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP) <15, 2020, Valetta, Malta>

Spatial perception and three-dimensional imagination are important characteristics for many construction tasks in civil engineering. In order to support people in these tasks, worldwide research is being carried out on assistance systems based on machine learning and augmented reality. In this paper, we examine the machine learning component and compare it to human performance. The test scenario is to recognize a partly-assembled model, identify its current status, i.e. the current instruction step, and to return the next step. Thus, we created a database of 2D images containing the complete set of instruction steps of the corresponding 3D model. Afterwards, we trained the deep neural network RotationNet with these images. Usually, the machine learning approaches are compared to each other; our contribution evaluates the machine learning results with human performance tested in a survey: in a clean-room setting the survey and RotationNet results are comparable and neither is significantly better. The real-world results show that the machine learning approaches need further improvements.

Show publication details
Krämer, Michel; Willkomm, Philipp

#### Schnelle und effiziente Verarbeitung von Erdbeobachtungsdaten

2020

Mit dem Start des europäischen Copernicus-Programms wurden die Möglichkeiten im Hinblick auf Erfassung und Verfügbarkeit von Erdbeobachtungsdaten erheblich erweitert. Durch Copernicus werden mehr Erdbeobachtungsdaten als je zuvor aufgenommen, Aktualisierungszyklen werden stark verkürzt und alle Daten sind öffentlich frei verfügbar. Wie sieht es jedoch mit Zugriffsmöglichkeiten und Nutzbarkeit der Daten aus? Nur über einen nutzeroptimierten Zugang lässt sich auch die Wertschöpfung steigern. Hier steht die Branche vor einem IT-Paradigmenwechsel. Statt Desktop-basierten Fachanwendungen werden leistungsfähige und skalierbare Infrastrukturen für die Verarbeitung, Analyse und Visualisierung sehr großer Datenbestände aufgebaut. Die Algorithmen für die Analyse müssen effizient und skalierbar umgesetzt und es muss ein leichter Zugang zu Daten und Rechenkapazitäten gewährleistet werden.

Show publication details
Keßel, Julian Nicola Yves; Kuijper, Arjan [1. Review]; Bargiel, Damian [2. Review]

#### Segmentation of Cropland Field Parcels from SAR-based Phenological Stage Probability Maps

2020

Boundary data on cropland field parcels plays an increasing role for applications including precision farming, distribution of subsidies and yield estimation statistics. The demand on this data is so high and its acquisition area so large that automating the process is worth investigating. This thesis shows a novel method to derive arbitrarily shaped field parcel extents from synthetic probability maps originating from Sentinel-1 Synthetic Aperture Radar imagery. This instance segmentation process is completely automatic and generates a map of universally applicable GIS-Features from the given input rasters. In a two-tier process, the outlines of parcel candidates are first extracted from a multi-temporal stack of Phenological Sequence Pattern probability maps (PSP) using the Maximally Stable Extremal Regions blob detection algorithm for grayscale images. A Constrained Triangulation is applied to yield a 2D intermediate representation of the blob features. Secondly, classification results are used to merge adjacent triangle segments back into contiguous parcels. In this step, an adapted form of Statistical Region Merging (SRM) is applied, merging segments based on their cultivation record over multiple growing periods. An area-based accuracy assessment revealed an overall IoU of 0.58, under-segmentation of 0.18 and over-segmentation of 0.33 which is comparable to performance data presented for an alternative approach by Graesser and Ramankutty 2017. While F1-Scores range from 0.41 for parcels >7.5ha to 0.67 for those smaller than 4ha, the overall minimum precision lies at 43% (recall: 96%). This performance is comparable to another study on the topic (Nyandwi et al. 2019), whereas we exceed its recall of 45%. Our approach offers a new technique of field parcel delineation by using exclusively active remote sensing data. The findings of this work could guide the way towards accurate and reproducible acquisition of field’s shapes and sizes to ultimately replace cadastral and manually acquired data.

Show publication details
Bülow, Maximilian von; Tausch, Reimar; Knauthe, Volker; Wirth, Tristan; Guthe, Stefan; Santos, Pedro; Fellner, Dieter W.

#### Segmentation-Based Near-Lossless Compression of Multi-View Cultural Heritage Image Data

2020

GCH 2020

Eurographics Workshop on Graphics and Cultural Heritage (GCH) <18, 2020, online>

Cultural heritage preservation using photometric approaches received increasing significance in the past years. Capturing of these datasets is usually done with high-end cameras at maximum image resolution enabling high quality reconstruction results while leading to immense storage consumptions. In order to maintain archives of these datasets, compression is mandatory for storing them at reasonable cost. In this paper, we make use of the mostly static background of the capturing environment that does not directly contribute information to 3d reconstruction algorithms and therefore may be approximated using lossy techniques. We use a superpixel and figure-ground segmentation based near-lossless image compression algorithm that transparently decides if regions are relevant for later photometric reconstructions. This makes sure that the actual artifact or structured background parts are compressed with lossless techniques. Our algorithm achieves compression rates compared to the PNG image compression standard ranging from 1:2 to 1:4 depending on the artifact size.

Show publication details

#### Sensing Technology for Human Activity Recognition: a Comprehensive Survey

2020

IEEE Access

Sensors are devices that quantify the physical aspects of the world around us. This ability is important to gain knowledge about human activities. Human Activity recognition plays an import role in people’s everyday life. In order to solve many human-centered problems, such as health-care, and individual assistance, the need to infer various simple to complex human activities is prominent. Therefore, having a well defined categorization of sensing technology is essential for the systematic design of human activity recognition systems. By extending the sensor categorization proposed by White, we survey the most prominent research works that utilize different sensing technologies for human activity recognition tasks. To the best of our knowledge, there is no thorough sensor-driven survey that considers all sensor categories in the domain of human activity recognition with respect to the sampled physical properties, including a detailed comparison across sensor categories. Thus, our contribution is to close this gap by providing an insight into the state-of-the-art developments. We identify the limitations with respect to the hardware and software characteristics of each sensor category and draw comparisons based on benchmark features retrieved from the research works introduced in this survey. Finally, we conclude with general remarks and provide future research directions for human activity recognition within the presented sensor categorization.

Show publication details
Fu, Biying; Kuijper, Arjan [Erstgutachter]; Fellner, Dieter W. [Zweitgutachter]; Van Laerhoven, Kristof [Drittgutachter]

#### Sensor Applications for Human Activity Recognition in Smart Environments

2020

Human activity recognition (HAR) is the automated recognition of individual or group activities from sensor inputs. It deals with a wide range of application areas, such as for health care, assisting technologies, quantified-self and safety applications. HAR is the key to build human-centred applications and enables users to seamlessly and naturally interact with each other or with a smart environment. A smart environment is an instrumented room or space equipped with sensors and actuators to perceive the physical state or human activities within this space. The diversity of sensors makes it difficult to use the appropriate sensor to build specific applications. This work aims at presenting sensor-driven applications for human activity recognition in smart environments by using novel sensing categories beyond the existing sensor technologies commonly applied to these tasks. The intention is to improve the interaction for various sub-fields of human activities. Each application addresses the difficulties following the typical process pipeline for designing a smart environment application. At first, I survey most prominent research works with focus on sensor-driven categorization in the research domain of HAR to identify possible research gaps to position my work. I identify two use-cases: quantified-self and smart home applications. Quantified-self aims at self-tracking and self-knowledge through numbers. Common sensor technology for daily tracking of various aerobic endurance training activities, such as walking, running, or cycling are based on acceleration data with wearable. However, more stationary exercises, such as strength-based training or stretching are also important for a healthy life-style, as they improve body coordination and balance. These exercises are not well tracked by wearing only a single wearable sensor, as these activities rely on coordinated movement of the entire body. I leverage two sensing categories to design two portable mobile applications for remote sensing of these more stationary exercises of physical workout. Sensor-driven applications for smart home domain aim at building systems to make the life of the occupants safer and more convenient. In this thesis, I target at stationary applications to be integrated into the environment to allow a more natural interaction between the occupant and the smart environment. I propose two possible solutions to achieve this task. The first system is a surface acoustic based system which provides a sparse sensor setup to detect a basic set of activities of daily living including the investigation of minimalist sensor arrangement. The second application is a tag-free indoor positioning system. Indoor localization aims at providing location information to build intelligent services for smart homes. Accurate indoor position offers the basic context for high-level reasoning system to achieve more complex contexts. The floor-based localization system using electrostatic sensors is scalable to different room geometries due to its layout and modular composition. Finally, privacy with non-visual input is the main aspect for applications proposed in this thesis. In addition, this thesis addresses the issue of adaptivity from prototypes towards real-world applications. I identify the issues of data sparsity in the training data and data diversity in the real-world data. In order to solve the issue of data sparsity, I demonstrate the data augmentation strategy to be applied on time series to increase the amount of training data by generating synthetic data. Towards mitigating the inherent difference of the development dataset and the real-world scenarios, I further investigate several approaches including metric-based learning and fine-tuning. I explore these methods to finetune the trained model on limited amount of individual data with and without retrain the pre-trained inference model. Finally some examples are stated as how to deploy the offline model to online processing device with limited hardware resources. Personalization is the task that aims at improving quality of products and services by adapting itself to the current user. In the context of automo-tive applications, personalization is not only about how drivers sets up the position of their seat or their favorite radio channels. Going beyond that, personalization is also about the preference of driving styles and the individual behaviors in every maneuver executions. One key challenge in personalization is to be able to capture and understand the users from the historical data produced by the users. The data are usually presented in form of time series and in some cases, those time series can be remarkably long. Capturing and learning from such data poses a challenge for machine learning models. To deal with this problem, this thesis presents an approach that makes uses of recurrent neural networks to capture the time series of behavioral data of drivers and predict theirs lane change intentions. In comparison to previous works, our approach is capable of predicting not only driver’s intention as predefined discrete classes (i. e. left, right and lane keeping) but also as continuous values of the time left until the drivers cross the lane markings. This provides additional information for advanced driver-assistance systems to decide when to warn drivers and when to intervene. There are two further aspects that need to be considered when develop-ing a personalized assistance system: inter- and intra-personalization. The former refers to the differences between different users whereas the later indicates the changes in preferences in one user over time (i. e. the differ-ences in driving styles when driving to work versus when being on a city sightseeing tour). In the scope of this thesis, both problems of inter- and intra-personalization are addressed and tackled. Our approach exploits the correlation in driving style between consecutively executed maneuvers to quickly derive the driver’s current preferences. The introduced networks architecture outperforms non-personalized approaches in predicting the preference of driver when turning left. To tackle inter-personalization prob-lems, this thesis makes use of the Siamese architecture with long short-term memory networks for identifying drivers based on vehicle dynamic information. The evaluation, which is carried out on real-world data set collected from 32 test drivers, shows that the network is able to identify unseen drivers. Further analysis on the trained network indicates that it identifies drivers by comparing their behaviors, especially the approaching and turning behaviors.

Show publication details

#### SER-FIQ: Unsupervised Estimation of Face Image Quality Based on Stochastic Embedding Robustness

2020

2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition

IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) <2020, online>

Face image quality is an important factor to enable high-performance face recognition systems. Face quality assessment aims at estimating the suitability of a face image for the purpose of recognition. Previous work proposed supervised solutions that require artificially or human labelled quality values. However, both labelling mechanisms are error prone as they do not rely on a clear definition of quality and may not know the best characteristics for the utilized face recognition system. Avoiding the use of inaccurate quality labels, we proposed a novel concept to measure face quality based on an arbitrary face recognition model. By determining the embedding variations generated from random subnetworks of a face model, the robustness of a sample representation and thus, its quality is estimated. The experiments are conducted in a cross-database evaluation setting on three publicly available databases. We compare our proposed solution on two face embeddings against six state-of-the-art approaches from academia and industry. The results show that our unsupervised solution outperforms all other approaches in the majority of the investigated scenarios. In contrast to previous works, the proposed solution shows a stable performance over all scenarios. Utilizing the deployed face recognition model for our face quality assessment methodology avoids the training phase completely and further outperforms all baseline approaches by a large margin. Our solution can be easily integrated into current face recognition systems, and can be modified to other tasks beyond face recognition.

Show publication details
Araslanov, Nikita; Roth, Stefan

#### Single-Stage Semantic Segmentation From Image Labels

2020

2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition

IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) <2020, online>

Recent years have seen a rapid growth in new approaches improving the accuracy of semantic segmentation in a weakly supervised setting, i.e. with only image-level labels available for training. However, this has come at the cost of increased model complexity and sophisticated multi-stage training procedures. This is in contrast to earlier work that used only a single stage -- training one segmentation network on image labels -- which was abandoned due to inferior segmentation accuracy. In this work, we first define three desirable properties of a weakly supervised method: local consistency, semantic fidelity, and completeness. Using these properties as guidelines, we then develop a segmentation-based network model and a self-supervised training scheme to train for semantic masks from image-level annotations in a single stage. We show that despite its simplicity, our method achieves results that are competitive with significantly more complex pipelines, substantially outperforming earlier single-stage methods.

• 978-1-7281-7168-5
Show publication details
Berretti, Stefano [Ed.] [et al.]; Fellner, Dieter W. [Proceedings Production Ed.]

#### Smart Tools and Applications in Computer Graphics - Eurographics Italian Chapter Conference

2020

Eurographics Italian Chapter Conference - Smart Tools and Applications in computer Graphics (STAG) <2020, online>

• 978-3-03868-124-3
Show publication details
Vitek, M.; Das, A.; Pourcenoux, Yann; Missler, Alexandre; Paumier, C.; Das, S.; De Ghosh, Ishita; Lucio, Diego Rafael; Zanlorensi Jr., Luiz Antonio; Boutros, Fadi; Damer, Naser; Grebe, Jonas Henry; Kuijper, Arjan; Hu, J.; He, Y.; Wang, C.; Liu, H.; Wang, Y.; Sun, Z.; Osorio-Roig, D.; Rathgeb, Christian; Busch, Christoph; Tapia, Juan; Valenzuela, Andrés; Zampoukis, Georgios; Tsochatzidis, Lazaros; Pratikakis, Ioannis; Nathan, Sabari; Suganya, Ramamoorthy; Mehta, V.; Dhall, Abhinav; Raja, Kiran; Gupta, G.; Khiarak, Jalil Nourmohammadi; Akbari-Shahper, Mohsen; Jaryani, Farhang; Asgari-Chenaghl, Meysam; Vyas, Ritesh; Dakshit, Sagnik; Peer, Peter; Pal, Umapada; Struc, Vitomir; Menotti, David

#### SSBC 2020: Sclera Segmentation Benchmarking Competition in the Mobile Environment

2020

IJCB 2020. IEEE/IARP International Joint Conference on Biometrics

IEEE/IARP International Joint Conference on Biometrics (IJCB) <2020, online>

The paper presents a summary of the 2020 Sclera Segmentation Benchmarking Competition (SSBC), the 7th in the series of group benchmarking efforts centred around the problem of sclera segmentation. Different from previous editions, the goal of SSBC 2020 was to evaluate the performance of sclera-segmentation models on images captured with mobile devices. The competition was used as a platform to assess the sensitivity of existing models to i) differences in mobile devices used for image capture and ii) changes in the ambient acquisition conditions. 26 research groups registered for SSBC 2020, out of which 13 took part in the final round and submitted a total of 16 segmentation models for scoring. These included a wide variety of deep-learning solutions as well as one approach based on standard image processing techniques. Experiments were conducted with three recent datasets. Most of the segmentation models achieved relatively consistent performance across images captured with different mobile devices (with slight differences across devices), but struggled most with low-quality images captured in challenging ambient conditions, i.e., in an indoor environment and with poor lighting.

Show publication details

#### Style-transfer GANs for Bridging the Domain Gap in Synthetic Pose Estimator Training

2020

2020 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR). Proceedings

IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR) <2020, online>

Given the dependency of current CNN architectures on a large training set, the possibility of using synthetic data is alluring as it allows generating a virtually infinite amount of labeled training data. However, producing such data is a nontrivial task as current CNN architectures are sensitive to the domain gap between real and synthetic data.We propose to adopt general-purpose GAN models for pixellevel image translation, allowing to formulate the domain gap itself as a learning problem. The obtained models are then used either during training or inference to bridge the domain gap. Here, we focus on training the single-stage YOLO6D [20] object pose estimator on synthetic CAD geometry only, where not even approximate surface information is available. When employing paired GAN models, we use an edge-based intermediate domain and introduce different mappings to represent the unknown surface properties.Our evaluation shows a considerable improvement in model performance when compared to a model trained with the same degree of domain randomization, while requiring only very little additional effort.

Show publication details

#### STYLE: Style Transfer for Synthetic Training of a YoLo6D Pose Estimator

2020

Supervised training of deep neural networks requires a large amount of training data. Since labeling is time-consuming and error prone and many applications lack data sets of adequate size, research soon became interested in generating this data synthetically, e.g. by rendering images, which makes the annotation free and allows utilizing other sources of available data, for example, CAD models. However, unless much effort is invested, synthetically generated data usually does not exhibit the exact same properties as real-word data. In context of images, there is a difference in the distribution of image features between synthetic and real imagery, a domain gap. This domain gap reduces the transfer-ability of synthetically trained models, hurting their real world inference performance. Current state-of-the-art approaches trying to mitigate this problem concentrate on domain randomization: Overwhelming the model’s feature extractor with enough variation to force it to learn more meaningful features, effectively rendering real-world images nothing more but one additional variation. The main problem with most domain randomization approaches is that it requires the practitioner to decide on the amount of randomization required, a fact research calls "blind" randomization. Domain adaptation in contrast directly tackles the domain gap without the assistance of the practitioner, which makes this approach seem superior. This work deals with training of a DNN-based object pose estimator in three scenarios: First, a small amount of real-world images of the objects of interest is available, second, no images are available, but object specific texture is given, and third, no images and no textures are available. Instead of copying successful randomization techniques, these three problems are tackled mainly with domain adaptation techniques. The main proposition is the adaptation of general-purpose, widely-available, pixel-level style transfer to directly tackle the differences in features found in images from different domains. To that end several approaches are introduced and tested, corresponding to the three different scenarios. It is demonstrated that in scenario one and two, conventional conditional GANs can drastically reduce the domain gap, thereby improving performance by a large margin when compared to non-photo-realistic renderings. More importantly: ready-to-use style transfer solutions improve performance significantly when compared to a model trained with the same degree of randomization, even when there is no real-world data of the target objects available (scenario three), thereby reducing the reliance on domain randomization.

Show publication details
Peter, Christian; Mader, Steffen; Oertel, Karina; Blech, Michael; Schultz, Randolf; Voskamp, Jörg; Urban, Bodo

#### Technologies for Emotion-aware Systems

2020

HCI 2006 Proceedings. Vol. 2

BCS HCI Group Annual Conference (HCI) <20, 2006, London, UK>

Emotions influence physiological processes in humans and are controlled by the autonomous nervous system. Emotions result in generally observable variations in a human's behaviour or physiological parameters that can be accessed with modern technology. For use in HCI contexts, emotion detection technology has to be minimally obtrusive and needs to be easy to use. In this article we present the latest developments in the field of emotion recognition technology undertaken at Fraunhofer IGD, Rostock, with focus on easily measurable physiological parameters. Methods of data mining and knowledge discovery used to build emotion classifiers are briefly described as is the underlying communication framework for networking the emotion processing components involved. Finally, a novel, configurable emotion induction application is introduced along with an emotion visualisation tool, which have been designed especially for emotion studies in the HCI context.

Show publication details
Damer, Naser; Grebe, Jonas Henry; Chen, Cong; Boutros, Fadi; Kirchbuchner, Florian; Kuijper, Arjan

#### The Effect of Wearing a Mask on Face Recognition Performance: an Exploratory Study

2020

BIOSIG 2020

Conference on Biometrics and Electronic Signatures (BIOSIG) <19, 2020, Online>

GI-Edition - Lecture Notes in Informatics (LNI)
P-306

Face recognition has become essential in our daily lives as a convenient and contactless method of accurate identity verification. Process such as identity verification at automatic border control gates or the secure login to electronic devices are increasingly dependant on such technologies. The recent COVID-19 pandemic have increased the value of hygienic and contactless identity verification. However, the pandemic led to the wide use of face masks, essential to keep the pandemic under control. The effect of wearing a mask on face recognition in a collaborative environment is currently sensitive yet understudied issue. We address that by presenting a specifically collected database containing three session, each with three different capture instructions, to simulate realistic use cases.We further study the effect of masked face probes on the behaviour of three top-performing face recognition systems, two academic solutions and one commercial off-the-shelf (COTS) system.

Show publication details

#### Time-unfolding Object Existence Detection in Low-quality Underwater Videos using Convolutional Neural Networks

2020

Proceedings of the 15th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications

International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP) <15, 2020, Valetta, Malta>

Monitoring the environment for early recognition of changes is necessary for assessing the success of renaturation measures on a facts basis. It is also used in fisheries and livestock production for monitoring and for quality assurance. The goal of the presented system is to count sea trouts annually over the course of several months. Sea trouts are detected with underwater camera systems triggered by motion sensors. Such a scenario generates many videos that have to be evaluated manually. This article describes the techniques used to automate the image evaluation process. An effective method has been developed to classify videos and determine the times of occurrence of sea trouts, while significantly reducing the annotation effort. A convolutional neural network has been trained via supervised learning. The underlying images are frame compositions automatically extracted from videos on which sea trouts are to be detected. The accuracy of the resulting detection system reaches values of up to 97.7 %.

Show publication details

#### Towards 3D Digitization in the GLAM (Galleries, Libraries, Archives, and Museums) Sector – Lessons Learned and Future Outlook

2020

The IPSI BgD Transactions on Internet Research

The European Cultural Heritage Strategy for the 21st century, within the Digital Agenda, one of the flagship initiatives of the Europe 2020 Strategy, has led to an increased demand for fast, efficient and faithful 3D digitization technologies for cultural heritage artefacts. 3D digitization has proven to be a promising approach to enable precise reconstructions of objects. Yet, unlike the digital acquisition of cultural goods in 2D which is widely used and automated today, 3D digitization often still requires significant manual intervention, time and money. To enable heritage institutions to make use of large scale, economic, and automated 3D digitization technologies, the Competence Center for Cultural Heritage Digitization at the Fraunhofer Institute for Computer Graphics Research IGD has developed CultLab3D, the world’s first fully automatic 3D mass digitization technology for collections of three-dimensional objects. 3D scanning robots such as the CultArm3D-P are specifically designed to automate the entire 3D digitization process thus allowing to capture and archive objects on a large-scale and produce highly accurate photo-realistic representations. The unique setup allows to shorten the time needed for digitization from several hours to several minutes per artefact.

Show publication details

#### Transforming Seismocardiograms Into Electrocardiograms by Applying Convolutional Autoencoders

2020

2020 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings

International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2020) <45, 2020, online>

Electrocardiograms constitute the key diagnostic tool for cardiologists. While their diagnostic value is yet unparalleled, electrode placement is prone to errors, and sticky electrodes pose a risk for skin irritations and may detach in long-term measurements. Heart.AI presents a fundamentally new approach, transforming motion-based seismocardiograms into electrocardiograms interpretable by cardiologists. Measurements are conducted simply by placing a sensor on the user’s chest. To generate the transformation model, we trained a convolutional autoencoder with the publicly available CEBS dataset. The transformed ECG strongly correlates with the ground truth (r=.94, p<.01), and important features (number of R-peaks, QRS-complex durations) are modeled realistically (Bland-Altman analyses, p>0.12). On a 5- point Likert scale, 15 cardiologists rated the morphological and rhythmological validity as high (4.63/5 and 4.8/5, respectively). Our electrodeless approach solves crucial problems of ECG measurements while being scalable, accessible and inexpensive. It contributes to telemedicine, especially in low-income and rural regions worldwide.

Show publication details

#### Unconstrained Workout Activity Recognition on Unmodified Commercial Off-the-Shelf Smartphones

2020

Proceedings of the 13th ACM International Conference on PErvasive Technologies Related to Assistive Environments

ACM International Conference on PErvasive Technologies Related to Assistive Environments (PETRA) <13, 2020, Corfu, Greece>

ACM International Conference Proceedings Series (ICPS)

Smartphones have become an essential part of our lives. Especially its computing power and its current specifications make a modern smartphone even more powerful than the computers NASA used to send astronauts to the moon. Equipped with various integrated sensors, a modern smartphone can be leveraged for lots of smart applications. In this paper, we investigate the possibility of using a unmodified commercial off-the-shelf (COTS) smartphone to recognize 8 different workout exercises. App-based workout has become popular in the last few years. People do not need to go to the gym to practice. The advantage of using a mobile device is, that you can practice anywhere at anytime. In this work, we turned a COTS smartphone to an active sonar device to leverage the echo reflected from exercising movement close to the device. By conducting a test study with 14 participants performing these eight exercises, we show first results for cross person evaluation and the generalization ability of our inference models on unseen participants. A bidirectional LSTM model achieved an overall F1 score of 88.86 % for the cross subject case and 79.52 % for the holdout participants evaluation. Similar good results can be achieved by a VGG16 fine-tuned model in comparison to a 2D-CNN architecture trained from scratch.

Show publication details
Fellner, Dieter W. [Hrsg.]; Welling, Daniela [Red.]; Ackeren, Janine van [Red.]; Bergstedt, Bettina [Red.]; Krüger, Kathrin [Red.]; Prasche, Svenja [Advisor]; Bornemann, Heidrun [Red.]; Roth, Anahit [Red.]

#### Unser Jahr 2019

2020

Die Visual-Computing-Anwendungen des Fraunhofer IGD setzen auf eine realitätsgetreue Visualisierung und verbinden diese mit wichtigem Spezialwissen, um komplexe Sachverhalte bereits in der Planungsphase zu vermitteln. Wir bieten sowohl Fachleuten als auch Bürgerinnen und Bürgern eine interaktive 3D-Webanwendung an, welche die Projektentwicklung in einen nachvollziehbaren, realistischen Kontext stellt – dies führt zu deutlich mehr Akzeptanz der Ergebnisse. Die Stadt Hamburg setzt ein solches Szenario bereits um. Bürger können neue Pflanzorte für Bäume vorschlagen und erhalten eine direkte Rückmeldung, ob alle Richtlinien eingehalten werden – zugleich informiert die städtische Planungssoftware über mögliche Ausweichorte. Transparenz und ein schnelles Feedback lassen die Bürgerinnen und Bürger aktiv und produktiv an urbanen Planungsprozessen mitwirken. Das Prinzip ist übertragbar. Ob es um Infrastruktur für den Breitbandausbau geht, um Verkehr oder erneuerbare Energien: Alle Beteiligten treffen sich zeit- und ortsunabhängig, allen liegen die gleichen umfassenden Informationen vor – diskutiert wird auf virtueller Ebene. Auch im Bildungsbereich hat das Fraunhofer IGD neue Möglichkeiten geschaffen: ökonomisch, ökologisch, effizient. Wer über mehrere Sinneskanäle lernt, etwa über Sprache und Bilder, kann Wissen besser abspeichern. So üben seit 2019 ehrenamtliche Helfer beim Deutschen Roten Kreuz in virtuellen Trainingswelten, wie sich ein Einsatz im Rettungswagen genau gestaltet – manchmal ist eben kein Rettungswagen zum Üben verfügbar. Oder: Was passiert, wenn Auszubildende bei der Heidelberger Druckmaschinen AG komplexes Gerät verstehen, warten und reparieren sollen? Die Produktion stoppen und die Maschine auseinanderbauen und wieder zusammensetzen? Dank virtueller Lernräume können die Auszubildenden die Abläufe im Inneren der Maschine »sehen«, erkennen und verstehen. Visual Computing mit Virtual Reality (VR) und Augmented Reality (AR) ist und bleibt spannend, nicht nur für die Wissenschaft: Laut einer Studie von PricewaterhouseCoopers haben VR und AR großes Potenzial. 2030 werden allein in Deutschland 400 000 Menschen am Arbeitsplatz damit zu tun haben, derzeit sind es 15 000.

Show publication details

#### Vibroarthrography using Convolutional Neural Networks

2020

Proceedings of the 13th ACM International Conference on PErvasive Technologies Related to Assistive Environments

ACM International Conference on PErvasive Technologies Related to Assistive Environments (PETRA) <13, 2020, Corfu, Greece>

ACM International Conference Proceedings Series (ICPS)

Knees, hip, and other human joints generate noise and vibration while they move. The vibration and sound pattern is characteristic not only for the type of joint but also for the condition. The pattern vary due to abrasion, damage, injury, and other causes. Therefore, the vibration and sound analysis, also known as vibroarthrography (VAG), provides information and possible conclusions about the joint condition, age and health state. The analysis of the pattern is very sophisticated and complex and so approaches of machine learning techniques were applied before. In this paper, we are using convolutional neural networks for the analysis of vibroarthrographic signals and compare the results with already known machine learning techniques.

Show publication details

#### Virtual Lenses as Embodied Tools for Immersive Analytics

2020

Virtuelle und Erweiterte Realität

Workshop der GI-Fachgruppe VR/AR: Virtuelle und Erweiterte Realität <17, 2020, online>

Interactive lenses are useful tools for supporting the analysis of data in differentways. Most existing lenses are designed for 2D visualization and are operated using standardmouse and keyboard interaction. On the other hand, research on virtual lenses for novel3D immersive visualization environments is scarce. Our work aims to narrow this gap inthe literature. We focus particularly on the interaction with lenses. Inspired by naturalinteraction with magnifying glasses in the real world, our lenses are designed as graspabletools that can be created and removed as needed, manipulated and parameterized dependingon the task, and even combined to flexibly create new views on the data. We implementedour ideas in a system for the visual analysis of 3D sonar data. Informal user feedback frommore than 100 people suggests that the designed lens interaction is easy to use for the taskof finding a hidden wreck in sonar data.

Show publication details
Müller, Martin; Petzold, Markus; Wunderlich, Marcel; Baumgartl, Tom; Höhn, Markus; Eichel, Vanessa; Mutters, Nico T.

#### Visual Analysis for Hospital Infection Control using a RNN Model

2020

EuroVA 2020

International EuroVis Workshop on Visual Analytics (EuroVA) <2020, Norrköping, Sweden>

Bacteria and viruses are transmitted among patients in the hospital. Infection control experts develop strategies for infection control. Currently, this is done mostly manually, which is time-consuming and error-prone. Visual analysis approaches mainly focus disease spread on population level.We learn a RNN model for detection of potential infections, transmissions and infection factors. We present a novel interactive visual interface to explore the model results. Together with infection control experts, we apply our approach to real hospital data. The experts could identify factors for infections and derive infection control measures.

Show publication details
Metzler, Simon Konstantin; Kuijper, Arjan [1. Review]; Yeste Magdaleno, Javier [2. Review]

#### Visually-aware Recommendation System for Interior Design

2020

Suitable recommendations are critical for a successful e-commerce experience, especially for product categories such as furniture. A well thought-out choice of furniture is decisive for the visual appearance and the comfort of a room. Interior design can take much time and not everyone is capable to do it. Some furniture stores offer recommendation systems on their website, which are usually based on collaborative filters that are very restrictive, can be inaccurate and require many data at first. This work aims to develop a method to provide set recommendations that adhere to a cohesive visual style. The method can automatically advise the user on what set of furniture to choose for a room around one seed piece. The proposed system uses a database where learned attributes of the dataset are previously stored. Once the user select a seed, the system extracts the attributes from the image to execute a query in the database. Finally, a visual search performed in the filtered subset will return the best candidates. This way has the advantage to receive the results faster and to reduce the searching space thereby improving efficiency. The system is presented that is both powerful and efficient enough to give useful user-specific recommendations in real-time.

Show publication details
Gove, Robert [General Co-Chair]; Kohlhammer, Jörn [Program Co-Chair] [et al.]

#### VizSec 2019

2020

IEEE Symposium on Visualization for Cyber Security (VizSec) <16, 2019>

The Symposium on Visualization for Cyber Security is now in its 16th year, and this year it will be in conjunction with IEEE VIS for the 11th time. As cyber systems continue to permeate crucial elements of our lives it becomes increasingly important to critically examine how we can ensure the security of cyber systems related to things like election security, self-driving cars, and machine learning technology. With these technologies affecting all corners of society, it continues to be important to bring together diverse perspectives from researchers and practitioners from academia, government, and industry to address the needs of cyber security through effective and scalable visualization and analysis techniques.

• 978-1-7281-3876-3
Show publication details
Kohlhammer, Jörn [Program Co-Chair] [et al.]

#### VizSec 2020

2020

IEEE Symposium on Visualization for Cyber Security (VizSec) <17, 2020, online>

• 978-1-7281-8262-9
Show publication details
Altenhofen, Christian; Fellner, Dieter W. [1. Gutachten]; Stork, André [2. Gutachten]; Augsdörfer, Ursula H. [3. Gutachten]

#### Volumetric Subdivision for Efficient Integrated Modeling and Simulation

2020

Continuous surface representations, such as B-spline and Non-Uniform Rational B-spline (NURBS) surfaces are the de facto standard for modeling 3D objects – thin shells and solid objects alike – in the field of Computer-Aided Design (CAD). For performing physically based simulation, Finite Element Analysis (FEA) has been the industry standard for many years. In order to analyze physical properties such as stability, aerodynamics, or heat dissipation, the continuous models are discretized into finite element (FE) meshes. A tight integration of and a smooth transition between geometric design and physically based simulation are key factors for an eÿcient design and engineering workflow. Con-verting a CAD model from its continuous boundary representation (B-Rep) into a discrete volumetric representation for simulation is a time-consuming process that introduces approximation errors and often requires manual interaction by the engineer. Deriving design changes directly from the simulation results is especially diÿcult as the meshing process is irreversible. Isogeometric Analysis (IGA) tries to overcome this meshing hurdle by using the same representation for describing the geometry and for performing the simulation. Most commonly, IGA is performed on bivariate and trivariate spline representations (B-spline or NURBS surfaces and volumes) [HCB05]. While existing CAD B-Rep models can be used directly for simulating thin-shell objects, simulating solid objects requires a conversion from spline surfaces to spline volumes. As spline volumes need a trivariate tensor-product topology, complex 3D objects must be represented via trimming or by connecting multiple spline volumes, limiting the continuity to C0 [ME16; DSB19]. As an alternative to NURBS or B-splines, subdivision models allow for representing complex topologies with as a single entity, removing the need for trimming or tiling and potentially providing higher continuity. While subdivision surfaces have shown promising results for designing and simulating shells [WHP11; Pan+15; RAF16], IGA on subdivision volumes remained mostly unexplored apart from the work of Burkhart et al. [BHU10b; Bur11]. In this dissertation, I investigate how volumetric subdivision representations are beneficial for a tighter integration of geometric modeling and physically based simulation. Focusing on Catmull-Clark (CC) solids, I present novel techniques in the areas of eÿcient limit evaluation, volumetric modeling, numerical integration, and mesh quality analysis. I present an eÿcient link to FEA, as well as my IGA approach on CC solids that improves upon Burkhart et al.’s proof of concept [BHU10b] with constant-time limit evaluation, more accurate integration, and higher mesh quality. Eÿcient limit evaluation is a key requirement when working with subdivision models in geometric design, visualization, simulation, and 3D printing. In this dissertation, I present the first method for constant-time volumetric limit evaluation of CC solids. It is faster than the subdivision-based approach by Burkhart et al. [BHU10b] for every topological constellation and parameter point that would require more than two local subdivision steps. Adapting the concepts of well-known surface modeling tools, I present a volumetric modeling environment for CC-solid control meshes. Consistent volumetric modeling operations built from a set of novel volumetric Euler operators allow for creating and modifying topologically consistent volumetric meshes. Furthermore, I show how to manipulate groups of control points via parameters, how to avoid intersections with inner control points while modeling the outer surface, and how to use CC solids in the context of multi-material additive manufacturing. For coupling of volumetric subdivision models with established FE frameworks, I present an eÿcient and consistent tetrahedral mesh generation technique for CC solids. The technique exploits the inherent volumetric structure of CC-solid models and is at least 26× faster than the tetrahedral meshing algorithm provided by CGAL [Jam+15]. This allows to re-create or update the tetrahedral mesh almost instantly when changing the CC-solid model. However, the mesh quality strongly depends on the quality of the control mesh. In the context of structural analysis, I present my IGA approach on CC solids. The IGA approach yields converging stimulation results for models with fewer elements and fewer degrees of freedom than FE simulations on tetrahedral meshes with linear and higher-order basis functions. The solver also requires fewer iterations to solve the linear system due to the higher continuity throughout the simulation model provided by the subdivision basis functions. Extending Burkhart et al.’s method [BHU10b], my hierarchical quadrature scheme for irregular CC-solid cells increases the accuracy of the integrals for computing surface areas and element sti˛nesses. Furthermore, I introduce a quality metric that quantifies the parametrization quality of the limit volume, revealing distortions, inversions, and singularities. The metric shows that cells with multiple adjacent boundary faces induce singularities in the limit, even for geometrically well-shaped control meshes. Finally, I present a set of topological operations for splitting such boundary cells – resolving the singularities. These improvements further reduce the amount of elements required to obtain converging results as well as the time required for solving the linear system.

Show publication details

#### Web-based Prostate Visualization Tool

2020

Proceedings of the 2020 Annual Meeting of the German Society of Biomedical Engineering

Jahrestagung der Deutschen Gesellschaft für Biomedizinische Technik im VDE (BMT) <54, 2020, online>

Current Directions in Biomedical Engineering

Proper treatment of prostate cancer is essential toincrease the survival chance. In this sense, numerous studiesshow how important the communication between all stakeholders in the clinic is. This communication is difficult because of the lack of conventions while referring to the locationwhere a biopsy for diagnosis was taken. This becomes evenmore challenging taking into account that experts of differentfields work on the data and have different requirements. In thispaper a web-based communication tool is proposed that incorporates a visualization of the prostate divided into 27 segments according to the PI-RADS protocol. The tool provides2 working modes that consider the requirements of radiologistand pathologist while keeping it consistent. The tool comprisesall relevant information given by pathologists and radiologists,such as, severity grades of the disease or tumor length. Everything is visualized using a colour code for better undestanding.

Show publication details

#### Web-Based Visualization of Big Geospatial Vector Data

2020

Geospatial Technologies for Local and Regional Development

Conference on Geographic Information Science (AGILE) <22, 2019, Limassol, Cyprus>

Lecture Notes in Geoinformation and Cartography (LNGC)

Today, big data is one of the most challenging topics in computer science. To give customers, developers or domain experts an overview of their data, it needs to be visualized. In case data contains geospatial information, it becomes more difficult, because most users have a well-trained experience how to explore geographic information. A common map interface allows users zooming and panning to explore the whole dataset. This paper focuses on an approach to visualize huge sets of geospatial data in modern web browsers along with maintaining a dynamic tile tree. The contribution of this work is, to make it possible to render over one million polygons integrated in a modern web application by using 2D Vector Tiles. A major challenge is the map interface providing interaction features such as data-driven filtering and styling of vector data for intuitive data exploration. A web application requests, handles and renders the vector tiles. Such an application has to keep its responsiveness for a better user experience. Our approach to build and maintain the tile tree database provides an interface to import new data and more valuable a flexible way to request Vector Tiles. This is important to face the issues regarding memory allocation in modern web applications.

Show publication details
Höhn, Markus; Wunderlich, Marcel; Ballweg, Kathrin; Landesberger, Tatiana von

#### Width-Scale Bar Charts for Data with Large Value Range

2020

EuroVis 2020. Eurographics / IEEE VGTC Conference on Visualization 2020. Short Papers

Eurographics / IEEE VGTC Conference on Visualization (EuroVis) <22, 2020, online>

Data sets with large value range are difficult to visualize with traditional linear bar charts. Usually, a logarithmic scale isused in these cases. However, the logarithmic scale suffers from non-linearity. Recently, scale-stack bar charts and magnitudemarkers, improve the readability of values. However, they have other disadvantages such as various scales or several objectsfor visualizing one value. We propose the width-scale bar chart that uses width, height and color to cover a large value rangewithin one linear scale. A quantitative user study shows advantages of our design – especially for reading values.

Show publication details
Kraft, Dimitri; Srinivasan, Karthik; Bieber, Gerald

#### Wrist-worn Accelerometer based Fall Detection for Embedded Systems and IoT devices using Deep Learning Algorithms

2020

Proceedings of the 13th ACM International Conference on PErvasive Technologies Related to Assistive Environments

ACM International Conference on PErvasive Technologies Related to Assistive Environments (PETRA) <13, 2020, Corfu, Greece>

ACM International Conference Proceedings Series (ICPS)

With increasing age, elderly persons are falling more often. While a third of people over 65 years are falling once a year, hospitalized people over 80 years are falling multiple times per year. A reliable fall detection is absolutely necessary for a fast help. Therefore, wristworn accelerometer based fall detection systems are developed but the accuracy and precision is not standardized, comparable or sometimes even known. In this paper, we present an overview about existing public databases with sensor based fall datasets and harmonize existing wrist-worn datasets for a broader and robust evaluation. Furthermore, we are analyzing the current possible recognition rate of fall detection using deep learning algorithms for mobile and embedded systems. The presented results and databases can be used for further research and optimizations in order to increase the recognition rate to enhance the independent life of the elderly.