The “Selected Readings in Computer Graphics 2019” consist of 40 articles selected from a total of 112 scientific publications.

They were contributed by the Fraunhofer Institute for Computer Graphics Research IGD with offices in Darmstadt as well as in Rostock, Singapore, and Graz, the partner institutes at the respective universities, the Interactive Graphics Systems Group of Technische Universität Darmstadt, the Computergraphics and Communication Group of the Institute of Computer Science at Rostock University, Nanyang Technological University (NTU), Singapore, and the Visual Computing Cluster of Excellence of Graz University of Technology, that cooperate closely within projects and research and development in the field of Computer Graphics.

All articles previously appeared in various scientific books, journals, conferences and workshops, and are reprinted with permission of the respective copyright holders. The publications had to undergo a thorough review process by internationally leading experts and established technical societies. Therefore, the Selected Readings should give a fairly good and detailed overview of the scientific developments in Computer Graphics in the year 2019. They are put together by Professor Dieter W. Fellner, the director of Fraunhofer Institute for Computer Graphics Research IGD in Darmstadt, at the same time professor at the Department of Computer Science at Technische Universität Darmstadt, and professor at the Faculty of Computer Science at Graz University of Technology.

The Selected Readings in Computer Graphics 2019 touch aspects and trends in Computer Graphics research and development in the areas of

Liste der Publikationen

Show publication details

Ulmer, Alex; Sessler, David; Kohlhammer, Jörn

NetCapVis: Web-based Progressive Visual Analytics for Network Packet Captures

2020

VizSec 2019

IEEE Symposium on Visualization for Cyber Security (VizSec) <16, 2019>

Network traffic log data is a key data source for forensic analysis of cybersecurity incidents. Packet Captures (PCAPs) are the raw information directly gathered from the network device. As the bandwidth and connections to other hosts rise, this data becomes very large quickly. Malware analysts and administrators are using this data frequently for their analysis. However, the currently most used tool Wireshark is displaying the data as a table, making it difficult to get an overview and focus on the significant parts. Also, the process of loading large files into Wireshark takes time and has to be repeated each time the file is closed. We believe that this problem poses an optimal setting for a client-server infrastructure with a progressive visual analytics approach. The processing can be outsourced to the server while the client is progressively updated. In this paper we present NetCapVis, an web-based progressive visual analytics system where the user can upload PCAP files, set initial filters to reduce the data before uploading and then instantly interact with the data while the rest is progressively loaded into the visualizations.

Show publication details

Ma, Jingting; Wang, Anqi; Lin, Feng; Wesarg, Stefan; Erdt, Marius

A novel robust kernel principal component analysis for nonlinear statistical shape modeling from erroneous data

2019

Computerized Medical Imaging and Graphics

Statistical Shape Models (SSMs) have achieved considerable success in medical image segmentation. A high quality SSM is able to approximate the main plausible variances of a given anatomical structure to guide segmentation. However, it is technically challenging to derive such a quality model because: (1) the distribution of shape variance is often nonlinear or multi-modal which cannot be modeled by standard approaches assuming Gaussian distribution; (2) as the quality of annotations in training data usually varies, heavy corruption will degrade the quality of the model as a whole. In this work, these challenges are addressed by introducing a generic SSM that is able to model nonlinear distribution and is robust to outliers in training data. Without losing generality and assuming a sparsity in nonlinear distribution, a novel Robust Kernel Principal Component Analysis (RKPCA) for statistical shape modeling is proposed with the aim of constructing a low-rank nonlinear subspace where outliers are discarded. The proposed approach is validated on two different datasets: a set of 30 public CT kidney pairs and a set of 49 MRI ankle bones volumes. Experimental results demonstrate a significantly better performance on outlier recovery and a higher quality of the proposed model as well as lower segmentation errors compared to the state-of-the-art techniques.

Show publication details

Cheng, Wentao; Chen, Kan; Lin, Weisi; Goesele, Michael; Zhang, Xinfeng; Zhang, Yabin

A Two-stage Outlier Filtering Framework for City-Scale Localization using 3D SfM Point Clouds

2019

IEEE Transactions on Image Processing

3D Structure-based localization aims to estimate the 6-DOF camera pose of a query image by means of feature matches against a 3D Structure-from-Motion (SfM) point cloud. For city-scale SfM point clouds with tens of millions of points, it becomes more and more difficult to disambiguate matches. Therefore a 3D Structure-based localization method, which can efficiently handle matches with very large outlier ratios, is needed. We propose a two-stage outlier filtering framework for city-scale localization that leverages both visibility and geometry intrinsics of SfM point clouds. Firstly, we propose a visibility-based outlier filter, which is based on a bipartite visibility graph, to filter outliers on a coarse level. Secondly, we apply a geometry-based outlier filter to generate a set of fine-grained matches with a novel data-driven geometrical constraint for efficient inlier evaluation. The proposed two-stage outlier filtering framework only relies on intrinsic information of a SfM point cloud. It is thus widely applicable to be embedded into existing localization approaches. The experimental results on two real-world datasets demonstrate the effectiveness of the proposed two-stage outlier filtering framework for city-scale localization

Show publication details

Araslanov, Nikita; Rothkopf, Constantin A.; Roth, Stefan

Actor-Critic Instance Segmentation

2019

2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition

IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) <2019, Long Beach, California, USA>

Most approaches to visual scene analysis have emphasised parallel processing of the image elements. However,one area in which the sequential nature of vision is apparent, is that of segmenting multiple, potentially similar andpartially occluded objects in a scene. In this work, we revisit the recurrent formulation of this challenging problemin the context of reinforcement learning. Motivated by thelimitations of the global max-matching assignment of theground-truth segments to the recurrent states, we developan actor-critic approach in which the actor recurrently predicts one instance mask at a time and utilises the gradientfrom a concurrently trained critic network. We formulatethe state, action, and the reward such as to let the criticmodel long-term effects of the current prediction and incorporate this information into the gradient signal. Furthermore, to enable effective exploration in the inherentlyhigh-dimensional action space of instance masks, we learna compact representation using a conditional variationalauto-encoder. We show that our actor-critic model consistently provides accuracy benefits over the recurrent baselineon standard instance segmentation benchmarks.

Show publication details

Wilmsdorff, Julian von; Kirchbuchner, Florian; Fu, Biying; Braun, Andreas; Kuijper, Arjan

An Experimental Overview on Electric Field Sensing

2019

Journal of Ambient Intelligence and Humanized Computing

Electric fields exist everywhere. They are influenced by living beings, conductive materials, and other charged entities. Electric field sensing is a passive capacitive measurement technique that detects changes in electric fields and has a very low power consumption. We explore potential applications of this technology and compare it to other measurement approaches, such as active capacitive sensing. Five prototypes have been created that give an overview of the potential use cases and how they compare to other technologies. Our results reveal that electric field sensing can be used for indoor applications as well as outdoor applications. Even a mobile usage is possible due to the low energy consumption of this technology.

Show publication details

Laxman Ahire, Amit; Basgier, Dennis

AR Tracking with Hybrid, Agnostic And Browser Based Approach

2019

2019 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR 2019). Proceedings

IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR) <2, 2019, San Diego, CA>

Mobile platform tools are desirable when it comes to practical augmented reality applications. With the convenience and portability that the form factor has to offer, it lays an ideal basic foundation for a feasible use case in industry and commercial applications. Here, we present a novel approach of using the monocular Simultaneous Localization and Mapping (SLAM) [1], [2] information provided by a Cross-Reality (XR) device [3] to augment the linked 3D CAD models. The main objective is to use the tracking technology for an augmented and mixed reality experience by tracking a 3D model and superimposing its respective 3D CAD model data over the images we receive from the camera feed of the XR device without any scene preparation (e.g markers or feature maps). The intent is to conduct a visual analysis and evaluations based on the intrinsic and extrinsic of the model in the visualization system that instant3Dhub [4] has to offer. To achieve this we make use of the Apple’s ARKit to obtain the images, sensor data and SLAM heuristic of client XR device, remote marker-less model based 3D object tracking from monocular RGB image data and hybrid client server architecture. Our approach is agnostic of any SLAM system or Augmented Reality (AR) framework. We make use of the Apple’s ARKit because of the its ease of use, affordability, stability and maturity as a platform and as an integrated system.

Show publication details

Oyarzun Laura, Cristina; Hofmann, Patrick; Drechsler, Klaus; Wesarg, Stefan

Automatic Detection of the Nasal Cavities and Paranasal Sinuses Using Deep Neural Networks

2019

2019 IEEE International Symposium on Biomedical Imaging

IEEE International Symposium on Biomedical Imaging (ISBI) <16, 2019, Venice, Italy>

The nasal cavity and paranasal sinuses present large interpatient variabilities. Additional circumstances like for example, concha bullosa or nasal septum deviations complicate their segmentation. As in other areas of the body a previous multistructure detection could facilitate the segmentation task. In this paper an approach is proposed to individually detect all sinuses and the nasal cavity. For a better delimitation of their borders the use of an irregular polyhedron is proposed. For an accurate prediction the Darknet-19 deep neural network is used which combined with the You Only Look Once method has shown very promising results in other fields of computer vision. 57 CT scans were available of which 85% were used for training and the remaining 15% for validation.

978-1-5386-3640-4

Show publication details

Cheng, Wentao; Lin, Weisi; Chen, Kan; Zhang, Xinfeng

Cascaded Parallel Filtering for Memory-Efficient Image-Based Localization

2019

2019 International Conference on Computer Vision Workshops. Proceedings

International Conference on Computer Vision (ICCV) <17, 2019, Seoul, Korea>

Image-based localization (IBL) aims to estimate the 6DOF camera pose for a given query image. The camera pose can be computed from 2D-3D matches between a query image and Structure-from-Motion (SfM) models. Despite recent advances in IBL, it remains difficult to simultaneously resolve the memory consumption and match ambiguity problems of large SfM models. In this work, we propose a cascaded parallel filtering method that leverages the feature, visibility and geometry information to filter wrong matches under binary feature representation. The core idea is that we divide the challenging filtering task into two parallel tasks before deriving an auxiliary camera pose for final filtering. One task focuses on preserving potentially correct matches, while another focuses on obtaining high quality matches to facilitate subsequent more powerful filtering. Moreover, our proposed method improves the localization accuracy by introducing a quality-aware spatial reconfiguration method and a principal focal length enhanced pose estimation method. Experimental results on real-world datasets demonstrate that our method achieves very competitive localization performances in a memory-efficient manner.

Show publication details

Al Hajj, Hassan; Mukhopadhyay, Anirban; Lamard, Mathieu; Conze, Pierre-Henri; Roychowdhury, Soumali; Hu, Xiaowei; Marsalkaite, Gabija; Zisimopoulos, Odysseas; Dedmari, Muneer Ahmad; Zhao, Fenqiang; Prellberg, Jonas; Sahu, Manish; Galdran, Adrian; Araujo, Teresa; Vo, Duc My; Panda, Chandan; Dahiya, Navdeep; Kondo, Satoshi; Bian, Zhengbing; Vahdat, Arash; Bialopetravicius, Jonas; Flouty, Evangello; Qiu, Chenhui; Dill, Sabrina; Costa, Pedro; Aresta, Guilherme; Ramamurthy, Senthil; Lee, Sang-Woong; Campilho, Aurelio; Zachow, Stefan; Xia, Shunren; Conjeti, Sailesh; Stoyanov, Danail; Armaitis, Jogundas; Heng, Pheng-Ann; Macready, William G.; Cochener, Beatrice; Quellec, Gwenole

CATARACTS: Challenge on Automatic Tool Annotation for cataRACT Surgery

2019

Medical Image Analysis

Surgical tool detection is attracting increasing attention from the medical image analysis community. The goal generally is not to precisely locate tools in images, but rather to indicate which tools are being used by the surgeon at each instant. The main motivation for annotating tool usage is to design e_cient solutions for surgical workflow analysis, with potential applications in report generation, surgical training and even real-time decision support. Most existing tool annotation algorithms focus on laparoscopic surgeries. However, with 19 million interventions per year, the most common surgical procedure in the world is cataract surgery. The CATARACTS challenge was organized in 2017 to evaluate tool annotation algorithms in the specific context of cataract surgery. It relies on more than nine hours of videos, from 50 cataract surgeries, in which the presence of 21 surgical tools was manually annotated by two experts. With 14 participating teams, this challenge can be considered a success. As might be expected, the submitted solutions are based on deep learning. This paper thoroughly evaluates these solutions: in particular, the quality of their annotations are compared to that of human interpretations. Next, lessons learnt from the di_erential analysis of these solutions are discussed. We expect that they will guide the design of e_cient surgery monitoring tools in the near future.

Show publication details

Damer, Naser; Boutros, Fadi; Kirchbuchner, Florian; Kuijper, Arjan

D-ID-Net: Two-Stage Domain and Identity Learning for Identity-Preserving Image Generation From Semantic Segmentation

2019

2019 International Conference on Computer Vision Workshops. Proceedings

International Conference on Computer Vision (ICCV) <17, 2019, Seoul, Korea>

Training functionality-demanding AR/VR systems require accurate and robust gaze estimation and tracking solutions. Achieving such a performance requires the availability of diverse eye image data that might only be acquired by the means of image generation. Works addressing the generation of such images did not target realistic and identity-specific images, nor did they address the practicalrelevant case of generation from semantic labels. Therefore, this work proposes a solution to generate realistic and identity-specific images that correspond to semantic labels, given samples of a specific identity. Our proposed solution consists of two stages. In the first stage, a network is trained to transform the semantic label into a corresponding eye image of a generic identity. The second stage is an identity-specific network that induces identity details on the generic eye image. The results of our D-ID-Net solutions shows a high degree of identity-preservation and similarity to the ground-truth images, with an RMSE of 7.235.

Show publication details

Damer, Naser; Boller, Viola; Wainakh, Yaza; Boutros, Fadi; Terhörst, Philipp; Braun, Andreas; Kuijper, Arjan

Detecting Face Morphing Attacks by Analyzing the Directed Distances of Facial Landmarks Shifts

2019

Pattern Recognition

German Conference on Pattern Recognition (GCPR) <40, 2018, Stuttgart, Germany>

Lecture Notes in Computer Science (LNCS), 11269

Face morphing attacks create face images that are verifiable to multiple identities. Associating such images to identity documents lead to building faulty identity links, causing attacks on operations like border crossing. Most of previously proposed morphing attack detection approaches directly classified features extracted from the investigated image. We discuss the operational opportunity of having a live face probe to support the morphing detection decision and propose a detection approach that take advantage of that. Our proposed solution considers the facial landmarks shifting patterns between reference and probe images. This is represented by the directed distances to avoid confusion with shifts caused by other variations. We validated our approach using a publicly available database, built on 549 identities. Our proposed detection concept is tested with three landmark detectors and proved to outperform the baseline concept based on handcrafted and transferable CNN features.

978-3-030-12938-5

Show publication details

Rosbach, Sascha; James, Vinit; Großjohann, Simon; Homoceanu, Silviu; Roth, Stefan

Driving with Style: Inverse Reinforcement Learning in General-Purpose Planning for Automated Driving

2019

IEEE/RSJ 2019 International Conference on Intelligent Robots and Systems

IEEE/RSJ International Conference on Intelligent Robotics and Systems (IROS) <2019, Macau, China>

Behavior and motion planning play an important role in automated driving. Traditionally, behavior planners instruct local motion planners with predefined behaviors. Due to the high scene complexity in urban environments, unpredictable situations may occur in which behavior planners fail to match predefined behavior templates. Recently, general-purpose planners have been introduced, combining behavior and local motion planning. These general-purpose planners allow behavior-aware motion planning given a single reward function. However, two challenges arise: First, this function has to map a complex feature space into rewards. Second, the reward function has to be manually tuned by an expert. Manually tuning this reward function becomes a tedious task. In this paper, we propose an approach that relies on human driving demonstrations to automatically tune reward functions. This study offers important insights into the driving style optimization of general-purpose planners with maximum entropy inverse reinforcement learning. We evaluate our approach based on the expected value difference between learned and demonstrated policies. Furthermore, we compare the similarity of human driven trajectories with optimal policies of our planner under learned and expert-tuned reward functions. Our experiments show that we are able to learn reward functions exceeding the level of manual expert tuning without prior domain knowledge.

Show publication details

Luu, Thu Huong; Altenhofen, Christian; Ewald, Tobias; Stork, André; Fellner, Dieter W.

Efficient slicing of Catmull–Clark solids for 3D printed objects with functionally graded material

2019

Computers & Graphics

In the competition for the volumetric representation most suitable for functionally graded materials in additively manufactured (AM) objects, volumetric subdivision schemes, such as Catmull-Clark (CC) solids, are widely neglected. Although they show appealing properties, e_cient implementations of some fundamental algorithms are still missing. In this paper, we present a fast algorithm for direct slicing of CC-solids generating bitmaps printable by multi-material AMmachines. Our method optimizes runtime by exploiting constant time limit evaluation and other structural characteristics of CCsolids. We compare our algorithm with the state of the art in trivariate trimmed spline representations and show that our algorithm has similar runtime behavior as slicing trivariate splines, fully supporting the benefits of CC-solids.

Show publication details

Silva, Nelson; Blascheck, Tanja; Jianu, Radu; Rodrigues, Nils; Weiskopf, Daniel; Raubal, Martin; Schreck, Tobias

Eye Tracking Support for Visual Analytics Systems

2019

ETRA '19

ACM Symposium on Eye Tracking Research & Applications (ETRA) <11, 2019, Denver, Colorado>

Visual analytics (VA) research provides helpful solutions for interactive visual data analysis when exploring large and complex datasets. Due to recent advances in eye tracking technology, promising opportunities arise to extend these traditional VA approaches. Therefore, we discuss foundations for eye tracking support in VA systems. We first review and discuss the structure and range of typical VA systems. Based on a widely used VA model, we present five comprehensive examples that cover a wide range of usage scenarios. Then, we demonstrate that the VA model can be used to systematically explore how concrete VA systems could be extended with eye tracking, to create supportive and adaptive analytics systems. This allows us to identify general research and application opportunities, and classify them into research themes. In a call for action, we map the road for future research to broaden the use of eye tracking and advance visual analytics.

Show publication details

Boutros, Fadi; Damer, Naser; Kirchbuchner, Florian; Kuijper, Arjan

Eye-MMS: Miniature Multi-Scale Segmentation Network of Key Eye-Regions in Embedded Applications

2019

2019 International Conference on Computer Vision Workshops. Proceedings

International Conference on Computer Vision (ICCV) <17, 2019, Seoul, Korea>

Segmentation of the iris or sclera is an essential processing block in ocular biometric systems. However, humancomputer interaction, as in VR/AR applications, requires multiple region segmentation to enable smoother interaction and eye-tracking. Such application does not only demand highly accurate and generalizable segmentation, it requires such segmentation model to be appropriate for the limited computational power of embedded systems. This puts strict limits on the size of the deployed deep learning models. This work presents a miniature multi-scale segmentation network consisting of inter-connected convolutional modules. We present a baseline multi-scale segmentation network and modify it to reduce its parameters by more than 80 times, while reducing its accuracy by less than 3%, resulting in our Eye-MMS model containing only 80k parameters. This work is developed on the OpenEDS database and is conducted in preparation for the OpenEDS Semantic Segmentation Challenge.

Show publication details

Krispel, Ulrich; Ullrich, Torsten; Tamke, Martin

Formalising Expert Knowledge for Building Information Models: Automated Identification of Electrical Wiring from 3D Scans

2019

Proceedings of International Academic Conference on Places and Technologies

International Academic Conference on Places and Technologies <3, 2016, Belgrade, Serbia>

New computational methods provide means to deduce semantic information from measurements, such as range scans and photographs of building interiors. In this paper, we showcase a method that allows to estimate elements that are not directly observable – ducts and power lines in walls. For this, we combine explicit information, which is deduced by algorithms from measured data, with implicit information that is publicly available: technical standards that restrict the placement of electrical power lines. We present a complete pipeline from measurements to a hypothesis of these power lines within walls. The approach is structured into the following steps: First, a coarse geometry is extracted from input measurements; i.e., the unstructured point cloud which was acquired by laser scanning is transformed into a simplistic building model. Then, visible endpoints of electrical appliances (e.g. sockets, switches) are detected from photos using machine learning techniques and a pre-trained classifier. Afterwards, positions of installation zones in walls are generated. Finally, a hypothesis of non-visible cable ducts is generated, under the assumption that (i) the real configuration obeys the rules of legal requirements and standards and (ii) the configuration connects all endpoints using a minimal amount of resources, i.e. cable length.

Show publication details

Bartschat, Andreas; Allgeier, Stephan; Scherr, Tim; Stegmaier, Johannes; Bohn, Sebastian; Reichert, Klaus-Martin; Kuijper, Arjan; Reischl, Markus; Stachs, Oliver; Köhler, Bernd; Mikut, Ralf

Fuzzy tissue detection for real-time focal control in corneal confocal microscopy

2019

at - Automatisierungstechnik

Corneal confocal laser scanning microscopy is a promising method for in vivo investigation of cellular structures, e. g., of nerve fibers in the sub-basal nerve plexus. During recording, even slight displacements of the focal plane lead to images of adjacent tissue layers. In this work, we propose a closed-loop control of the focal plane. To detect and evaluate the visible tissues, we utilize the Bag of Visual Words approach to implement a customizable image processing pipeline for real-time applications. Furthermore, we show that the proposed model can be trained with small classification datasets and can be applied as a segmentation method. The proposed control loop, including tissue detection, is implemented in a proof-of-concept setup and shows promising results in a first evaluation with a human subject.

Show publication details

Preiner, Reinhold; Boubekeur, Tamy; Wimmer, Michael

Gaussian-Product Subdivision Surfaces

2019

ACM Transactions on Graphics

International Conference on Computer Graphics and Interactive Techniques (SIGGRAPH) <46, 2019, Los Angeles, USA>

Probabilistic distribution models like Gaussian mixtures have shown greatpotential for improving both the quality and speed of several geometricoperators. This is largely due to their ability to model large fuzzy data usingonly a reduced set of atomic distributions, allowing for large compressionrates at minimal information loss. We introduce a new surface model thatutilizes these qualities of Gaussian mixtures for the definition and controlof a parametric smooth surface. Our approach is based on an enrichedmesh data structure, which describes the probability distribution of spatialsurface locations around each vertex via a Gaussian covariance matrix. Byincorporating this additional covariance information, we show how to definea smooth surface via a nonlinear probabilistic subdivision operator based onproducts of Gaussians, which is able to capture rich details at fixed controlmesh resolution. This entails new applications in surface reconstruction,modeling, and geometric compression.

Show publication details

Abrams, Jesse F.; Vashishtha, Anand; Wong, Seth T.; Nguyen, An; Mohamed, Azlan; Wieser, Sebastian; Kuijper, Arjan; Wilting, Andreas; Mukhopadhyay, Anirban

Habitat-Net: Segmentation of Habitat Images Using Deep Learning

2019

Ecological Informatics

Understanding environmental factors that influence forest health, as well as the occurrence and abundance of wildlife, is a central topic in forestry and ecology. However, the manual processing of field habitat data is time-consuming and months are often needed to progress from data collection to data interpretation. To shorten the time to process the data we propose here Habitat-Net: a novel deep learning application based on Convolutional Neural Networks (CNN) to segment habitat images of tropical rainforests. Habitat-Net takes color images as input and after multiple layers of convolution and deconvolution, produces a binary segmentation of the input image. We worked on two different types of habitat datasets that are widely used in ecological studies to characterize the forest conditions: canopy closure and understory vegetation. We trained the model with 800 canopy images and 700 understory images separately and then used 149 canopy and 172 understory images to test the performance of Habitat-Net. We compared the performance of Habitat-Net to the performance of a simple threshold based method, manual processing by a second researcher and a CNN approach called U-Net, upon which Habitat-Net is based. Habitat-Net, U-Net and simple thresholding reduced total processing time to milliseconds per image, compared to 45 s per image for manual processing. However, the higher mean Dice coefficient of Habitat-Net (0.94 for canopy and 0.95 for understory) indicates that accuracy of Habitat-Net is higher than that of both the simple thresholding (0.64, 0.83) and U-Net (0.89, 0.94). Habitat-Net will be of great relevance for ecologists and foresters, who need to monitor changes in their forest structures. The automated workflow not only reduces the time, it also standardizes the analytical pipeline and, thus, reduces the degree of uncertainty that would be introduced by manual processing of images by different people (either over time or between study sites).

Show publication details

Kügler, David; Krumb, Henry John; Bredemann, Judith; Stenin, Igor; Kristin, Julia; Klenzner, Thomas; Schipper, Jörg; Schmitt, Robert; Sakas, Georgios; Mukhopadhyay, Anirban

High-precision Evaluation of Electromagnetic Tracking

2019

International Journal of Computer Assisted Radiology and Surgery

Purpose: Navigation in high-precision minimally invasive surgery (HP-MIS) demands high tracking accuracy in the absence of line of sight (LOS). Currently, no tracking technology can satisfy this requirement. Electromagnetic tracking (EMT) is the best tracking paradigm in the absence of LOS despite limited accuracy and robustness. Novel evaluation protocols are needed to ensure high-precision and robust EMT for navigation in HP-MIS. Methods: We introduce a novel protocol for EMT measurement evaluation featuring a high-accuracy phantom based on LEGO, which is calibrated by a coordinate measuring machine to ensure accuracy. Our protocol includes relative sequential positions and an uncertainty estimation of positioning. We show effects on distortion compensation using a learned interpolation model. Results: Our high-precision protocol clarifies properties of errors and uncertainties of EMT for high-precision use cases. For EMT errors reaching clinically relevant 0.2 mm, our design is 5–10 times more accurate than previous protocols with 95% confidence margins of 0.02 mm. This high-precision protocol ensures the performance improvement in compensated EMT by 0.05 mm. Conclusion: Our protocol improves the reliability of EMT evaluations because of significantly lower protocol-inherent uncertainties. To reduce patient risk in HP-MIS and to evaluate magnetic field distortion compensation, more high-accuracy protocols such as the one proposed here are required.

Show publication details

Krämer, Michel; Frese, Sven; Kuijper, Arjan

Implementing Secure Applications in Smart City Clouds Using Microservices

2019

Future Generation Computer Systems

Smart Cities make use of ICT technology to address the challenges of modern urban management. The cloud provides an efficient and cost-effective platform on which they can manage, store and process data, as well as build applications performing complex computations and analyses. The quickly changing requirements in a Smart City require flexible software architectures that let these applications scale in a distributed environment such as the cloud. Smart Cities have to deal with huge amounts of data including sensitive information about infrastructure and citizens. In order to leverage the benefits of the cloud, in particular in terms of scalability and cost-effectiveness, this data should be stored in a public cloud. However, in such an environment, sensitive data needs to be encrypted to prevent unauthorized access. In this paper, we present a software architecture design that can be used as a template for the implementation of Smart City applications. The design is based on the microservice architectural style, which provides properties that help make Smart City applications scalable and flexible. In addition, we present a hybrid approach to securing sensitive data in the cloud. Our architecture design combines a public cloud with a trusted private environment. To store data in a cost-effective manner in the public cloud, we encrypt metadata items with CP-ABE (Ciphertext-Policy Attribute-Based Encryption) and actual Smart City data with symmetric encryption. This approach allows data to be shared across multiple administrations and makes efficient use of cloud resources. We show the applicability of our design by implementing a web-based application for urban risk management. We evaluate our architecture based on qualitative criteria, benchmark the performance of our security approach, and discuss it regarding honest-but-curious cloud providers as well as attackers trying to access user data through eavesdropping. Our findings indicate that the microservice architectural style fits the requirements of scalable Smart City applications while the proposed security approach helps prevent unauthorized access.

Show publication details

Chegini, Mohammad; Bernard, Jürgen; Berger, Philip; Sourin, Alexei; Andrews, Keith; Schreck, Tobias

Interactive labelling of a multivariate dataset for supervised machine learning using linked visualisations, clustering, and active learning

2019

Visual Informatics

Supervised machine learning techniques require labelled multivariate training datasets. Many approaches address the issue of unlabelled datasets by tightly coupling machine learning algorithmswith interactive visualisations. Using appropriate techniques, analysts can play an active role in ahighly interactive and iterative machine learning process to label the dataset and create meaningfulpartitions. While this principle has been implemented either for unsupervised, semi-supervised, orsupervised machine learning tasks, the combination of all three methodologies remains challenging.In this paper, a visual analytics approach is presented, combining a variety of machine learningcapabilities with four linked visualisation views, all integrated within the mVis (multivariate Visualiser)system. The available palette of techniques allows an analyst to perform exploratory data analysis ona multivariate dataset and divide it into meaningful labelled partitions, from which a classifier canbe built. In the workflow, the analyst can label interesting patterns or outliers in a semi-supervisedprocess supported by active learning. Once a dataset has been interactively labelled, the analyst cancontinue the workflow with supervised machine learning to assess to what degree the subsequentclassifier has effectively learned the concepts expressed in the labelled training dataset. Using a noveltechnique called automatic dimension selection, interactions the analyst had with dimensions of themultivariate dataset are used to steer the machine learning algorithms.A real-world football dataset is used to show the utility of mVis for a series of analysis and labellingtasks, from initial labelling through iterations of data exploration, clustering, classification, and activelearning to refine the named partitions, to finally producing a high-quality labelled training datasetsuitable for training a classifier. The tool empowers the analyst with interactive visualisations includingscatterplots, parallel coordinates, similarity maps for records, and a new similarity map for partitions

Show publication details

Terhörst, Philipp; Damer, Naser; Braun, Andreas; Kuijper, Arjan

Minutiae-Based Gender Estimation for Full and Partial Fingerprints of Arbitrary Size and Shape

2019

Computer Vision - ACCV 2018

Asian Conference on Computer Vision (ACCV) <14, 2018, Perth, Australia>

Lecture Notes in Computer Science (LNCS), 11361

Since fingerprints are one of the most widely deployed biometrics, accurate fingerprint gender estimation can positively affect several applications. For example, in criminal investigations, gender classification may significantly minimize the list of potential subjects. Previous work mainly offered solutions for the task of gender classification based on complete fingerprints. However, partial fingerprint captures are frequently occurring in many applications, including forensics and the fast growing field of consumer electronics. Due to its huge variability in size and shape, gender estimation on partial fingerprints is a challenging problem. Therefore, in this work we propose a flexible gender estimation scheme by building a gender classifier based on an ensemble of minutiae. The outputs of the single minutia gender predictions are combined by a novel adjusted score fusion approach to obtain an enhanced gender decision. Unlike classical solutions this allows to deal with unconstrained fingerprint parts of arbitrary size and shape. We performed investigations on a publicly available database and our proposed solution proved to significantly outperform state-of-the-art approaches on both full and partial fingerprints. The experiments indicate a reduction in the gender estimation error by 19.34% on full fingerprints and 28.33% on partial captures in comparison to previous work.

978-3-030-20886-8

Show publication details

Lengauer, Stefan; Komar, Alexander; Labrada, Arniel; Karl, Stephan; Trinkl, Elisabeth; Preiner, Reinhold; Bustos, Benjamin; Schreck, Tobias

Motif-driven Retrieval of Greek Painted Pottery

2019

GCH 2019

Eurographics Workshop on Graphics and Cultural Heritage (GCH) <17, 2019, Sarajevo, Bosnia and Herzegovina>

The analysis of painted pottery is instrumental for understanding ancient Greek society and human behavior of past cultures in Archaeology. A key part of this analysis is the discovery of cross references to establish links and correspondences. However, due to the vast amount of documented images and 3D scans of pottery objects in today’s domain repositories, manual search is very time consuming. Computer aided retrieval methods are of increasing importance. Mostly, current retrieval systems for this kind of cultural heritage data only allow to search for pottery of similar vessel’s shape. However, in many cases important similarity cues are given by motifs painted on these vessels. We present an interactive retrieval system that makes use of this information to allow for a motif-driven search in cultural heritage repositories. We address the problem of unsupervised motif extraction for preprocessing and the shape-based similarity search for Greek painted pottery. Our experimental evaluation on relevant repository data demonstrates effectiveness of our approach on examples of different motifs of interests.

Show publication details

Chegini, Mohammad; Andrews, Keith; Schreck, Tobias; Sourin, Alexei

Multiple Linked-View Exploration on Large Displays Facilitated by a Secondary Handheld Device

2019

International Workshop on Advanced Image Technology (IWAIT) 2019

International Workshop on Advanced Image Technology (IWAIT) <2019, Singapore>

Proceedings of SPIE, 11049

Large displays are capable of visualising a large amount of data on multiple views including scatterplots and parallel coordinates and are often present in meeting rooms. They can be used to interact with a dataset and foster discussion among team members. Although some of these large screens have multi-touch capabilities, in many cases it is cumbersome to have to stand close to the display in order to interact with it. One of the solutions is to use a small handheld display to interact with the large display. This paper discusses how traditional interactions such as selection, brushing, and linking can be performed using a secondary handheld device. As a proof of concept, a system including scatterplots and parallel coordinates views is implemented. The interactions are straightforward and are useful for any interactive visual analysis application on a large display with wireless connectivity.

Show publication details

Krause, Silvio; Haescher, Marian; Chodan, Wencke; Bieber, Gerald

Nocturnal Respiration Pattern of healthy people as a hint for sleep state detection

2019

Proceedings of the 12th ACM International Conference on PErvasive Technologies Related to Assistive Environments

ACM International Conference on PErvasive Technologies Related to Assistive Environments (PETRA) <12, 2019, Rhodes, Greece>

ACM International Conference Proceedings Series (ICPS), 01608

Sleep state detection is important to distinguish between a healthy sleep and sleep disorders. Common sleep state analysis methods consist of identifying signals of EEG, EOG, or EMG etc. that can only be assessed in sleep laboratories. The respiration rate and pattern are also affected by the sleep states but are not included in the sleep state analysis method. Since sleep is very important for the recreation of humans, we assume that sleep is mirroring the strain of the day and the general health condition. In our research, we identified a certain respiration rate pattern during sleep in 5 out of 17 healthy persons that might be an identifier for sleep states or for interactions of daytime activity and sleep. Therefore, we introduce this new respiration pattern as “pumping breathing” and compare it with other known respiration patterns.

Show publication details

Fauser, Johannes; Stenin, Igor; Kristin, Julia; Klenzner, Thomas; Schipper, Jörg; Mukhopadhyay, Anirban

Optimizing Clearance of Bézier Spline Trajectories for Minimally-Invasive Surgery

2019

Medical Image Computing and Computer Assisted Intervention - MICCAI 2019

International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI) <22, 2019, Shenzhen, China>

Lecture Notes in Computer Science (LNCS), 11768

Preoperative planning of nonlinear trajectories is a key element in minimally-invasive surgery. Interpolating between start and goal of an intervention while circumnavigating risk structures provides the necessary feasible solutions for such procedure. While recent research shows that Rapidly-exploring Random Trees (RRT) on B´ezier Splines efficiently solve this task, access paths computed by this method do not provide optimal clearance to surrounding anatomy. We propose an approach based on sequential convex optimization that rearranges B´ezier Splines computed by an RRT-connect, thereby achieving locally optimal clearance to risk structures. Experiments on real CT data of patients demonstrate the applicability of our approach on two scenarios: catheter insertion through the aorta and temporal bone surgery. We compare distances to risk structures along computed trajectories with the state of the art solution and show that our method results in clinically safer paths.

978-3-030-32254-0

Show publication details

Fauser, Johannes; Chadda, Romol; Goergen, Yannik; Hessinger, Markus; Motzki, Paul; Stenin, Igor; Kristin, Julia; Klenzner, Thomas; Schipper, Jörg; Seelecke, Stefan; Werthschützky, Roland; Kupnik, Mario; Mukhopadhyay, Anirban

Planning for Flexible Surgical Robots via Bézier Spline Translation

2019

IEEE Robotics and Automation Letters

In a minimally invasive surgery, new flexible instruments enable safer and easier access to difficult-to-reach anatomical regions. However, their introduction into the clinical workflow requires robust replanning because navigation errors during surgery render initially planned trajectories infeasible. Such replanning requires to regularly solve an expensive two-point boundary value problem (BVP) that connects the target pose of the instrument with the currently measured one. We propose a hybrid planning scheme that features both robust and safe replanning. This two-step approach first solves the BVP and then transforms the result to circular arcs that fit the motion of our instruments' models. We exploit implicitly defined Bézier splines as a robust method for interpolation in the first step. A novel geometric translation of these splines, then, provides a convenient solution for movement along circular arcs. We consider two example applications: 1) planning for a drilling unit in temporal bone surgery; and 2) guidewires in catheter insertion. Evaluation on real patient data of both temporal bone and aorta show that our proposed hybrid two-step approach achieves, on average, 55% higher replanning rates and provides 31% larger clearance to risk structures, thus improving trajectory quality with regard to clinical safety.

Show publication details

Haescher, Marian; Matthies, Denys J.C.; Krause, Silvio; Bieber, Gerald

Presenting a Data Imputation Concept to Support the Continuous Assessment of Human Vital Data and Activities

2019

Proceedings of the 12th ACM International Conference on PErvasive Technologies Related to Assistive Environments

ACM International Conference on PErvasive Technologies Related to Assistive Environments (PETRA) <12, 2019, Rhodes, Greece>

ACM International Conference Proceedings Series (ICPS), 01608

Data acquisition of mobile tracking devices often suffers from invalid and non-continuous input data streams. This issue especially occurs with current wearables tracking the user’s activity and vital data. Typical reasons include the short battery life and the fact that the body-worn tracking device may be doffed. Other reasons, such as technical issues, can corrupt the data and render it unusable. In this paper, we introduce a data imputation concept which complements and thus fixes incomplete datasets by using a new merging approach that is particularly suitable for assessing activities and vital data. Our technique enables the dataset to become coherent and comprehensive so that it is ready for further analysis. In contrast to previous approaches, our technique enables the controlled creation of continuous data sets that also contain information on the level of uncertainty for possible reconversions, approximations, or later analysis.

Show publication details

Rojtberg, Pavel; Kuijper, Arjan

Real-time texturing for 6D object instance detection from RGB Images

2019

Adjunct Proceedings of the 2019 IEEE International Symposium on Mixed and Augmented Reality

IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct) <18, 2019, Beijing, China>

For objected detection, the availability of color cues strongly influences detection rates and is even a prerequisite for many methods. However, when training on synthetic CAD data, this information is not available. We therefore present a method for generating a texture-map from image sequences in real-time. The method relies on 6 degree-of-freedom poses and a 3D-model being available. In contrast to previous works this allows interleaving detection and texturing for upgrading the detector on-the-fly. Our evaluation shows that the acquired texture-map significantly improves detection rates using the LINEMOD [5] detector on RGB images only. Additionally, we use the texture-map to differentiate instances of the same object by surface color.

Show publication details

Urban, Philipp; Tanksale, Tejas Madan; Brunton, Alan; Vu, Bui Minh; Nakauchi, Shigeki

Redefining A in RGBA: Towards a Standard for Graphical 3D Printing

2019

ACM Transactions on Graphics

Advances inmultimaterial 3D printing have the potential to reproduce various visual appearance attributes of an object in addition to its shape. Since many existing 3D file formats encode color and translucency by RGBA texturesmapped to 3Dshapes, RGBA information is particularly important for practical applications. In contrast to color (encoded by RGB), which is specified by the object’s reflectance, selected viewing conditions, and a standard observer, translucency (encoded by A) is neither linked to any measurable physical nor perceptual quantity. Thus, reproducing translucency encoded by A is open for interpretation. In this article, we propose a rigorous definition for A suitable for use in graphical 3D printing, which is independent of the 3D printing hardware and software, and which links both optical material properties and perceptual uniformity for human observers. By deriving our definition from the absorption and scattering coefficients of virtual homogenous reference materials with an isotropic phase function, we achieve two important properties. First, a simple adjustment of A is possible, which preserves the translucency appearance if an object is rescaled for printing. Second, determining the value of A for a real (potentially non-homogenous) material, can be achieved by minimizing a distance function between light transport measurements of this material and simulated measurements of the reference materials. Such measurements can be conducted by commercial spectrophotometers used in graphic arts. Finally, we conduct visual experiments employing the method of constant stimuli, and we derive from them an embedding of A into a nearly perceptually uniform scale of translucency for the reference materials.

Show publication details

Frank, Sebastian; Kuijper, Arjan

Robust Driver Foot Tracking and Foot Gesture Recognition Using Capacitive Proximity Sensing

2019

Journal of Ambient Intelligence and Smart Environments

Nowadays, there is an increasing trend towards automated driving. This is supported by both driver assistance systems getting more and more available and powerful, and research for car manufacturing industries. As a consequence, driver hands and feet are less involved in vehicle control. Increasing automation will even let them become idle. Recent gesture recognition mainly focuses on hand interaction. This work focuses on possibilities for feet gesture interaction. Many gesture recognition systems rely on computing intensive, privacy concerns causing video systems. Furthermore, these systems require a line of sight and therefore visible interior design integration. The proposed system shows that invisibly integrated capacitive proximity sensors can do the job, too. They do not cause privacy issues and they can be integrated under non-conductive materials. Therefore, there is no visible interior design impact. The proposed solution distinguishes between four feet gestures. There is no limitation to feet movement. Further, an evaluation including six participants and a vehicle legroom mockup shows the system function. This work contributes to the basis of driver foot gesture recognition pointing to further applications and more comprehensive investigations.

Show publication details

Ritz, Martin; Breitfelder, Simon; Santos, Pedro; Kuijper, Arjan; Fellner, Dieter W.

Seamless and Non-repetitive 4D Texture Variation Synthesis and Real-time Rendering for Measured Optical Material Behavior

2019

Computational Visual Media

We show how to overcome the single weakness of an existing fully automatic system for acquisition of spatially varying optical material behavior of real object surfaces. While the expression of spatially varying material behavior with spherical dependence on incoming light as a 4D texture (an ABTF material model) allows flexible mapping onto arbitrary 3D geometry, with photo-realistic rendering and interaction in real time, this very method of texture-like representation exposes it to common problems of texturing, striking in two disadvantages. Firstly, non-seamless textures create visible artifacts at boundaries. Secondly, even a perfectly seamless texture causes repetition artifacts due to their organised placement in large numbers over a 3D surface. We have solved both problems through our novel texture synthesis method that generates a set of seamless texture variations randomly distributed over the surface at shading time. When compared to regular 2D textures, the inter-dimensional coherence of the 4D ABTF material model poses entirely new challenges to texture synthesis, which includes maintaining the consistency of material behavior throughout the 4D space spanned by the spatial image domain and the angular illumination hemisphere. In addition, we tackle the increased memory consumption caused by the numerous variations through a fitting scheme specifically designed to reconstruct the most prominent effects captured in the material model.

Show publication details

Wang, Anqi; Franke, Andreas; Wesarg, Stefan

Semi-automatic Segmentation of JIA-induced Inflammation in MRI Images of Ankle Joints

2019

Medical Imaging 2019: Image Processing

SPIE Medical Imaging Symposium <2019, San Diego, CA, USA>

Proceedings of SPIE, 10949

The autoimmune disease Juvenile Idiopathic Arthritis (JIA) affects children of under 16 years and leads to the symptom of inflamed synovial membranes in affected joints. In clinical practice, characteristics of these inflamed membranes are used to stage the disease progression and to predict erosive bone damage. Manual outlining of inflammatory regions in each slide of a MRI dataset is still the gold standard for detection and quantification, however, this process is very tiresome and time-consuming. In addition, the inter- and intra-observer variability is a known problem of human annotators. We have developed the first method to detect inflamed regions in and around major joints in the human ankle. First, we use an adapted coupled shape model framework to segment the ankle bones in a MRI dataset. Based on these segmentations, joints are defined as locations where two bones are particularly close to each other. A number of potential inflammation candidates are generated using multi-level thresholding. Since it is known that inflamed synovial membranes occur in the proximity of joints, we filter out structures with similar intensities such as vessels and tendons sheaths using not only a vesselness filter, but also their distance to the joints and their size. The method has been evaluated on a set of 10 manually annotated clinical MRI datasets and achieved the following results: Precision 0.6785 ± 0.1584, Recall 0.5388 ± 0.1213, DICE 0.5696 ± 0.0976.

Show publication details

Damer, Naser; Saladie, Alexandra Moseguí; Zienert, Steffen; Wainakh, Yaza; Kirchbuchner, Florian; Kuijper, Arjan; Terhörst, Philipp

To Detect or not to Detect: The Right Faces to Morph

2019

The 12th IAPR International Conference On Biometrics

IAPR International Conference on Biometrics (ICB) <12, 2019, Crete, Greece>

Recent works have studied the face morphing attack detection performance generalization over variations in morphing approaches, image re-digitization, and image source variations. However, these works assumed a constant approach for selecting the images to be morphed (pairing) across their training and testing data. A realistic variation in the pairing protocol in the training data can result in challenges and opportunities for a stable attack detector. This work extensively study this issue by building a novel database with three different pairing protocols and two different morphing approaches. We study the detection generalization over these variations for single image and differential attack detection, along with handcrafted and CNNbased features. Our observations included that training an attack detection solution on attacks created from dissimilar face images, in contrary to the common practice, can result in an overall more generalized detection performance. Moreover, we found that differential attack detection is very sensitive to variations in morphing and pairing protocols.

Show publication details

Fauser, Johannes; Stenin, Igor; Bauer, Markus; Hsu, Wei-Hung; Kristin, Julia; Klenzner, Thomas; Schipper, Jörg; Mukhopadhyay, Anirban

Toward an Automatic Preoperative Pipeline for Image-guided Temporal Bone Surgery

2019

International Journal of Computer Assisted Radiology and Surgery

International Conference on Information Processing in Computer-Assisted Interventions (IPCAI) <10, 2019, Rennes, France>

Purpose: Minimally invasive surgery is often built upon a time-consuming preoperative step consisting of segmentation and trajectory planning. At the temporal bone, a complete automation of these two tasks might lead to faster interventions and more reproducible results, benefiting clinical workflow and patient health. Methods: We propose an automatic segmentation and trajectory planning pipeline for image-guided interventions at the temporal bone. For segmentation, we use a shape regularized deep learning approach that is capable of automatically detecting even the cluttered tiny structures specific for this anatomy.We then perform trajectory planning for both linear and nonlinear interventions on these automatically segmented risk structures. Results: We evaluate the usability of segmentation algorithms for planning access canals to the cochlea and the internal auditory canal on 24 CT data sets of real patients. Our new approach achieves similar results to the existing semiautomatic method in terms of Dice but provides more accurate organ shapes for the subsequent trajectory planning step. The source code of the algorithms is publicly available. Conclusion: Automatic segmentation and trajectory planning for various clinical procedures at the temporal bone are feasible. The proposed automatic pipeline leads to an efficient and unbiased workflow for preoperative planning.

Show publication details

Hashisho, Yousif; Albadawi, Mohamad; Krause, Tom; Lukas, Uwe von

Underwater Color Restoration Using U-Net Denoising Autoencoder

2019

Proceedings of the 11th International Symposium Image and Signal Processing and Analysis

International Symposium on Image and Signal Processing and Analysis (ISPA) <11, 2019, Dubrovnik, Croatia>

Visual inspection of underwater structures by vehicles, e.g. remotely operated vehicles (ROVs), plays an important role in scientific, military, and commercial sectors. However, the automatic extraction of information using software tools is hindered by the characteristics of water which degrade the quality of captured videos. As a contribution for restoring the color of underwater images, Underwater Denoising Autoencoder (UDAE) model is developed using a denoising autoencoder with U-Net architecture. The proposed network takes into consideration the accuracy and the computation cost to enable realtime implementation on underwater visual tasks using end-toend autoencoder network. Underwater vehicles perception is improved by reconstructing captured frames; hence obtaining better performance in underwater tasks. Related learning methods use generative adversarial networks (GANs) to generate color corrected underwater images, and to our knowledge this paper is the first to deal with a single autoencoder capable of producing same or better results. Moreover, image pairs are constructed for training the proposed network, where it is hard to obtain such dataset from underwater scenery. At the end, the proposed model is compared to a state-of-the-art method.

Show publication details

Terhörst, Philipp; Damer, Naser; Kirchbuchner, Florian; Kuijper, Arjan

Unsupervised Privacy-enhancement of Face Representations Using Similarity-sensitive Noise Transformations

2019

Applied Intelligence

Face images processed by a biometric system are expected to be used for recognition purposes only. However, recent work presented possibilities for automatically deducing additional information about an individual from their face data. By using soft-biometric estimators, information about gender, age, ethnicity, sexual orientation or the health state of a person can be obtained. This raises a major privacy issue. Previous works presented supervised solutions that require large amount of private data in order to suppress a single attribute. In this work, we propose a privacy-preserving solution that does not require these sensitive information and thus, works in an unsupervised manner. Further, our approach offers privacy protection that is not limited to a single known binary attribute or classifier. We do that by proposing similarity-sensitive noise transformations and investigate their effect and the effect of dimensionality reduction methods on the task of privacy preservation. Experiments are done on a publicly available database and contain analyses of the recognition performance, as well as investigations of the estimation performance of the binary attribute of gender and the continuous attribute of age. We further investigated the estimation performance of these attributes when the prior knowledge about the used privacy mechanism is explicitly utilized. The results show that using this information leads to significantly enhancement of the estimation quality. Finally, we proposed a metric to evaluate the trade-off between the privacy gain and the recognition loss for privacy-preservation techniques. Our experiments showed that the proposed cosine-sensitive noise transformation was successful in reducing the possibility of estimating the soft private information in the data, while having significantly smaller effect on the intended recognition performance.

Show publication details

Bernard, Jürgen; Hutter, Marco; Reinemuth, Heiko; Pfeifer, Hendrik; Bors, Christian; Kohlhammer, Jörn

Visual-Interactive Preprocessing of Multivariate Time Series Data

2019

Computer Graphics Forum

Eurographics Conference on Visualization (EuroVis) <21, 2019, Porto, Portugal>

Pre-processing is a prerequisite to conduct effective and efficient downstream data analysis. Pre-processing pipelines often require multiple routines to address data quality challenges and to bring the data into a usable form. For both the construction and the refinement of pre-processing pipelines, human-in-the-loop approaches are highly beneficial. This particularly applies to multivariate time series, a complex data type with multiple values developing over time. Due to the high specificity of this domain, it has not been subject to in-depth research in visual analytics. We present a visual-interactive approach for preprocessing multivariate time series data with the following aspects. Our approach supports analysts to carry out six core analysis tasks related to pre-processing of multivariate time series. To support these tasks, we identify requirements to baseline toolkits that may help practitioners in their choice. We characterize the space of visualization designs for uncertainty-aware pre-processing and justify our decisions. Two usage scenarios demonstrate applicability of our approach, design choices, and uncertainty visualizations for the six analysis tasks. This work is one step towards strengthening the visual analytics support for data pre-processing in general and for uncertainty-aware pre-processing of multivariate time series in particular.

Show publication details

Cibulski, Lena; May, Thorsten; Preim, Bernhard; Bernard, Jürgen; Kohlhammer, Jörn

Visualizing Time Series Consistency for Feature Selection

2019

Journal of WSCG

International Conference in Central Europe on Computer Graphics, Visualization and Computer Vision (WSCG) <27, 2019, Plzen, Czech Republic>

Feature selection is an effective technique to reduce dimensionality, for example when the condition of a system is to be understood from multivariate observations. The selection of variables often involves a priori assumptions about underlying phenomena. To avoid the associated uncertainty, we aim at a selection criterion that only considers the observations. For nominal data, consistency criteria meet this requirement: a variable subset is consistent, if no observations with equal values on the subset have different output values. Such a model-agnostic criterion is also desirable for forecasting. However, consistency has not yet been applied to multivariate time series. In this work, we propose a visual consistency-based technique for analyzing a time series subset’s discriminating ability w.r.t. characteristics of an output variable. An overview visualization conveys the consistency of output progressions associated with comparable observations. Interaction concepts and detail visualizations provide a steering mechanism towards inconsistencies. We demonstrate the technique’s applicability based on two real-world scenarios. The results indicate that the technique is open to any forecasting task that involves multivariate time series, because analysts could assess the combined discriminating ability without any knowledge about underlying phenomena.