List of publications
A Two-stage Outlier Filtering Framework for City-Scale Localization using 3D SfM Point Clouds
IEEE Transactions on Image Processing
3D Structure-based localization aims to estimate the 6-DOF camera pose of a query image by means of feature matches against a 3D Structure-from-Motion (SfM) point cloud. For city-scale SfM point clouds with tens of millions of points, it becomes more and more difficult to disambiguate matches. Therefore a 3D Structure-based localization method, which can efficiently handle matches with very large outlier ratios, is needed. We propose a two-stage outlier filtering framework for city-scale localization that leverages both visibility and geometry intrinsics of SfM point clouds. Firstly, we propose a visibility-based outlier filter, which is based on a bipartite visibility graph, to filter outliers on a coarse level. Secondly, we apply a geometry-based outlier filter to generate a set of fine-grained matches with a novel data-driven geometrical constraint for efficient inlier evaluation. The proposed two-stage outlier filtering framework only relies on intrinsic information of a SfM point cloud. It is thus widely applicable to be embedded into existing localization approaches. The experimental results on two real-world datasets demonstrate the effectiveness of the proposed two-stage outlier filtering framework for city-scale localization
An Experimental Overview on Electric Field Sensing
Journal of Ambient Intelligence and Humanized Computing
Electric fields exist everywhere. They are influenced by living beings, conductive materials, and other charged entities. Electric field sensing is a passive capacitive measurement technique that detects changes in electric fields and has a very low power consumption. We explore potential applications of this technology and compare it to other measurement approaches, such as active capacitive sensing. Five prototypes have been created that give an overview of the potential use cases and how they compare to other technologies. Our results reveal that electric field sensing can be used for indoor applications as well as outdoor applications. Even a mobile usage is possible due to the low energy consumption of this technology.
CATARACTS: Challenge on Automatic Tool Annotation for cataRACT Surgery
Medical Image Analysis
Surgical tool detection is attracting increasing attention from the medical image analysis community. The goal generally is not to precisely locate tools in images, but rather to indicate which tools are being used by the surgeon at each instant. The main motivation for annotating tool usage is to design e_cient solutions for surgical workflow analysis, with potential applications in report generation, surgical training and even real-time decision support. Most existing tool annotation algorithms focus on laparoscopic surgeries. However, with 19 million interventions per year, the most common surgical procedure in the world is cataract surgery. The CATARACTS challenge was organized in 2017 to evaluate tool annotation algorithms in the specific context of cataract surgery. It relies on more than nine hours of videos, from 50 cataract surgeries, in which the presence of 21 surgical tools was manually annotated by two experts. With 14 participating teams, this challenge can be considered a success. As might be expected, the submitted solutions are based on deep learning. This paper thoroughly evaluates these solutions: in particular, the quality of their annotations are compared to that of human interpretations. Next, lessons learnt from the di_erential analysis of these solutions are discussed. We expect that they will guide the design of e_cient surgery monitoring tools in the near future.
Detecting Face Morphing Attacks by Analyzing the Directed Distances of Facial Landmarks Shifts
German Conference on Pattern Recognition (GCPR) <40, 2018, Stuttgart, Germany>
Lecture Notes in Computer Science (LNCS), 11269
Face morphing attacks create face images that are verifiable to multiple identities. Associating such images to identity documents lead to building faulty identity links, causing attacks on operations like border crossing. Most of previously proposed morphing attack detection approaches directly classified features extracted from the investigated image. We discuss the operational opportunity of having a live face probe to support the morphing detection decision and propose a detection approach that take advantage of that. Our proposed solution considers the facial landmarks shifting patterns between reference and probe images. This is represented by the directed distances to avoid confusion with shifts caused by other variations. We validated our approach using a publicly available database, built on 549 identities. Our proposed detection concept is tested with three landmark detectors and proved to outperform the baseline concept based on handcrafted and transferable CNN features.
Driving with Style: Inverse Reinforcement Learning in General-Purpose Planning for Automated Driving
IEEE/RSJ 2019 International Conference on Intelligent Robots and Systems
IEEE/RSJ International Conference on Intelligent Robotics and Systems (IROS) <2019, Macau, China>
Behavior and motion planning play an important role in automated driving. Traditionally, behavior planners instruct local motion planners with predefined behaviors. Due to the high scene complexity in urban environments, unpredictable situations may occur in which behavior planners fail to match predefined behavior templates. Recently, general-purpose planners have been introduced, combining behavior and local motion planning. These general-purpose planners allow behavior-aware motion planning given a single reward function. However, two challenges arise: First, this function has to map a complex feature space into rewards. Second, the reward function has to be manually tuned by an expert. Manually tuning this reward function becomes a tedious task. In this paper, we propose an approach that relies on human driving demonstrations to automatically tune reward functions. This study offers important insights into the driving style optimization of general-purpose planners with maximum entropy inverse reinforcement learning. We evaluate our approach based on the expected value difference between learned and demonstrated policies. Furthermore, we compare the similarity of human driven trajectories with optimal policies of our planner under learned and expert-tuned reward functions. Our experiments show that we are able to learn reward functions exceeding the level of manual expert tuning without prior domain knowledge.
Gaussian-Product Subdivision Surfaces
ACM Transactions on Graphics
International Conference on Computer Graphics and Interactive Techniques (SIGGRAPH) <46, 2019, Los Angeles, USA>
Probabilistic distribution models like Gaussian mixtures have shown greatpotential for improving both the quality and speed of several geometricoperators. This is largely due to their ability to model large fuzzy data usingonly a reduced set of atomic distributions, allowing for large compressionrates at minimal information loss. We introduce a new surface model thatutilizes these qualities of Gaussian mixtures for the definition and controlof a parametric smooth surface. Our approach is based on an enrichedmesh data structure, which describes the probability distribution of spatialsurface locations around each vertex via a Gaussian covariance matrix. Byincorporating this additional covariance information, we show how to definea smooth surface via a nonlinear probabilistic subdivision operator based onproducts of Gaussians, which is able to capture rich details at fixed controlmesh resolution. This entails new applications in surface reconstruction,modeling, and geometric compression.
Implementing Secure Applications in Smart City Clouds Using Microservices
Future Generation Computer Systems
Smart Cities make use of ICT technology to address the challenges of modern urban management. The cloud provides an efficient and cost-effective platform on which they can manage, store and process data, as well as build applications performing complex computations and analyses. The quickly changing requirements in a Smart City require flexible software architectures that let these applications scale in a distributed environment such as the cloud. Smart Cities have to deal with huge amounts of data including sensitive information about infrastructure and citizens. In order to leverage the benefits of the cloud, in particular in terms of scalability and cost-effectiveness, this data should be stored in a public cloud. However, in such an environment, sensitive data needs to be encrypted to prevent unauthorized access. In this paper, we present a software architecture design that can be used as a template for the implementation of Smart City applications. The design is based on the microservice architectural style, which provides properties that help make Smart City applications scalable and flexible. In addition, we present a hybrid approach to securing sensitive data in the cloud. Our architecture design combines a public cloud with a trusted private environment. To store data in a cost-effective manner in the public cloud, we encrypt metadata items with CP-ABE (Ciphertext-Policy Attribute-Based Encryption) and actual Smart City data with symmetric encryption. This approach allows data to be shared across multiple administrations and makes efficient use of cloud resources. We show the applicability of our design by implementing a web-based application for urban risk management. We evaluate our architecture based on qualitative criteria, benchmark the performance of our security approach, and discuss it regarding honest-but-curious cloud providers as well as attackers trying to access user data through eavesdropping. Our findings indicate that the microservice architectural style fits the requirements of scalable Smart City applications while the proposed security approach helps prevent unauthorized access.
Motif-driven Retrieval of Greek Painted Pottery
Eurographics Workshop on Graphics and Cultural Heritage (GCH) <17, 2019, Sarajevo, Bosnia and Herzegovina>
The analysis of painted pottery is instrumental for understanding ancient Greek society and human behavior of past cultures in Archaeology. A key part of this analysis is the discovery of cross references to establish links and correspondences. However, due to the vast amount of documented images and 3D scans of pottery objects in today’s domain repositories, manual search is very time consuming. Computer aided retrieval methods are of increasing importance. Mostly, current retrieval systems for this kind of cultural heritage data only allow to search for pottery of similar vessel’s shape. However, in many cases important similarity cues are given by motifs painted on these vessels. We present an interactive retrieval system that makes use of this information to allow for a motif-driven search in cultural heritage repositories. We address the problem of unsupervised motif extraction for preprocessing and the shape-based similarity search for Greek painted pottery. Our experimental evaluation on relevant repository data demonstrates effectiveness of our approach on examples of different motifs of interests.
Redefining A in RGBA: Towards a Standard for Graphical 3D Printing
ACM Transactions on Graphics
Advances inmultimaterial 3D printing have the potential to reproduce various visual appearance attributes of an object in addition to its shape. Since many existing 3D file formats encode color and translucency by RGBA texturesmapped to 3Dshapes, RGBA information is particularly important for practical applications. In contrast to color (encoded by RGB), which is specified by the object’s reflectance, selected viewing conditions, and a standard observer, translucency (encoded by A) is neither linked to any measurable physical nor perceptual quantity. Thus, reproducing translucency encoded by A is open for interpretation. In this article, we propose a rigorous definition for A suitable for use in graphical 3D printing, which is independent of the 3D printing hardware and software, and which links both optical material properties and perceptual uniformity for human observers. By deriving our definition from the absorption and scattering coefficients of virtual homogenous reference materials with an isotropic phase function, we achieve two important properties. First, a simple adjustment of A is possible, which preserves the translucency appearance if an object is rescaled for printing. Second, determining the value of A for a real (potentially non-homogenous) material, can be achieved by minimizing a distance function between light transport measurements of this material and simulated measurements of the reference materials. Such measurements can be conducted by commercial spectrophotometers used in graphic arts. Finally, we conduct visual experiments employing the method of constant stimuli, and we derive from them an embedding of A into a nearly perceptually uniform scale of translucency for the reference materials.