• Vita
  • Publications
Show publication details

Chen, Cong; Kuijper, Arjan [1. Review]; Damer, Naser [2. Review]

Advanced analyses of CrazyFaces attacks on face identification systems

2020

Darmstadt, TU, Bachelor Thesis, 2020

After 5 years in prison, the greedy criminal was released. He never gave up the idea of sinagain. But he didn’t want to spend another 5 years in prison. So he began to summarizethe lessons of his last arrest. Five years ago, he was arrested at a bank because surveillancecameras identified him. This was a bit of a surprise to him, because this clever criminal hadrepeatedly escaped the pursuit of surveillance cameras by changing his facial expressions.After investigating, he learned that the face recognition system in that bank is a differentone. Therefore, his previously trained facial expressions failed. So one new idea comesin his mind now, "can i just find one or more facial expressions that can disable moststate-of-the-art face recognition systems?". To known the end of this story, please read therest of this thesis.

Show publication details

Venkatesh, Sushma; Zhang, Haoyu; Ramachandra, Raghavendra; Raja, Kiran; Damer, Naser; Busch, Christoph

Can GAN Generated Morphs Threaten Face Recognition Systems Equally as Landmark Based Morphs ? - Vulnerability and Detection

2020

IWBF 2020. Proceedings

International Workshop on Biometrics and Forensics (IWBF) <8, 2020, Porto, Portugal>

The primary objective of face morphing is to com-bine face images of different data subjects (e.g. an malicious actor and an accomplice) to generate a face image that can be equally verified for both contributing data subjects. In this paper, we propose a new framework for generating face morphs using a newer Generative Adversarial Network (GAN) - StyleGAN. In contrast to earlier works, we generate realistic morphs of both high-quality and high resolution of 1024 × 1024 pixels. With the newly created morphing dataset of 2500 morphed face images, we pose a critical question in this work. (i) Can GAN generated morphs threaten Face Recognition Systems (FRS) equally as Landmark based morphs? Seeking an answer, we benchmark the vulnerability of a Commercial-Off-The-Shelf FRS (COTS) and a deep learning-based FRS (ArcFace). This work also benchmarks the detection approaches for both GAN generated morphs against the landmark based morphs using established Morphing Attack Detection (MAD) schemes.

Show publication details

Boutros, Fadi; Damer, Naser; Fang, Meiling; Raja, Kiran; Kirchbuchner, Florian; Kuijper, Arjan

Compact Models for Periocular Verification Through Knowledge Distillation

2020

BIOSIG 2020

Conference on Biometrics and Electronic Signatures (BIOSIG) <19, 2020, Online>

GI-Edition - Lecture Notes in Informatics (LNI), P-306

Despite the wide use of deep neural network for periocular verification, achieving smaller deep learning models with high performance that can be deployed on low computational powered devices remains a challenge. In term of computation cost, we present in this paper a lightweight deep learning model with only 1.1m of trainable parameters, DenseNet-20, based on DenseNet architecture. Further, we present an approach to enhance the verification performance of DenseNet-20 via knowledge distillation. With the experiments on VISPI dataset captured with two different smartphones, iPhone and Nokia, we show that introducing knowledge distillation to DenseNet-20 training phase outperforms the same model trained without knowledge distillation where the Equal Error Rate (EER) reduces from 8.36% to 4.56% EER on iPhone data, from 5.33% to 4.64% EER on Nokia data, and from 20.98% to 15.54% EER on cross-smartphone data.

Show publication details

Terhörst, Philipp; Tran, Mai Ly; Damer, Naser; Kirchbuchner, Florian; Kuijper, Arjan

Comparison-Level Mitigation of Ethnic Bias in Face Recognition

2020

IWBF 2020. Proceedings

International Workshop on Biometrics and Forensics (IWBF) <8, 2020, online>

Current face recognition systems achieve high performance on several benchmark tests. Despite this progress,recent works showed that these systems are strongly biasedagainst demographic sub-groups. Previous works introducedapproaches that aim at learning less biased representations.However, applying these approaches in real applications requiresa complete replacement of the templates in the database. Thisreplacement procedure further requires that a face image ofeach enrolled individual is stored as well. In this work, wepropose the first bias-mitigating solution that works on thecomparison-level of a biometric system. We propose a fairnessdriven neural network classifier for the comparison of twobiometric templates to replace the systems similarity function.This fair classifier is trained with a novel penalization termin the loss function to introduce the criteria of group andindividual fairness to the decision process. This penalization termforces the score distributions of different ethnicities to be similar,leading to a reduction of the intra-ethnic performance differences.Experiments were conducted on two publicly available datasetsand evaluated the performance of four different ethnicities. Theresults showed that for both fairness criteria, our proposedapproach is able to significantly reduce the ethnic bias, whileit preserves a high recognition ability. Our model, build onindividual fairness, achieves bias reduction rate between 15.35%and 52.67%. In contrast to previous work, our solution is easy tointegrate into existing systems by simply replacing the systemssimilarity functions with our fair template comparison approach.

Show publication details

Fang, Meiling; Damer, Naser; Boutros, Fadi; Kirchbuchner, Florian; Kuijper, Arjan

Deep Learning Multi-layer Fusion for an Accurate Iris Presentation Attack Detection

2020

FUSION 2020

International Conference on Information Fusion (FUSION) <23, 2020, Online>

Iris presentation attack detection (PAD) algorithms are developed to address the vulnerability of iris recognition systems to presentation attacks. Taking into account that the deep features successfully improved computer vision performance in various fields including iris recognition, it is natural to use features extracted from deep neural networks for iris PAD. Each layer in a deep learning network carries features of different level of abstraction. The features extracted from the first layer to the higher layers become more complex and more abstract. This might point our complementary information in these features that can collaborate towards an accurate PAD decision. Therefore, we propose an iris PAD solution based on multi-layer fusion. The information extracted from the last several convolutional layers are fused on two levels, feature-level and score-level. We demonstrated experiments on both, off-theshelf pre-trained network and network trained from scratch. An extensive experiment also explores the complementary between different layer combinations of deep features. Our experimental results show that feature-level based multi-layer fusion method performs better than the best single layer feature extractor in most cases. In addition, our fusion results achieve similar or better results than the state-of-the-art algorithms on the Notre Dame and IIITD-WVU databases of the Iris Liveness Detection Competition 2017 (LivDet-Iris 2017).

Show publication details

Drozdowski, Pawel; Rathgeb, Christian; Dantcheva, Antitza; Damer, Naser; Busch, Christoph

Demographic Bias in Biometrics: A Survey on an Emerging Challenge

2020

IEEE Transactions on Technology and Society

Systems incorporating biometric technologies have become ubiquitous in personal, commercial, and governmental identity management applications. Both cooperative (e.g. access control) and non-cooperative (e.g. surveillance and forensics) systems have benefited from biometrics. Such systems rely on the uniqueness of certain biological or behavioural characteristics of human beings, which enable for individuals to be reliably recognised using automated algorithms. Recently, however, there has been a wave of public and academic concerns regarding the existence of systemic bias in automated decision systems (including biometrics). Most prominently, face recognition algorithms have often been labelled as “racist” or “biased” by the media, non-governmental organisations, and researchers alike. The main contributions of this article are: (1) an overview of the topic of algorithmic bias in the context of biometrics, (2) a comprehensive survey of the existing literature on biometric bias estimation and mitigation, (3) a discussion of the pertinent technical and social matters, and (4) an outline of the remaining challenges and future work items, both from technological and social points of view.

Show publication details

Fang, Meiling; Damer, Naser; Kirchbuchner, Florian; Kuijper, Arjan

Demographic Bias in Presentation Attack Detection of Iris Recognition Systems

2020

With the widespread use of biometric systems, the demographic bias problem raises more attention. Although many studies addressed bias issues in biometric verification, there is no works that analyse the bias in presentation attack detection (PAD) decisions. Hence, we investigate and analyze the demographic bias in iris PAD algorithms in this paper. To enable a clear discussion, we adapt the notions of differential performance and differential outcome to the PAD problem. We study the bias in iris PAD using three baselines (hand-crafted, transfer-learning, and training from scratch) using the the NDCLD-2013 database. The experimental results points out that female users will be significantly less protected by the PAD, in comparison to males.

Show publication details

Boutros, Fadi; Damer, Naser; Raja, Kiran; Ramachandra, Raghavendra; Kirchbuchner, Florian; Kuijper, Arjan

Fusing Iris and Periocular Region for User Verification in Head Mounted Displays

2020

FUSION 2020

International Conference on Information Fusion (FUSION) <23, 2020, Online>

The growing popularity of Virtual Reality and Augmented Reality (VR/AR) devices in many applications also demands authentication of users. As the devices inherently capture the eye image while capturing the user interaction, the authentication can be devised using the iris and periocular recognition. While both iris and periocular data being non-ideal unlike the data captured from standard biometric sensors, the authentication performance is expected to be lower. In this work, we present and evaluate a fusion framework for improving the biometric authentication performance. Specifically, we employ score-level fusion for two independent biometric systems of iris and periocular region to avoid expensive feature-level fusion. With a detailed evaluation of three different score-level fusion after the score normalization on a dataset of 12579 images, we report the performance gain in authentication using score-level fusion for iris and periocular recognition.

Show publication details

Boutros, Fadi; Damer, Naser; Raja, Kiran; Ramachandra, Raghavendra; Kirchbuchner, Florian; Kuijper, Arjan

Iris and periocular biometrics for head mounted displays: Segmentation, recognition, and synthetic data generation

2020

Image and Vision Computing

Augmented and virtual reality deployment is finding increasing use in novel applications. Some of these emerging and foreseen applications allow the users to access sensitive information and functionalities. Head Mounted Displays (HMD) are used to enable such applications and they typically include eye facing cameras to facilitate advanced user interaction. Such integrated cameras capture iris and partial periocular region during the interaction. This work investigates the possibility of using the captured ocular images from integrated cameras from HMD devices for biometric verification, taking into account the expected limited computational power of such devices. Such an approach can allow user to be verified in a manner that does not require any special and explicit user action. In addition to our comprehensive analyses, we present a light weight, yet accurate, segmentation solution for the ocular region captured from HMD devices. Further, we benchmark a number of well-established iris and periocular verification methods along with an in-depth analysis on the impact of iris sample selection and its effect on iris recognition performance for HMD devices. To the end, we also propose and validate an identity-preserving synthetic ocular image generation mechanism that can be used for large scale data generation for training purposes or attack generation purposes. We establish the realistic image quality of generated images with high fidelity and identity preserving capabilities through benchmarking them for iris and periocular verification.

Show publication details

Bortolato, Blaz; Ivanovska, Marija; Rot, Peter; Križaj, Janez; Terhörst, Philipp; Damer, Naser; Peer, Peter; Struc, Vitomir

Learning Privacy-Enhancing Face Representations through Feature Disentanglement

2020

15th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2020). Proceedings

International Conference on Automatic Face and Gesture Recognition (FG) <15, 2020, Buenos Aires, Argentina>

Convolutional Neural Networks (CNNs) are today the de-facto standard for extracting compact and discriminative face representations (templates) from images in automatic face recognition systems. Due to the characteristics of CNN models, the generated representations typically encode a multitude of information ranging from identity to soft-biometric attributes, such as age, gender or ethnicity. However, since these representations were computed for the purpose of identity recognition only, the soft-biometric information contained in the templates represents a serious privacy risk. To mitigate this problem, we present in this paper a privacy-enhancing approach capable of suppressing potentially sensitive soft-biometric information in face representations without significantly compromising identity information. Specifically, we introduce a Privacy-Enhancing Face-Representation learning Network (PFRNet) that disentangles identity from attribute information in face representations and consequently allows to efficiently suppress soft-biometrics in face templates. We demonstrate the feasibility of PFRNet on the problem of gender suppression and show through rigorous experiments on the CelebA, Labeled Faces in the Wild (LFW) and Adience datasets that the proposed disentanglement-based approach is highly effective and improves significantly on the existing state-of-the-art.

Show publication details

Terhörst, Philipp; Riehl, Kevin; Damer, Naser; Rot, Peter; Bortolato, Blaz; Kirchbuchner, Florian; Struc, Vitomir; Kuijper, Arjan

PE-MIU: A Training-Free Privacy-Enhancing Face Recognition Approach Based on Minimum Information Units

2020

IEEE Access

Research on soft-biometrics showed that privacy-sensitive information can be deduced from biometric data. Utilizing biometric templates only, information about a persons gender, age, ethnicity, sexual orientation, and health state can be deduced. For many applications, these templates are expected to be used for recognition purposes only. Thus, extracting this information raises major privacy issues. Previous work proposed two kinds of learning-based solutions for this problem. The first ones provide strong privacy-enhancements, but limited to pre-defined attributes. The second ones achieve more comprehensive but weaker privacy-improvements. In this work, we propose a Privacy-Enhancing face recognition approach based on Minimum Information Units (PE-MIU). PE-MIU, as we demonstrate in this work, is a privacy-enhancement approach for face recognition templates that achieves strong privacy-improvements and is not limited to pre-defined attributes. We exploit the structural differences between face recognition and facial attribute estimation by creating templates in a mixed representation of minimal information units. These representations contain pattern of privacy-sensitive attributes in a highly randomized form. Therefore, the estimation of these attributes becomes hard for function creep attacks. During verification, these units of a probe template are assigned to the units of a reference template by solving an optimal best-matching problem. This allows our approach to maintain a high recognition ability. The experiments are conducted on three publicly available datasets and with five state-of-the-art approaches. Moreover, we conduct the experiments simulating an attacker that knows and adapts to the systems privacy mechanism. The experiments demonstrate that PE-MIU is able to suppress privacy-sensitive information to a significantly higher degree than previous work in all investigated scenarios. At the same time, our solution is able to achieve a verification performance close to that of the unmodified recognition system. Unlike previous works, our approach offers a strong and comprehensive privacy-enhancement without the need of training.

Show publication details

Boutros, Fadi; Damer, Naser; Raja, Kiran; Ramachandra, Raghavendra; Kirchbuchner, Florian; Kuijper, Arjan

Periocular Biometrics in Head-Mounted Displays: A Sample Selection Approach for Better Recognition

2020

IWBF 2020. Proceedings

International Workshop on Biometrics and Forensics (IWBF) <8, 2020, online>

Virtual and augmented reality technologies are increasingly used in a wide range of applications. Such technologies employ a Head Mounted Display (HMD) that typicallyincludes an eye-facing camera and is used for eye tracking.As some of these applications require accessing or transmittinghighly sensitive private information, a trusted verification ofthe operator’s identity is needed. We investigate the use ofHMD-setup to perform verification of operator using periocularregion captured from inbuilt camera. However, the uncontrollednature of the periocular capture within the HMD results inimages with a high variation in relative eye location and eyeopening due to varied interactions. Therefore, we propose a newnormalization scheme to align the ocular images and then, a newreference sample selection protocol to achieve higher verificationaccuracy. The applicability of our proposed scheme is exemplifiedusing two handcrafted feature extraction methods and two deeplearning strategies.We conclude by stating the feasibility of sucha verification approach despite the uncontrolled nature of thecaptured ocular images, especially when proper alignment andsample selection strategy is employed.

978-1-7281-6232-4

Show publication details

Grimm, Niklas; Damer, Naser [1. Gutachten]; Kuijper, Arjan [2. Gutachten]

Poseninvariante Handerkennung mit generativer Bildkorrektur

2020

Darmstadt, TU, Bachelor Thesis, 2020

Auf Händen basierende biometrische Verfahren sind in weiten Bevölkerungsschichten akzeptiert und können kontaktlos angewendet werden. Bei diesen kontaktlosen Authentifizierungsverfahren sind variierende Handposen eines der größten Probleme. Diese Arbeit erforscht, ob es möglich ist, aus Bildern mit variierenden Handposen solche zu synthetisieren, die einer normalisierten Handpose entsprechen. Auf der Grundlage dieser normalisierten Handpose wäre dann ein besserer Vergleich mit dem Referenzbild möglich. Das Fehlen eines großen Datensatzes und die variierende Skalierung bei kontaktlos akquirierten Handbildern bringt viele Herausforderungen. Von diesen Herausforderungen motiviert beschreibt diese Arbeit mehrere Experimente mit einer begrenzten Menge an Trainingsdaten und variierenden Skalierungen der Eingabebilder, echt wirkende Bilder mit normalisierten Handposen, zu synthetisieren. Die synthetisierten Bilder werden in Verifikationsverfahren mit den Orginalbildern verglichen. Am Ende zeigten die Experimente, dass es nicht möglich ist, mit diesem Versuchsaufbau echt wirkende Handbilder mit normalisierten Posen zu synthetisieren.

Show publication details

Terhörst, Philipp; Huber, Marco; Damer, Naser; Rot, Peter; Kirchbuchner, Florian; Struc, Vitomir; Kuijper, Arjan

Privacy Evaluation Protocols for the Evaluation of Soft-Biometric Privacy-Enhancing Technologies

2020

BIOSIG 2020

Conference on Biometrics and Electronic Signatures (BIOSIG) <19, 2020, Online>

GI-Edition - Lecture Notes in Informatics (LNI), P-306

Biometric data includes privacy-sensitive information, such as soft-biometrics. Soft-biometric privacy enhancing technologies aim at limiting the possibility of deducing such information. Previous works proposed several solutions to this problem using several different evaluation processes, metrics, and attack scenarios. The absence of a standardized evaluation protocol makes a meaningful comparison of these solutions difficult. In this work, we propose privacy evaluation protocols (PEPs) for privacy-enhancing technologies (PETs) dealing with soft-biometric privacy. Our framework evaluates PETs in the most critical scenario of an attacker that knows and adapts to the systems privacy-mechanism. Moreover, our PEPs differentiate between PET of learning-based or training-free nature. To ensure that our protocol meets the highest standards in both cases, it is based on Kerckhoffs‘s principle of cryptography.

Show publication details

Fu, Biying; Damer, Naser; Kirchbuchner, Florian; Kuijper, Arjan

Sensing Technology for Human Activity Recognition: a Comprehensive Survey

2020

IEEE Access

Sensors are devices that quantify the physical aspects of the world around us. This ability is important to gain knowledge about human activities. Human Activity recognition plays an import role in people’s everyday life. In order to solve many human-centered problems, such as health-care, and individual assistance, the need to infer various simple to complex human activities is prominent. Therefore, having a well defined categorization of sensing technology is essential for the systematic design of human activity recognition systems. By extending the sensor categorization proposed by White, we survey the most prominent research works that utilize different sensing technologies for human activity recognition tasks. To the best of our knowledge, there is no thorough sensor-driven survey that considers all sensor categories in the domain of human activity recognition with respect to the sampled physical properties, including a detailed comparison across sensor categories. Thus, our contribution is to close this gap by providing an insight into the state-of-the-art developments. We identify the limitations with respect to the hardware and software characteristics of each sensor category and draw comparisons based on benchmark features retrieved from the research works introduced in this survey. Finally, we conclude with general remarks and provide future research directions for human activity recognition within the presented sensor categorization.

Show publication details

Terhörst, Philipp; Kolf, Jan Niklas; Damer, Naser; Kirchbuchner, Florian; Kuijper, Arjan

SER-FIQ: Unsupervised Estimation of Face Image Quality Based on Stochastic Embedding Robustness

2020

2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition

IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) <2020, virtual>

Face image quality is an important factor to enable high-performance face recognition systems. Face quality assessment aims at estimating the suitability of a face image for the purpose of recognition. Previous work proposed supervised solutions that require artificially or human labelled quality values. However, both labelling mechanisms are error prone as they do not rely on a clear definition of quality and may not know the best characteristics for the utilized face recognition system. Avoiding the use of inaccurate quality labels, we proposed a novel concept to measure face quality based on an arbitrary face recognition model. By determining the embedding variations generated from random subnetworks of a face model, the robustness of a sample representation and thus, its quality is estimated. The experiments are conducted in a cross-database evaluation setting on three publicly available databases. We compare our proposed solution on two face embeddings against six state-of-the-art approaches from academia and industry. The results show that our unsupervised solution outperforms all other approaches in the majority of the investigated scenarios. In contrast to previous works, the proposed solution shows a stable performance over all scenarios. Utilizing the deployed face recognition model for our face quality assessment methodology avoids the training phase completely and further outperforms all baseline approaches by a large margin. Our solution can be easily integrated into current face recognition systems, and can be modified to other tasks beyond face recognition.

Show publication details

Damer, Naser; Grebe, Jonas Henry; Chen, Cong; Boutros, Fadi; Kirchbuchner, Florian; Kuijper, Arjan

The Effect of Wearing a Mask on Face Recognition Performance: an Exploratory Study

2020

BIOSIG 2020

Conference on Biometrics and Electronic Signatures (BIOSIG) <19, 2020, Online>

GI-Edition - Lecture Notes in Informatics (LNI), P-306

Face recognition has become essential in our daily lives as a convenient and contactless method of accurate identity verification. Process such as identity verification at automatic border control gates or the secure login to electronic devices are increasingly dependant on such technologies. The recent COVID-19 pandemic have increased the value of hygienic and contactless identity verification. However, the pandemic led to the wide use of face masks, essential to keep the pandemic under control. The effect of wearing a mask on face recognition in a collaborative environment is currently sensitive yet understudied issue. We address that by presenting a specifically collected database containing three session, each with three different capture instructions, to simulate realistic use cases.We further study the effect of masked face probes on the behaviour of three top-performing face recognition systems, two academic solutions and one commercial off-the-shelf (COTS) system.

Show publication details

Damer, Naser; Zienert, Steffen; Wainakh, Yaza; Kirchbuchner, Florian; Kuijper, Arjan; Moseguí Saladié, Alexandra

A Multi-detector Solution Towards an Accurate and Generalized Detection of Face Morphing Attacks

2019

FUSION 2019

International Conference on Information Fusion (FUSION) <22, 2019, Ottawa, Canada>

Face morphing attack images are built to be verifiable to multiple identities. Associating such images to identity documents leads to building faulty identity links, causing vulnerabilities in security critical processes. Recent works have studied the face morphing attack detection performance over variations in morphing approaches, pointing out low generalization. This work introduces a multi-detector fusion solution that aims at gaining both, accuracy and generalization over different morphing types. This is performed by fusing classification scores produced by detectors trained on databases with variations in morphing type and image pairing protocols. This work develop and evaluate the proposed solution along with baseline solutions by building a database with three different pairing protocols and two different morphing approaches. This proposed solution successfully lead to decreasing the Bona Fide Presentation Classification Error Rate at 1.0% Attack Presentation Classification Error Rate from 15.7% and 3.0% of the best performing single detector to 2.7% and 0.0%, respectively on two face morphing techniques, pointing out a highly generalized performance.

Show publication details

Raja, Kiran; Damer, Naser; Ramachandra, Raghavendra; Boutros, Fadi; Busch, Christoph

Cross-Spectral Periocular Recognition by Cascaded Spectral Image Transformation

2019

2019 Conference Proceedings

IEEE International Conference on Imaging Systems and Techniques (IST 2019) <2019, Abu Dhabi, UAE>

Recent efforts in biometrics have focused on crossdomain face recognition where images from one domain are either transformed or synthesized. In this work, we focus on a similar problem for cross spectral periocular recognition where the images from Near Infra Red (NIR) domain are matched against Visible (VIS) spectrum images. Specifically, we propose to adapt a cascaded image transformation network that can produce NIR image given a VIS image. The proposed approach is first validated with regards to the quality of the image produced by employing various quality factors. Second the applicability is demonstrated with images generated by the proposed approach. We employ a publicly available cross-spectral periocular image data of 240 unique periocular instances captured in 8 different capture sessions. We experimentally validate that the proposed image transformation scheme can produce NIR like images and also can be used with any existing feature extraction scheme. To this extent, we demonstrate the biometric applicability by using both hand-crafted and deep neural network based features under verification setting. The obtained EER of 0.7% indicates the suitability of proposed approach for image transformation from the VIS to the NIR domain.

Show publication details

Mallat, Khawla; Damer, Naser; Boutros, Fadi; Kuijper, Arjan; Dugelay, Jean-Luc

Cross-spectrum thermal to visible face recognition based on cascaded image synthesis

2019

The 12th IAPR International Conference On Biometrics

IAPR International Conference on Biometrics (ICB) <12, 2019, Crete, Greece>

Face synthesis from thermal to visible spectrum is fundamental to perform cross-spectrum face recognition as it simplifies its integration in existing commercial face recognition systems and enables manual face verification. In this paper, a new solution based on cascaded refinement networks is proposed. This method generates visible-like colored images of high visual quality without requiring large amounts of training data. By employing a contextual loss function during training, the proposed network is inherently scale and rotation invariant. We discuss the visual perception of the generated visible-like faces in comparison with recent works. We also provide an objective evaluation in terms of cross-spectrum face recognition, where the generated faces were compared against a gallery in visible spectrum using two state-of-the-art deep learning based face recognition systems. When compared to the recently published TV-GAN solution, the performance of the face recognition systems, OpenFace and LightCNN, was improved by a 42.48% (i.e. from 10.76% to 15.37%) and a 71.43% (i.e. from 33.606% to 57.612%), respectively.

Show publication details

Damer, Naser; Boutros, Fadi; Kirchbuchner, Florian; Kuijper, Arjan

D-ID-Net: Two-Stage Domain and Identity Learning for Identity-Preserving Image Generation From Semantic Segmentation

2019

2019 International Conference on Computer Vision Workshops. Proceedings

International Conference on Computer Vision (ICCV) <17, 2019, Seoul, Korea>

Training functionality-demanding AR/VR systems require accurate and robust gaze estimation and tracking solutions. Achieving such a performance requires the availability of diverse eye image data that might only be acquired by the means of image generation. Works addressing the generation of such images did not target realistic and identity-specific images, nor did they address the practicalrelevant case of generation from semantic labels. Therefore, this work proposes a solution to generate realistic and identity-specific images that correspond to semantic labels, given samples of a specific identity. Our proposed solution consists of two stages. In the first stage, a network is trained to transform the semantic label into a corresponding eye image of a generic identity. The second stage is an identity-specific network that induces identity details on the generic eye image. The results of our D-ID-Net solutions shows a high degree of identity-preservation and similarity to the ground-truth images, with an RMSE of 7.235.

Show publication details

Damer, Naser; Boller, Viola; Wainakh, Yaza; Boutros, Fadi; Terhörst, Philipp; Braun, Andreas; Kuijper, Arjan

Detecting Face Morphing Attacks by Analyzing the Directed Distances of Facial Landmarks Shifts

2019

Pattern Recognition

German Conference on Pattern Recognition (GCPR) <40, 2018, Stuttgart, Germany>

Lecture Notes in Computer Science (LNCS), 11269

Face morphing attacks create face images that are verifiable to multiple identities. Associating such images to identity documents lead to building faulty identity links, causing attacks on operations like border crossing. Most of previously proposed morphing attack detection approaches directly classified features extracted from the investigated image. We discuss the operational opportunity of having a live face probe to support the morphing detection decision and propose a detection approach that take advantage of that. Our proposed solution considers the facial landmarks shifting patterns between reference and probe images. This is represented by the directed distances to avoid confusion with shifts caused by other variations. We validated our approach using a publicly available database, built on 549 identities. Our proposed detection concept is tested with three landmark detectors and proved to outperform the baseline concept based on handcrafted and transferable CNN features.

978-3-030-12938-5

Show publication details

Boutros, Fadi; Damer, Naser; Terhörst, Philipp; Kirchbuchner, Florian; Kuijper, Arjan

Exploring the Channels of Multiple Color Spaces for Age and Gender Estimation from Face Images

2019

FUSION 2019

International Conference on Information Fusion (FUSION) <22, 2019, Ottawa, Canada>

Soft biometrics identify certain traits of individuals based on their sampled biometric characteristics. The automatic identification of traits like age and gender provides valuable information in applications ranging from forensics to service personalization. Color images are stored within a color space containing different channels. Each channel represents a different portion of the information contained in the image, including these of soft biometric patterns. The analysis of the age and gender information in the different channels and different color spaces was not previously studied. This work discusses the soft biometric performances using these channels and analyzes the sample error overlap between all possible channels to successfully prove that different information is considered in the decision making from each channel. We also present a multi-channel selection protocols and fusion solution of the selected channels. Beside the analyzes of color spaces and their channels, our proposed multi-channel fusion solution extends beyond state-of-the-art performance in age estimation on the widely used Adience dataset.

Show publication details

Boutros, Fadi; Damer, Naser; Kirchbuchner, Florian; Kuijper, Arjan

Eye-MMS: Miniature Multi-Scale Segmentation Network of Key Eye-Regions in Embedded Applications

2019

2019 International Conference on Computer Vision Workshops. Proceedings

International Conference on Computer Vision (ICCV) <17, 2019, Seoul, Korea>

Segmentation of the iris or sclera is an essential processing block in ocular biometric systems. However, humancomputer interaction, as in VR/AR applications, requires multiple region segmentation to enable smoother interaction and eye-tracking. Such application does not only demand highly accurate and generalizable segmentation, it requires such segmentation model to be appropriate for the limited computational power of embedded systems. This puts strict limits on the size of the deployed deep learning models. This work presents a miniature multi-scale segmentation network consisting of inter-connected convolutional modules. We present a baseline multi-scale segmentation network and modify it to reduce its parameters by more than 80 times, while reducing its accuracy by less than 3%, resulting in our Eye-MMS model containing only 80k parameters. This work is developed on the OpenEDS database and is conducted in preparation for the OpenEDS Semantic Segmentation Challenge.

Show publication details

Wochner, Paul Frederik Franz Ludwig; Walther, Thomas [1. Review]; Damer, Naser [2. Review]; Terhörst, Philipp [3. Review]

How Do Demographic Soft-Biometric Attributes Affect Kinship Verification ?

2019

Darmstadt, TU, Bachelor Thesis, 2019

In recent years, facial kinship verification has received considerable attention due to the easy acquisition of facial images and a large potential application area. Facial kinship verification is defined as the process to determine whether two identities are kin or not by automatically comparing their facial images. Facial kinship verification may have a wide range of potential uses, including aiding in the fight against human trafficking, handling conflicts resulting from the refugee crisis, family album organization, and social media analysis. Other potential applications lie in the academic field, such as genealogical studies and in the identification of the kin of victims or suspects by law enforcement [1], [2]. In Germany, from March 1951 to April 2019, a total of 1995 cases of missing children are unresolved as reported by the Bundeskriminalamt [3]. Due to the significant change in the look of children at adult age, the high similarity of a child’s appearance to their parents, and the much easier acquisition of photos than DNA, facial kinship verification could help resolve these, and similar cases. Unfortunately, the performance of such kinship verification systems is still too underdeveloped to be used for real-world applications [4]. One issue consists of the non-generalizability of currently available data sets to the real-world data distribution [2]. Lopez et al. received an acceptable accuracy on two data sets by only comparing the chrominance [5]. Inspired by this, Dawson et al. built a “From Same Photo” classifier to compete for the kinship verification task by only assigning those pictures as kin which originated from the same photo [6]. As another trait, Guo et al. included gender and age as information in the kinship verification process by only considering this information to determine whether a person of the potential kin pair is older or the age is approximately the same [7]. Although age and ethnicity have not yet been explicitly implemented into the kinship verification process, the present thesis analyzes the impact of gender and these attributes on the kinship verification process. Accordingly, two widely used data sets were labeled manually with gender, age, and ethnicity. The impact of the addition of these traits to the baseline model was then analyzed. These additional traits could improve the accuracy of the kinship verification process slightly. Only a softbiometric classifier, including gender, age, and ethnicity, was built. A significant fraction of the kinship verification process may be explained solely by these attributes because of an inappropriate data set composition. Moreover, an incorrect construction in the two analyzed data sets can be found, which evoked majorly from the same pictures and the same identities in different folds. Understanding the shortcomings of previously conducted research can help future researchers improve their development of the kinship verification process.

Show publication details

Subasioglu, Meltem; Damer, Naser [Supervisor]; Kaschube, Matthias [Supervisor]

Humans vs Machines: A Comparison of Human and Machine Learning Performance in Inferring Professions from Facial Images

2019

Frankfurt am Main, Univ., Master Thesis, 2019

Convolutional neural networks have seen a rise in the computer vision community in the recent years, even surpassing human accuracies in face recognition tasks. However, research regarding different face classification and face clustering experiments, which do not focus on the discrimination of face images according to unique individuals in the dataset, are still very limited. Many psychological studies indicate that humans are capable in inferring leaders, status and competence from facial images. This work focused on the question whether humans are indeed capable in telling profession – given by the branch of the profession and the career status – from face images and whether current state-of-the-art machine learning approaches are able to do the same. Furthermore, the performance of humans and machine learning systems were compared to give insights in the underlying processes. The results indicate that both human and machine learning models are capable to infer professions from facial images with better than chance accuracies. Both humans and machine learning systems perform almost equally well in these tasks, however, performance differences in individual tasks indicate that humans and machine learning algorithms solve the same tasks while relying on different cues in the face images.

Show publication details

Boller, Viola; Pesavento, Marius [1. Gutachten]; Damer, Naser [2. Gutachten]

Investigating the Use of Deeply Calculated Flows and Dynamic Routed Network Capsules in Face Presentation Attack Detection

2019

Darmstadt, TU, Master Thesis, 2019

Identifying an individual based on their facial characteristics is convenient and is gaining more and more importance. Only a normal camera is needed for data capturing, the individual characteristic can’t be stolen or forgot like a password, and it’s a non-intrusive capturing method, which enables identification from distances. With increasing usage of face recognition systems, also the amount of sophisticated attacks increase. Therefore vulnerabilities of such a recognition system are used. Especially the attack directly at the sensor, by presenting fake biometrics at the sensor, can be performed with low efforts and costs, and without detailed knowledge about the biometric system. That’s why this kind of attacks have caused wide attention in the biometric community. The attack is called presentation attack and there are many detection algorithms developed to solve this issue. Most existing presentation attack detection algorithms work great, when deployed in the same environment as they are trained in. But different environmental conditions are still challenging to solve. This thesis deals with the presentation attack issue by developing an approach, which combines deep optical flows and a capsule neural network. Optical flows have already proven to be a good preprocessing step, because only motion information is kept and environmental conditions won’t have a big impact on it. Deep optical flows are produced by end-to-end trained convolutional neural networks for the optical flow calculation. Capsule neural networks train neurons in a more general way by forwarding information of found pattern’s poses in the network additionally to it’s probabilities. Forwarding is achieved by a so called routing-by-agreement method, which allows to take relationships between the found patterns into account. For presentation attack detection development not enough data is available to train a deep neural network from scratch. This could be overcome by using a capsule neural network for classification. A deep understanding can be learned, needing less data. The contribution of this thesis is the combination of deep optical flows and a capsule neural network, which hasn’t been tested so far. This is realized by stacking multiple optical flows over the time and downsample this information, thus that the capsule neural network is able to train a classification. This pipeline is structurally optimized by testing different optical flow sizes, data normalization and multiple class training. Also a new downsampling and time stacking method is developed to keep outliers in the reduced data and order a stack according to the contained data information. This has proven to help the network to classify. In the end, the behavior of the network on various attacks is analyzed. In this thesis, performance improvements are achieved by structure optimizations and a novel downsampling and time stacking method. The structural improvements increased the model performance up to 12% and the new downsampling and time stacking method leads to an average performance improvement of 7.56% for two databases. However, the achieved performance does not outperform recent state-of-the-art presentation attack detection methods and the analysis over different attacks reveals that the developed structure adapts weakly to changes in the attack performance or to new attacks. Nevertheless, stable results for different environmental conditions are achieved.

Show publication details

Terhörst, Philipp; Damer, Naser; Braun, Andreas; Kuijper, Arjan

Minutiae-Based Gender Estimation for Full and Partial Fingerprints of Arbitrary Size and Shape

2019

Computer Vision - ACCV 2018

Asian Conference on Computer Vision (ACCV) <14, 2018, Perth, Australia>

Lecture Notes in Computer Science (LNCS), 11361

Since fingerprints are one of the most widely deployed biometrics, accurate fingerprint gender estimation can positively affect several applications. For example, in criminal investigations, gender classification may significantly minimize the list of potential subjects. Previous work mainly offered solutions for the task of gender classification based on complete fingerprints. However, partial fingerprint captures are frequently occurring in many applications, including forensics and the fast growing field of consumer electronics. Due to its huge variability in size and shape, gender estimation on partial fingerprints is a challenging problem. Therefore, in this work we propose a flexible gender estimation scheme by building a gender classifier based on an ensemble of minutiae. The outputs of the single minutia gender predictions are combined by a novel adjusted score fusion approach to obtain an enhanced gender decision. Unlike classical solutions this allows to deal with unconstrained fingerprint parts of arbitrary size and shape. We performed investigations on a publicly available database and our proposed solution proved to significantly outperform state-of-the-art approaches on both full and partial fingerprints. The experiments indicate a reduction in the gender estimation error by 19.34% on full fingerprints and 28.33% on partial captures in comparison to previous work.

978-3-030-20886-8

Show publication details

Terhörst, Philipp; Huber, Marco; Kolf, Jan Niklas; Damer, Naser; Kirchbuchner, Florian; Kuijper, Arjan

Multi-algorithmic Fusion for Reliable Age and Gender Estimation from Face Images

2019

FUSION 2019

International Conference on Information Fusion (FUSION) <22, 2019, Ottawa, Canada>

Automated estimation of demographic attributes, such as gender and age, became of great importance for many potential applications ranging from forensics to social media. Although previous works reported performances that closely match human level. These solutions lack of human intuition that allows human beings to state the confidences of their predictions. While the human intuition subconsciously considers surrounding conditions or the lack of experience in a certain task, current algorithmic solutions tend to mispredict with high confidence scores. In this work, we propose a multi-algorithmic fusion approach for age and gender estimation that is able to accurately state the model’s prediction reliability. Our solution is based on stochastic forward passes through a dropout-reduced neural network ensemble. By utilizing multiple stochastic forward passes combined from the neural network ensemble, the centrality and dispersion of these predictions are used to derive a confidence statement about the prediction. Our experiments were conducted on the Adience benchmark.We showed that the proposed solution reached and exceeded state-of-the-art performance for the age and gender estimation tasks. Further, we demonstrated that the reliability statements of the predictions of our proposed solution capture challenging conditions and underrepresented training samples.

Show publication details

Damer, Naser; Dimitrov, Kristiyan; Braun, Andreas; Kuijper, Arjan

On Learning Joint Multi-biometric Representations by Deep Fusion

2019

IEEE 10th International Conference on Biometrics: Theory, Applications and Systems

IEEE International Conference on Biometrics: Theory, Applications and Systems (BTAS) <10, 2019, Tampa, Florida, USA>

Multi-biometrics combines different biometric sources to enhance recognition, template protection, and indexing performances. One of the main challenges here is the need for joint discriminant feature representation of multi-biometric data. This is typically achieved by feature-level fusion, imposing limitations on the combinations of biometric characteristics and algorithms. Including multiple imaging sources within deep-learning networks was generally limited to multiple sources of images of the same physical object, e.g., multi-spectral object detection. Previous biometrics works were limited to use deep-learning to extract representations of single biometric characteristics. In contrast to that, our work studies creating representations of one identity by sampling different physical objects, i.e. biometric characteristics. We adapted three architectures successfully to produce and discuss jointly learned representations for different levels of correlated data, modalities, instances, and presentations. Our evaluation proved the applicability of jointly learning biometric representations, especially when the data correlation is low.

Show publication details

Debiasi, Luca; Damer, Naser; Moseguí Saladié, Alexandra; Rathgeb, Christian; Scherhag, Ulrich; Busch, Christoph; Kirchbuchner, Florian; Uhl, Andreas

On the Detection of GAN-Based Face Morphs Using Established Morph Detectors

2019

Image Analysis and Processing - ICIAP 2019

International Conference of Image Analysis and Processing (ICIAP) <20, 2019, Trento, Italy>

Lecture Notes in Computer Science (LNCS), 11752

Face recognition systems (FRS) have been found to be highly vulnerable to face morphing attacks. Due to this severe security risk, morph detection systems do not only need to be robust against classical landmark-based face morphing approach (LMA), but also future attacks such as neural network based morph generation techniques. The focus of this paper lies on an experimental evaluation of the morph detection capabilities of various state-of-the-art morph detectors with respect to a recently presented novel face morphing approach, MorGAN, which is based on Generative Adversarial Networks (GANs). In this work, existing detection algorithms are confronted with different attack scenarios: known and unknown attacks comprising different morph types (LMA and MorGAN). The detectors’ performance results are highly dependent on the features used by the detection algorithms. In addition, the image quality of the morphed face images produced with the MorGAN approach is assessed using well-established no-reference image quality metrics and compared to LMA morphs. The results indicate that the image quality of MorGAN morphs is more similar to bona fide images compared to classical LMA morphs.

978-3-030-30644-1

Show publication details

Damer, Naser; Grebe, Jonas Henry; Zienert, Steffen; Kirchbuchner, Florian; Kuijper, Arjan

On the Generalization of Detecting Face Morphing Attacks as Anomalies: Novelty vs. Outlier Detection

2019

IEEE 10th International Conference on Biometrics: Theory, Applications and Systems

IEEE International Conference on Biometrics: Theory, Applications and Systems (BTAS) <10, 2019, Tampa, Florida, USA>

Face morphing attacks are verifiable to multiple identities, leading to faulty identity links. Recent works have studied the face morphing attack detection performance over variations in morphing approaches, pointing out low generalization. This work studies detecting these attacks as anomalies and discusses the performance and generalization over different morphing types. We also analyze the accuracy and generalization effect of including different amounts of attack contamination in the anomaly training data (novelty vs. outlier). This is performed with two baseline 2-class classifiers, two approaches for anomaly detection, two image feature extractions, two morphing types, and variations in contamination levels and tolerated training errors. The results points out the relative lower performance, but higher generalization ability, of anomaly detection in comparison to 2-class classifiers, along with the benefits of contaminating the anomaly training data.

Show publication details

Damer, Naser; Boutros, Fadi; Saladie, Alexandra Moseguí; Kirchbuchner, Florian; Kuijper, Arjan

Realistic Dreams: Cascaded Enhancement of GAN-generated Imageswith an Example in Face Morphing Attacks

2019

IEEE 10th International Conference on Biometrics: Theory, Applications and Systems

IEEE International Conference on Biometrics: Theory, Applications and Systems (BTAS) <10, 2019, Tampa, Florida, USA>

The quality of images produced by generative adversarial networks (GAN) is commonly a trade-off between the model size, its training data needs, and the generation resolution. This trad-off is clear when applying GANs to issues like generating face morphing attacks, where the latent vector used by the generator is manipulated. In this paper, we propose an image enhancement solution designed to increase the quality and resolution of GAN-generated images. The solution is designed to require limited training data and be extendable to higher resolutions. We successfully apply our solution on GAN-based face morphing attacks. Beside the face recognition vulnerability and attack detectability analysis, we prove that the images enhanced by our solution are of higher visual and quantitative quality in comparison to unprocessed attacks and attack images enhanced by state-of-the-art super-resolution approaches.

Show publication details

Boutros, Fadi; Kuijper, Arjan [1. Gutachten]; Damer, Naser [2. Gutachten]

Reducing Ethnic Bias of Face Recognition by Ethnic Augmentation

2019

Darmstadt, TU, Master Thesis, 2019

Automated face recognition has gained wider deployment ground after the recent accuracy gains achieved by deep learning techniques. Despite the rapid performance gains, face recognition still suffers from very critical issues. One of the recently uncovered, and very sensitive, challenges is the ethnicity bias in face recognition decision. This is the case, unfortunately, even in the latest commercial and academic face recognition system. In 2018, the National Institute of Standards and Technology (NIST) published the latest report regarding the evaluation result of commercial face recognition solutions from several major biometric vendors. The report specifically evaluated and demonstrated the variance of the error rates of the evaluated solutions based on demographic variations. This thesis is one of the first research efforts in addressing the decision bias challenge in biometrics. It builds its hypothesis on the strong assumption that ethnicity bias is caused by the relative under-representation of certain ethnicities in training databases. This work introduces a novel ethnicity-driven data augmentation approach to reduce biometric bias. The proposed approach successfully utilize a generative image model to generate new face images that preserve the identity of their source images while partially transforming their ethnicity to the targeted ethnicity group. A large-scale ethnicity-labeled face images database is created in order to develop and evaluate the proposed approach. To achieve that, part of this thesis focused on creating an ethnicity classifier to annotate face images, achieving accuracy in the state-of-the-art range. The proposed ethnicity-driven face generative model is developed based on the ethnicity labeled images to generate realistically and high-resolution face images, depending on a limit amount of training data. More importantly, the thesis proves that the proposed augmentation approach strongly preserves the identity of the input images and partially transforms the ethnicity. The augmented images are used as part of the training data of a face recognition model. The achieved verification results prove that the proposed ethnicity augmentation methods significantly and consistently reduced the ethnicity bias of the face recognition model. For examples, the ERR was reduced from 0.159 to 0.130 when verifying inter-ethnicity samples of Black individuals on a model trained on Asian individual images, respectively before and after applying the proposed training data augmentation. Moreover, the overall performance of the face recognition model was improved. However, this improvement was more significant, as intended, in the targeted ethnicity groups.

Show publication details

Terhörst, Philipp; Huber, Marco; Kolf, Jan Niklas; Zelch, Ines; Damer, Naser; Kirchbuchner, Florian; Kuijper, Arjan

Reliable Age and Gender Estimation from Face Images: Stating the Confidence of Model Predictions

2019

IEEE 10th International Conference on Biometrics: Theory, Applications and Systems

IEEE International Conference on Biometrics: Theory, Applications and Systems (BTAS) <10, 2019, Tampa, Florida, USA>

Automated age and gender estimation became of great importance for many potential applications ranging from forensics to social media. Although previous works reported high increased performances, these solutions tend to mispredict under challenging conditions or when the trained model faces a sample that was underrepresented in the training data. In this work, we propose an age and gender estimation model, as well as a novel reliability measure to quantify the confidence of the model’s prediction. Our solution is based on stochastic forward passes through dropout-reduced neural networks that were theoretically proven to approximate Gaussian processes. By utilizing multiple stochastic forward passes, the centrality and dispersion of these predictions are used to derive a confidence statement about the prediction. Experiments were conducted on the Adience benchmark. We showed that the proposed solution reached and exceeded state-ofthe-art performance. Further, we demonstrated that the proposed reliability measure correlates with the prediction performance and thus, is highly successful in quantifying the prediction reliability.

Show publication details

Mallat, Khawla; Damer, Naser; Boutros, Fadi; Dugelay, Jean-Luc

Robust Face Authentication Based on Dynamic Quality-weighted Comparison of Visible and Thermal-to-visible images to Visible Enrollments

2019

FUSION 2019

International Conference on Information Fusion (FUSION) <22, 2019, Ottawa, Canada>

We introduce, in this paper, a new scheme of score level fusion for face authentication from visible and thermal face data. This proposed scheme provides a fast and straightforward integration into existing face recognition systems and does not require recollection of enrollment data in thermal spectrum. In addition to be used as a possible countermeasure against spoofing, this paper investigates the potential role of thermal spectrum in improving face recognition performances when employed under adversarial acquisition conditions. We consider a context where individuals have been enrolled solely in visible spectrum, and their identity will be verified using 2 sets of probes: visible and thermal. We show that the optimal way to proceed is to synthesis a visible image from the thermal face in order to create a synthetic-visible probe; and then to fuse scores resulting from comparisons between visible gallery with both visible probe and synthetic-visible probe. The thermal-to-visible face synthesis is performed using a Cascaded Refinement Network (CRN) and face features were extracted and matched using LightCNN and Local Binary Patterns (LBP). The fusion procedure is performed based on several quality measures computed on both visible and thermal-to-visible generated probes and compared to the visible gallery images.

Show publication details

Terhörst, Philipp; Damer, Naser; Kirchbuchner, Florian; Kuijper, Arjan

Suppressing Gender and Age in Face Templates Using Incremental Variable Elimination

2019

The 12th IAPR International Conference On Biometrics

IAPR International Conference on Biometrics (ICB) <12, 2019, Crete, Greece>

Recent research on soft-biometrics showed that more information than just the person’s identity can be deduced from biometric data. Using face templates only, information about gender, age, ethnicity, health state of the person, and even the sexual orientation can be automatically obtained. Since for most applications these templates are expected to be used for recognition purposes only, this raises major privacy issues. Previous work addressed this problem purely on image level regarding function creep attackers without knowledge about the systems privacy mechanism. In this work, we propose a soft-biometric privacy enhancing approach that reduces a given biometric template by eliminating its most important variables for predicting soft-biometric attributes. Training a decision tree ensemble allows deriving a variable importance measure that is used to incrementally eliminate variables that allow predicting sensitive attributes. Unlike previous work, we consider a scenario of function creep attackers with explicit knowledge about the privacy mechanism and evaluated our approach on a publicly available database. The experiments were conducted to eight baseline solutions. The results showed that in many cases IVE is able to suppress gender and age to a high degree with a negligible loss of the templates recognition ability. Contrary to previous work, which is limited to the suppression of binary (gender) attributes, IVE is able, by design, to suppress binary, categorical, and continuous attributes.

Show publication details

Bartuzi, Ewelina; Damer, Naser

Thermal and Cross-spectral Palm Image Matching in the Visual Domain by Robust Image Transformation

2019

The 12th IAPR International Conference On Biometrics

IAPR International Conference on Biometrics (ICB) <12, 2019, Crete, Greece>

Synthesizing visual-like images from those captured in the thermal spectrum allows for direct cross-domain comparisons. Moreover, it enables thermal-to-thermal comparisons that take advantage of feature extraction methodologies developed for the visual domain. Hand based biometrics are socially accepted and can operate in a touchless mode. However, certain deployment scenarios requires captures in non-visual spectrums due to impractical illumination requirements. Generating visual-like palm images from thermal ones faces challenges related to the nature of hand biometrics. Such challenges are the dynamic nature of the hand and the difficulties in accurately aligning hand’s scale and rotation, especially in the understudied thermal domain. Building such a synthetic solution is also challenged by the lack of large-scale databases that contain images collected in both spectra, as well as generating images of appropriate resolutions. Driven by these challenges, this paper presents a novel solution to transfer thermal palm images into high-quality visual-like images, regardless of the limited training data, or scale and rotational variations. We proved quality similarity and high correlation of the generated images to the original visual images. We used the synthesized images within verification approaches based on CNN and hand crafted-features. This allowed significantly improved the cross-spectral and thermal-to-thermal verification performances, reducing the EER from 37.12% to 16.25% and from 3.04% to 1.65%, respectively in both cases when using CNN-based features.

Show publication details

Damer, Naser; Saladie, Alexandra Moseguí; Zienert, Steffen; Wainakh, Yaza; Kirchbuchner, Florian; Kuijper, Arjan; Terhörst, Philipp

To Detect or not to Detect: The Right Faces to Morph

2019

The 12th IAPR International Conference On Biometrics

IAPR International Conference on Biometrics (ICB) <12, 2019, Crete, Greece>

Recent works have studied the face morphing attack detection performance generalization over variations in morphing approaches, image re-digitization, and image source variations. However, these works assumed a constant approach for selecting the images to be morphed (pairing) across their training and testing data. A realistic variation in the pairing protocol in the training data can result in challenges and opportunities for a stable attack detector. This work extensively study this issue by building a novel database with three different pairing protocols and two different morphing approaches. We study the detection generalization over these variations for single image and differential attack detection, along with handcrafted and CNNbased features. Our observations included that training an attack detection solution on attacks created from dissimilar face images, in contrary to the common practice, can result in an overall more generalized detection performance. Moreover, we found that differential attack detection is very sensitive to variations in morphing and pairing protocols.

Show publication details

Terhörst, Philipp; Damer, Naser; Kirchbuchner, Florian; Kuijper, Arjan

Unsupervised Privacy-enhancement of Face Representations Using Similarity-sensitive Noise Transformations

2019

Applied Intelligence

Face images processed by a biometric system are expected to be used for recognition purposes only. However, recent work presented possibilities for automatically deducing additional information about an individual from their face data. By using soft-biometric estimators, information about gender, age, ethnicity, sexual orientation or the health state of a person can be obtained. This raises a major privacy issue. Previous works presented supervised solutions that require large amount of private data in order to suppress a single attribute. In this work, we propose a privacy-preserving solution that does not require these sensitive information and thus, works in an unsupervised manner. Further, our approach offers privacy protection that is not limited to a single known binary attribute or classifier. We do that by proposing similarity-sensitive noise transformations and investigate their effect and the effect of dimensionality reduction methods on the task of privacy preservation. Experiments are done on a publicly available database and contain analyses of the recognition performance, as well as investigations of the estimation performance of the binary attribute of gender and the continuous attribute of age. We further investigated the estimation performance of these attributes when the prior knowledge about the used privacy mechanism is explicitly utilized. The results show that using this information leads to significantly enhancement of the estimation quality. Finally, we proposed a metric to evaluate the trade-off between the privacy gain and the recognition loss for privacy-preservation techniques. Our experiments showed that the proposed cosine-sensitive noise transformation was successful in reducing the possibility of estimating the soft private information in the data, while having significantly smaller effect on the intended recognition performance.

Show publication details

Damer, Naser; Kuijper, Arjan [Referent]; Fellner, Dieter W. [Referent]; Ramachandra, Raghavendra [Referent]

Application-driven Advances in Multi-biometric Fusion

2018

Darmstadt, TU, Diss., 2018

Biometric recognition is the automated recognition of individuals based on their behavioral or biological characteristics. Beside forensic applications, this technology aims at replacing the outdated and attack prone, physical and knowledge-based, proofs of identity. Choosing one biometric characteristic is a tradeoff between universality, acceptability, and permanence, among other factors. Moreover, the accuracy cap of the chosen characteristic may limit the scalability and usability for some applications. The use of multiple biometric sources within a unified frame, i.e. multi-biometrics, aspires to tackle the limitations of single source biometrics and thus enables a wider implementation of the technology. This work aims at presenting application-driven advances in multi-biometrics by addressing different elements of the multi-biometric system work-flow. At first, practical oriented pre-fusion issues regarding missing data imputation and score normalization are discussed. This includes presenting a novel performance anchored score normalization technique that aligns certain performance-related score values in the fused biometric sources leading to more accurate multi-biometric decisions when compared to conventional normalization approaches. Missing data imputation within scorelevel multi-biometric fusion is also addressed by analyzing the behavior of different approaches under different operational scenarios. Within the multi-biometric fusion process, different information sources can have different degrees of reliability. This is usually influenced in the fusion process by assigning relative weights to the fused sources. This work presents a number of weighting approaches aiming at optimizing the decision made by the multi-biometric system. First, weights that try to capture the overall performance of the biometric source, as well as an indication of its confidence, are proposed and proved to outperform the state-of-the-art weighting approaches. The work also introduces a set of weights derived from the identification performance representation, the cumulative match characteristics. The effect of these weights is analyzed under the verification and identification scenarios. To further optimize the multi-biometric process, information besides the similarity between two biometric captures can be considered. Previously, the quality measures of biometric captures were successfully integrated, which requires accessing and processing raw captures. In this work, supplementary information that can be reasoned from the comparison scores are in focus. First, the relative relation between different biometric comparisons is discussed and integrated in the fusion process resulting in a large reduction in the error rates. Secondly, the coherence between scores of multi-biometric sources in the same comparison is defined and integrated into the fusion process leading to a reduction in the error rates, especially when processing noisy data. Large-scale biometric deployments are faced by the huge computational costs of running biometric searches and duplicate enrollment checks. Data indexing can limit the search domain leading to faster searches. Multibiometrics provides richer information that can enhance the retrieval performance. This work provides an optimizable and configurable multi-biometric data retrieval solution that combines and enhances the robustness of rank-level solutions and the performance of feature-level solutions. Furthermore, this work presents biometric solutions that complement and utilize multi-biometric fusion. The first solution captures behavioral and physical biometric characteristics to assure a continuous user authentication. Later, the practical use of presentation attack detection is discussed by investigating the more realistic scenario of cross-database evaluation and presenting a state-of-the-art performance comparison. Finally, the use of multibiometric fusion to create face references from videos is addressed. Face selection, feature-level fusion, and score-level fusion approaches are evaluated under the scenario of face recognition in videos.

Show publication details

Damer, Naser; Wainakh, Yaza; Boller, Viola; Berken von den, Sven; Terhörst, Philipp; Braun, Andreas; Kuijper, Arjan

CrazyFaces: Unassisted Circumvention of Watchlist Face Identification

2018

IEEE 9th International Conference on Biometrics: Theory, Applications and Systems

IEEE International Conference on Biometrics: Theory, Applications and Systems (BTAS) <9, 2018, Redondo Beach, CA, USA>

Once upon a time, there was a blacklisted criminal who usually avoided appearing in public. He was surfing the Web, when he noticed, what had to be a targeted advertisement announcing a concert of his favorite band. The concert was in a near town, and the only way to get there was by train. He was worried, because he heard in the news about the new face identification system installed at the train station. From his last stay with the police, he remembers that they took these special face images with the white background. He thought about what can he do to avoid being identified and an idea popped in his mind “what if I can make a crazy-face, as the kids call it, to make my face look different? What do I exactly have to do? And will it work?”. He called his childhood geeky friend and asked him if he can build him a face recognition application he can tinker with. The geeky friend was always interested in such small projects where he can use open-source resources and didn’t really care about the goal, as usual. The criminal tested the application and played around, trying to figure out how can he make a crazy-face that won’t be identified as himself. On the day of the concert, he took off to the train station with some doubt in his mind and fear in his soul. To know what happened next, you should read the rest of this paper.

978-1-5386-7180-1

Show publication details

Moseguí Saladié, Alexandra; Yildirim Yayilgan, Sule [Supervisor]; Damer, Naser [Supervisor]

Creating Face Morphing Attacks with Generative Adversarial Networks

2018

Saint-Étienne, Univ., Master Thesis, 2018

Nowadays, the use of technologies related to biometrics is increasing significantly. The recent deep learning improvement in face recognition tasks, along with the relatively high social acceptance, have pushed automatic face recognition systems to be a key technology in identity verification in border controls. In this scenario, face recognition is used to link the identity of a passenger to their e-document. However, recent studies have highlighted the threat of morphing attacks against automatic face recognition systems. In this thesis, we present a novel morphed face attack, called MorGANA, created by using Generative Adversarial Networks. By creating a new morphing database, MorGAN dataset, we investigate the vulnerabilities of current face recognition systems against MorGANA attacks, alongside with baseline attacks. Moreover, the morphing detectability under face morphing attacks is further studied noticing an insufficient performance from common detectors. Examining the obtained results, we strongly consider that further studies need to be addressed in generating and detecting morphed face attacks in order to bring face recognition systems to real-case scenarios.

Show publication details

Anvekar, Roshan; Damer, Naser [Supervisor]; Pech, Andreas [Supervisor]; Nauth, Peter [Supervisor]

Cross-device Biometric Verification and the Benefit of Multi-algorithmic Biometric Fusion

2018

Frankfurt am Main, Univ. of Applied Science, Master Thesis, 2018

Biometrics is a technology that aims to identify or verify people identities based on their physical characteristics or behavioral properties. Multi-biometrics is implemented to use a number of biometric information sources to create a unified decision to increase the accuracy of the biometric system. This aims at enhancing performance and avoiding the shortcomings of conventional single source biometrics such as sensitivity to noisy data or capture environment while maintaining high accuracy. One of these shortcomings are the negative effect of cross-device biometric verification, e.g. using different smart phones and capture devices for the enrolment and verification of face images. The previous work deals with multi-biometrics to obtain the fused score by incorporating the coherence information on the biometric sources to enhance the performance of the Biometric system. In this thesis the main purpose is to improve the performance of the Biometric system for multi-modality (two modalities) with each biometric source acquired from different sensor devices which includes both same sensor and different sensor combinations. This includes biometric sources, i.e. face and one of the periocular region fed to the biometric system with multi-algorithmic approach to extract the features and create unified decision at the score level. Analysis of different realistic scenario with varying static and dynamic weights and its effect on the unified decision on score level. This approach could be used in biometric system with more than two modalities.

Show publication details

Terhörst, Philipp; Damer, Naser; Braun, Andreas; Kuijper, Arjan

Deep and Multi-algorithmic Gender Classification of Single Fingerprint Minutiae

2018

FUSION 2018

International Conference on Information Fusion (FUSION) <21, 2018, Cambridge, UK>

Accurate fingerprint gender estimation can positively affect several applications, since fingerprints are one of the most widely deployed biometrics. For example, gender classification in criminal investigations may significantly minimize the list of potential subjects. Previous work mainly offered solutions for the task of gender classification based on complete fingerprints. However, partial fingerprint captures are frequently occurring in many applications, including forensics and the fast growing field of consumer electronics. Moreover, partial fingerprints are not well-defined. Therefore, this work improves the gender decision performance on a well-defined partition of the fingerprint. It enhances gender estimation on the level of a single minutia. Working on this level, we propose three main contributions that were evaluated on a publicly available database. First, a convolutional neural network model is offered that outperformed baseline solutions based on hand crafted features. Second, several multi-algorithmic fusion approaches were tested by combining the outputs of different gender estimators that help further increase the classification accuracy. Third, we propose including minutia detection reliability in the fusion process, which leads to enhancing the total gender decision performance. The achieved gender classification performance of a single minutia is comparable to the accuracy that previous work reported on a quarter of aligned fingerprints including more than 25 minutiae.

978-0-9964527-6-2

Show publication details

Damer, Naser; Wainakh, Yaza; Henniger, Olaf; Croll, Christian; Berthe, Benoit; Braun, Andreas; Kuijper, Arjan

Deep Learning-based Face Recognition and the Robustness to Perspective Distortion

2018

24th International Conference on Pattern Recognition. Proceedings

International Conference on Pattern Recognition (ICPR) <24, 2018, Beijing, China>

Face recognition technology is spreading into a wide range of applications. This is mainly driven by social acceptance and the performance boost achieved by the deep learningbased solutions in the recent years. Perspective distortion is an understudied distortion in face recognition that causes converging verticals when imaging 3D objects depending on the distance to the object. The effect of this distortion on face recognition was previously studied for algorithms based on hand-crafted features with a clear negative effect on verification performance. Possible solutions were proposed by compensating the distortion effect on the face image level, which requires knowing the camera settings and capturing a high quality image. This work investigates the effect of perspective distortion on the performance of a deep learning-based face recognition solution. It also provides a device parameter-independent solution to decrease this effect by creating more perspective-robust face representations. This was achieved by training the deep learning model on perspective-diverse data, without increasing the size of the training data. Experiments performed on the deep model in hand and a specifically collected database concluded that the perspective distortion effects face verification performance if not considered in the training process, and that this can be improved by our proposal of creating robust face representations by properly selecting the training data.

978-1-5386-3787-6

Show publication details

Damer, Naser; Terhörst, Philipp; Braun, Andreas; Kuijper, Arjan

Fingerprint and Iris Multi-biometric Data Indexing and Retrieval

2018

FUSION 2018

International Conference on Information Fusion (FUSION) <21, 2018, Cambridge, UK>

Indexing of multi-biometric data is required to facilitatefast search in large-scale biometric systems. Previous worksaddressing this issue in multi-biometric databases focused onmulti-instance indexing, mainly iris data. Few works addressedthe indexing in multi-modal databases, with basic candidate listfusion solutions limited to joining face and fingerprint data. Irisand fingerprint are widely used in large-scale biometric systemswhere fast retrieval is a significant issue. This work proposes jointmulti-biometric retrieval solution based on fingerprint and irisdata. This solution is evaluated under eight different candidatelist fusion approaches with variable complexity on a databaseof 10,000 reference and probe records of irises and fingerprints.Our proposed multi-biometric retrieval of fingerprint and irisdata resulted in a reduction of the miss rate (1- hit rate) at 0.1%penetration rate by 93% compared to fingerprint indexing and88% compared to iris indexing.

Show publication details

Damer, Naser; Moseguí Saladié, Alexandra; Braun, Andreas; Kuijper, Arjan

MorGAN: Recognition Vulnerability and Attack Detectability of Face Morphing Attacks Created by Generative Adversarial Network

2018

IEEE 9th International Conference on Biometrics: Theory, Applications and Systems

IEEE International Conference on Biometrics: Theory, Applications and Systems (BTAS) <9, 2018, Redondo Beach, CA, USA>

Face morphing attacks aim at creating face images that are verifiable to be the face of multiple identities, which can lead to building faulty identity links in operations like border crossing. Research has been focused on creating more accurate attack detection approaches by considering different image properties. However, all the attacks considered so far are based on manipulating facial landmarks localized in the morphed face images. In contrast, this work presents novel face morphing attacks based on image generated by generative adversarial networks. We present the MorGAN structure that considers the representation loss to successfully create realistic morphing attacks. Based on that, we present a novel face morphing attacks database (MorGAN database) that contains 1000 morph images for both, the proposed MorGAN and landmark-based attacks. We present vulnerability analysis of two face recognition approaches facing the proposed attacks. Moreover, the detectability of the proposed MorGAN attacks is studied, in the scenarios where this type of attacks is know and unknown. We concluded with pointing out the challenge of detecting such unknown novel attacks and an analysis of detection performances of different features in detecting such attacks.

978-1-5386-7180-1

Show publication details

Damer, Naser; Boutros, Fadi; Terhörst, Philipp; Braun, Andreas; Kuijper, Arjan

P-score: Performance Aligned Normalization and an Evaluation in Score-level Multi-biometric Fusion

2018

2018 Proceedings of the 26th European Signal Processing Conference (EUSIPCO)

European Signal Processing Conference (EUSIPCO) <26, 2018, Rome, Italy>

Normalization is an important step for different fusion, classification, and decision making applications. Previous normalization approaches considered bringing values from different sources into a common range or distribution characteristics. In this work we propose a new normalization approach that transfers values into a normalized space where their relative performance in binary decision making is aligned across their whole range. Multi-biometric verification is a typical problem where information from different sources are normalized and fused to make a binary decision and therefore a good platform to evaluate the proposed normalization.We conducted an evaluation on two publicly available databases and showed that the normalization solution we are proposing consistently outperformed state-of-the-art and best practice approaches, e.g. by reducing the false rejection rate at 0.01% false acceptance rate by 60- 75% compared to the widely used z-score normalization under the sum-rule fusion.

978-90-827970-1-5

Show publication details

Samartzidis, Timotheos; Siegmund, Dirk; Gödde, Michael; Damer, Naser; Braun, Andreas; Kuijper, Arjan

The Dark Side of the Face: Exploring the Ultraviolet Spectrum for Face Biometrics

2018

2018 International Conference on Biometrics (ICB)

IAPR International Conference on Biometrics (ICB) <11, 2018, Gold Coast, Australia>

Facial recognition in the visible spectrum is a widelyused application but it is also still a major field of research.In this paper we present melanin face pigmentation (MFP)as a new modality to be used to extend classical face biometrics. Melanin pigmentation are sun-damaged cells thatoccur as revealed and/or unrevealed pattern on human skin.Most MFP can be found in the faces of some people whenusing ultraviolet (UV) imaging. To proof the relevance ofthis feature for biometrics, we present a novel image datasetof 91 multiethnic subjects in both, the visible and the UVspectrum. We show a method to extract the MFP featuresfrom the UV images, using the well known SURF featuresand compare it with other techniques. In order to proof itsbenefits, we use weighted score-level fusion and evaluatethe performance in an one against all comparison. As a resultwe observed a significant amplification of performancewhere traditional face recognition in the visible spectrum isextended with MFP from UV images. We conclude with afuture perspective about the use of these features for futureresearch and discuss observed issues and limitations.

Show publication details

Terhörst, Philipp; Damer, Naser; Braun, Andreas; Kuijper, Arjan

What Can a Single Minutia Tell about Gender?

2018

2018 International Workshop on Biometrics and Forensics (IWBF)

International Workshop on Biometrics and Forensics (IWBF) <2018, Sassari, Italy>

Since fingerprints are one of the most widely deployed biometrics, several applications can benefit from an accurate fingerprint gender estimation. Previous work mainly tackled the task of gender estimation based on complete fingerprints. However, partial fingerprint captures are frequently occurring in many applications including forensics and consumer electronics, with the considered ratio of the fingerprint is variable. Therefore, this work investigates gender estimation on a small, detectable, and well-defined partition of a fingerprint. It investigates gender estimation on the level of a single minutia. Working on this level, we propose a feature extraction process that is able to deal with the rotation and translation invariance problems of fingerprints. This is evaluated on a publicly available database and with five different binary classifiers. As a result, the information of a single minutia achieves a comparable accuracy on the gender classification task as previous work using quarters of aligned fingerprints with an average of more than 25 minutiae.

Show publication details

Damer, Naser; Terhörst, Philipp; Braun, Andreas; Kuijper, Arjan

Efficient, Accurate, and Rotation-Invariant Iris Code

2017

IEEE Signal Processing Letters

The large scale of the recently demanded biometric systems has put a pressure on creating a more efficient, accurate, and private biometric solutions. Iris biometrics is one of the most distinctive and widely used biometric characteristics. High-performing iris representations suffer from the curse of rotation inconsistency. This is usually solved by assuming a range of rotational errors and performing a number of comparisons over this range, which results in a high computational effort and limits indexing and template protection. This work presents a generic and parameter-free transformation of binary iris representation into a rotation-invariant space. The goal is to perform accurate and efficient comparison and enable further indexing and template protection deployment. The proposed approach was tested on a database of 10 000 subjects of the ISYN1 iris database generated by CASIA. Besides providing a compact and rotational-invariant representation, the proposed approach reduced the equal error rate by more than 55% and the computational time by a factor of up to 44 compared to the original representation.

Show publication details

Dimitrov, Kristiyan; Kuijper, Arjan [Advisor]; Damer, Naser [Supervisor]

Exploring Deep Multi-biometric Fusion

2017

Darmstadt, TU, Master Thesis, 2017

The field of biometrics aims at performing automatic recognition of individuals based on their biological traits and is, hence, increasingly applied in places with high security requirements. To make biometric systems truly robust and reliable, multiple biometric sources could be combined with a particular fusion scheme. Mainly due to its ease of access, score-level fusion is the most practical method of multi-biometric fusion and has, thus, received the most attention from the research community. Higher-level fusion schemes (e.g. data or feature), in contrast, are difficult to achieve in practice. Yet, they are expected to yield superior results, owing to the higher amount of information available at the point of fusion. A central problem with this type of fusion is the extraction of a discriminative joint feature set. Feature extractors have to be manually designed and typically require a great deal of technical knowledge; for many tasks it is also infeasible to find an appropriate solution. Deep learning offers the ability to automatically learn useful features from raw data for any particular task with minimal human intervention. As a result, it is considered a reasonable option for realizing fusion in a multi-biometric scenario. Within the scope of this work, several architectures, based on a convolutional neural network model, are considered. Their performance have been tested in a multi-modal and a multiinstance setup, respectively. The results have been compared against baseline score-level fusion solutions. It has been shown, that with comparable network structures and computational costs, the less sophisticated, score-level fusion approach performs better than utilizing deep learning for the multi-biometric fusion process. As a future outlook, this thesis proposes possible modifications to the deep fusion approach that might improve the overall performance.

Show publication details

Damer, Naser; Terhörst, Philipp; Braun, Andreas; Kuijper, Arjan

General Borda Count for Multi-biometric Retrieval

2017

2017 International Joint Conference on Biometrics

IEEE International Joint Conference on Biometrics (IJCB) <2017, Denver, CO, USA>

Indexing of multi-biometric data is required to facilitate fast search in large-scale biometric systems. Previous works addressing this issue were challenged by including biometric sources of different nature, utilizing the knowledge about the biometric sources, and optimizing and tuning the retrieval performance. This work presents a generalized multi-biometric retrieval approach that adapts the Borda count algorithm within an optimizable structure. The approach was tested on a database of 10k reference and probe instances of the left and the right irises. The experiments and comparisons to five baseline solutions proved to achieve advances in terms of general indexing performance, tunability to certain operating points, and response to missing data. A clear advantage of the proposed solution was noticed when faced by candidate lists of low quality.

Show publication details

Terhörst, Philipp; Walther, Thomas [1. Gutachter]; Braun, Andreas [2. Gutachter]; Damer, Naser [Betreuer]

Indexing of Multi-biometric Databases: Fast and Accurate Biometric Search

2017

Darmstadt, TU, Master Thesis, 2017

Biometrics is a rapidly developing field of research and biometric-based identification systems experience a massive growth all around the world caused by the gaining industrial, government and citizen acceptance. The US-VISIT program uses biometric systems to enforce homeland and border security, whereas in the United Arab Emirates (UAE), biometric systems play a major role in the border control process. Similar, in India, biometrics have gained a great deal of attention, as the Unique Identification Authority of India (UIDAI) have already registered over one billion Indian citizens in the last 7 years (uidai.gov.in). Despite the rapid propagation of large-scale databases, the majority of researchers are still focusing on the matching accuracy of small databases, while neglecting scalability and speed issues. Identity association is usually determined by comparing input data against every entry in the database, which causes computational problems when it comes to large-scale databases. Biometric indexing aims to reduce the number of candidate identities to be considered by an identification system when searching for a match in large biometric databases. However, this is a challenging task since biometric data is fuzzy and does not exhibit any natural sorting order. Current indexing methods are mainly based on tree traversal (using kd-trees, B-trees, R-trees) which suffer from the curse of dimensionality, while other indexing methods are based on hashing, which suffer from pure key generation. The goal of this thesis is to develop an indexing scheme based on multiple biometric modalities. It aims to present the main results of research focusing on iris and fingerprint indexing. Fingerprints are undisputedly the most studied biometric modality that are extensive used in civil and forensic recognition systems. Together with the potential rise of iris recognition accurateness along with enhanced robustness, indexing of this modalities becomes a promising field of research. Different unimodal and multimodal identification approaches have already been proposed in past years. However, most of them trade fast identification rates at the cost of accuracy, while the remaining make use of complex indexing structures, which results in a complete restructuring if insertions or deletions are necessary. This work offers a framework for fast and accurate iris indexing as well as effective indexing schemes to combine multiple modalities. To achieve that, three main contributions are made: First, a new rotation invariant iris representation was developed, reducing the equal error rate by more than 55% and the computation time by a factor up to 44 compared to the original representation. Second, this representation was used to construct an indexing scheme, which reaches a hit rate of 99.7% at 0.1% penetration rate, outperforming state of the art algorithms. And third, a general rank-level indexing fusion scheme was developed to effectively combine multiple sources, achieving over 99.98% hit rate at same penetration rate of 0.1%.

Show publication details

Damer, Naser; Terhörst, Philipp; Braun, Andreas; Kuijper, Arjan

Indexing of Single and Multi-instance Iris Data Based on LSH-Forest and Rotation Invariant Representation

2017

Computer Analysis of Images and Patterns

International Conference on Computer Analysis of Images and Patterns (CAIP) <17, 2017, Ystad, Sweden>

Indexing of iris data is required to facilitate fast search in large-scale biometric systems. Previous works addressing this issue were challenged by the tradeoffs between accuracy, computational efficacy, storage costs, and maintainability. This work presents an iris indexing approach based on rotation invariant iris representation and LSH-Forest to produce an accurate and easily maintainable indexing structure. The complexity of insertion or deletion in the proposed method is limited to the same logarithmic complexity of a query and the required storage grows linearly with the database size. The proposed approach was extended into a multi-instance iris indexing scheme resulting in a clear performance improvement. Single iris indexing scored a hit rate of 99.7% at a 0.1% penetration rate while multi-instance indexing scored a 99.98% hit rate at the same penetration rate. The evaluation of the proposed approach was conducted on a large database of 50k references and 50k probes of the left and the right irises. The advantage of the proposed solution was put into prospective by comparing the achieved performance to the reported results in previous works.

Show publication details

Damer, Naser; Alkhatib, Wael; Braun, Andreas; Kuijper, Arjan

Neighbor Distance Ratios and Dynamic Weighting in Multi-biometric Fusion

2017

Pattern Recognition and Image Analysis

Iberian Conference on Pattern Recognition and Image Analysis (IbPRIA) <8, 2017, Faro, Portugal>

Multi-biometrics aims at building more accurate unified biometric decisions based on the information provided by multiple biometric sources. Information fusion is used to optimize the process of creating this unified decision. In previous works dealing with score-level multibiometric fusion, the scores of different biometric sources belonging to the comparison of interest are used to create the fused score. This is usually achieved by assigning static weights for the different biometric sources. In contrast, we focus on integrating the information imbedded in the relative relation between the comparison scores (within a 1:N comparison) in the biometric fusion process using a dynamic weighting scheme. This is performed by considering the neighbors distance ratio in the ranked comparisons to influence the dynamic weights of the fused scores. The evaluation was performed on the Biometric Scores Set BSSR1 database. The enhanced performance induced by including the neighbors distance ratio information within a dynamic weighting scheme in comparison to the baseline solution was shown by an average reduction of the equal error rate by more than 40% over the different test scenarios.

Show publication details

Henniger, Olaf; Damer, Naser; Braun, Andreas

Opportunities for Biometric Technologies in Smart Environments

2017

Ambient Intelligence

European Conference on Ambient Intelligence (AmI) <13, 2017, Malaga, Spain>

Smart environments describe spaces that are equipped with sensors, computing facilities and output systems that aim at providing their inhabitants with targeted services and supporting them in their tasks. Increasingly these are faced with challenges in differentiating multiple users and secure authentication. This paper outlines how biometric technologies can be applied in smart environments to overcome these challenges. We give an introduction to these domains and show various applications that can benefit from the combination of biometrics and smart environments.

Show publication details

Damer, Naser; Rhaibani, Chadi Izzou; Braun, Andreas; Kuijper, Arjan

Trust the Biometric Mainstream: Multi-biometric Fusion and Score Coherence

2017

2017 Proceedings of the 25th European Signal Processing Conference (EUSIPCO)

European Signal Processing Conference (EUSIPCO) <25, 2017, Kos, Greece>

Multi-biometrics aims at building more accurate unified biometric decisions based on the information provided by multiple biometric sources. Information fusion is used to optimize the process of creating this unified decision. In previous works dealing with score-level multi-biometric fusion, the scores of different biometric sources belonging to the comparison of interest are used to create the fused score. This is usually achieved by assigning static weights for the different biometric sources with more advanced solutions considering supplementary dynamic information like sample quality and neighbours distance ratio. This work proposes embedding score coherence information in the fusion process. This is based on our assumption that a minority of biometric sources, which points out towards a different decision than the majority, might have faulty conclusions and should be given relatively smaller role in the final decision. The evaluation was performed on the BioSecure multimodal biometric database with different levels of simulated noise. The proposed solution incorporates, and was compared to, three baseline static weighting approaches. The enhanced performance induced by including the coherence information within a dynamic weighting scheme in comparison to the baseline solution was shown by the reduction of the equal error rate by 45% to 85% over the different test scenarios and proved to maintain high performance when dealing with noisy data.

Show publication details

Siegmund, Dirk; Ebert, Tina; Damer, Naser

Combining Low-level Features of Offline Questionnaires for Handwriting Identification

2016

Image Analysis and Recognition

International Conference on Image Analysis and Recognition (ICIAR) <13, 2016, Póvoa de Varzim, Portugal>

When using anonymous offline questionnaires for reviewing services or products it is often not guaranteed that a reviewer does this only once as intended. In this paper an applied combination of different features of handwritten characteristics and its fusion is presented to expose such manipulations. The presented approach covers the aspects of alignment normalization, segmentation, feature extraction, classification and fusion. Nine features from handwritten text, numbers and checkboxes are extracted and used to recognize handwriter duplicates. The proposed method has been tested on a novel database containing pages of handwritten text produced by 1,734 writers. Furthermore we show that the unified biometric decision using a weighted sum combination rule can significantly improve writer identification performance even on low level features.

Show publication details

Damer, Naser; Maul, Fabian

Multi-biometric Continuous Authentication: a Trust Model for an Asynchronous System

2016

FUSION 2016

International Conference on Information Fusion (FUSION) <19, 2016, Heidelberg, Deutschland>

Biometric technologies are used to grant specific users access to services and data. The access control is usually performed at the start of a session that spans over a period of time. Continuous authentication aims at insuring the identity of the user over this period of time, and not only at its start. Multi-biometrics aims at increasing the accuracy, robustness and usability of biometrics systems. This work presents a multibiometric continuous authentication solution that includes information from the face images and the keystroke dynamics of the user. A database representing a realistic scenario was collected to develop and evaluate the presented solution. A multi-biometric trust model was designed to cope with the asynchronous nature induced by the different biometric characteristics. A set of performance metrics are discussed and a comparison is presented between the performances of the single characteristic solutions and the fused solution.

Show publication details

Damer, Naser; Dimitrov, Kristiyan

Practical View on Face Presentation Attack Detection

2016

Proceedings of the British Machine Vision Conference 2016 [Online]

British Machine Vision Conference (BMVC) <27, 2016, York, UK>

Face recognition is one of the most socially accepted forms of biometric recognition. The recent availability of very accurate and efficient face recognition algorithms leaves the vulnerability to presentation attacks as the major challenge to face recognition solutions. Previous works have shown high preforming presentation attack detection PAD solutions under controlled evaluation scenarios. This work tried to analyze the practical use of PAD by investigating the more realistic scenario of cross-database evaluation and presenting a state-of-the-art performance comparison. The work also investigated the relation between the video duration and the PAD performance. This is done along with presenting an optical flow based approach that proves to outperform state-of-the-art solutions in most experiment settings.

Show publication details

Maul, Fabian; Damer, Naser

Fuzzy Logic and Multi-biometric Fusion

2015

ICPRAM 2015

International Conference on Pattern Recognition Applications and Methods (ICPRAM) <4, 2015, Lisbon, Portugal>

Fuzzy logic has been proposed to improve various aspects of multi-biometric applications including enhancements to the decision making of the application and the robustness to noisy data. This paper discusses recent work that utilized fuzzy logic techniques within the multi-biometric fusion problem. This discussion is presented under two categories, the type of authentication scenario and the nature of the fused data. The paper also presents an introduction to fuzzy logic and multi-biometric fusion. Based on the discussed works, this paper aims to establish current trends and research possibilities in this field.

Show publication details

Maul, Fabian; Busch, Christoph [Referent]; Damer, Naser [Betreuer]

Multi-Biometric Continuous Authentication

2015

Darmstadt, Hochschule, Master Thesis, 2015

Biometric recognition can be used to secure the access to a system, by recognizing individuals seeking access, based on their behavioural and biological characteristics. In some scenarios, this level of security is not high enough, since it leaves room for attackers to gain access to the system after the initial recognition. Continuous authentication can be used to solve this problem by monitoring the current user during the work session. A genuine user with legitimate access should not be interrupted during the working session. Thus, biometric characteristics which require interaction with sensors are not suited for continuous authentication systems. As a consequence, research has been focused on behavioural biometric characteristics. A trust model defines the behaviour of the continuous authentication system by describing how actions of the user affect the trust value. Decisions are based on this trust value. This work aims to research whether a trust model can be used to combine a biological biometric characteristic and a behavioural characteristic, namely face recognition as the biological component and keystroke dynamics as the behavioural component. Face recognition was chosen because it does neither require additional interaction with a sensor, nor does it interrupt the work session of the genuine user. In order to lessen the impact on the privacy of the user, it was decided to use periodically taken pictures from a webcam instead of a permanent video surveillance. This added the challenge of the information collected by the system being asynchronous. The goal of this work is to develop and evaluate the feasibility and performance of such a system. In order to evaluate this proposed system a database of biometric data suitable for the application scenario was collected and a prototype of the system developed. Face recognition was implemented by using a Local Binary Linear Discriminant Analysis (LBLDA), for keystroke dynamics, a statistical method was implemented. Results show clear improvements in one metric, while the results in the other three measured metrics fell in a range between those of the unfused components. However, results can be further improved by using a more sophisticated fusion approach and tuning the sub components.

Show publication details

Damer, Naser; Samartzidis, Timotheos; Nouak, Alexander

Personalized Face Reference from Video: Key-Face Selection and Feature-Level Fusion

2015

Face and Facial Expression Recognition from Real World Videos

International Workshop on Face and Facial Expression Recognition from Real World Videos (FFER) <2014, Stockholm, Sweden>

Face recognition from video in uncontrolled environments is an active research field that received a growing attention recently. This was mainly driven by the wide range of applications and the availability of large databases. This work presents an approach to create a robust and discriminant reference face model from video enrollment data. The work focuses on two issues, first is the key faces selection from video sequences. The second is the feature-level fusion of the key faces. The proposed fusion approaches focus on inducing subject specific feature weighting in the reference face model. Quality based sample weighting is also considered in the fusion process. The proposed approach is evaluated under different sittings on the YouTube Faces data-base and the performance gained by the proposed approach is shown in the form of EER values and ROC curves.

Show publication details

Anumudu, Lawrence Kelechi; Damer, Naser [Betreuer]

Utilizing Fuzzy Distances in Biometric Comparison

2015

Darmstadt, Hochschule, Bachelor Thesis, 2015

In recent years, the use of biometric systems has dramatically increased and offers substantial improvement in recognizing individuals. The field of biometrics aims at recognizing individuals using their biological and physical characteristics. These characteristics play an important role because they are not easily changed or misused. Furthermore, they cannot be forgotten as is the case for passwords, nor can they be lost in the same manner as identification cards. The increasing demand for biometrics has motivated a lot of research in this field of study, which aim at finding improved and more accurate methods of biometric recognition. All biometric systems use biometric comparison to build their decisions. This comparison process is usually done by calculating the distance between the two feature vectors, each represents the biometric characteristics of different acquisition. Euclidean distance is one of the most commonly used distance measure. Other distance measures such as Manhattan Distance, Canberra Distance, Chebyshev Distance and Cosine Metric was used as the baseline distances in this thesis. The aim of this thesis is to construct a fuzzy distance for biometric comparison and to compare its performance with the baseline distances mentioned above. The main purpose behind a fuzzy logic system is that it allows partial set membership instead of crisp membership. Fuzzy logic system has been successfully being implemented in a lot of appliances that require artificial intelligence. The dataset used in this thesis is called "Labeled Faces in the Wild" gotten from a database of face photographs designed for studying the problem of unconstrained face recognition. The database contains over 13,000 images of faces collected online. Face recognition can be seen as a process of recognizing individuals using their facial characteristics. After implementing the proposed fuzzy distance, its performance was compared with the baseline distances using EER values. Results show that the proposed fuzzy distance is also a good distance measure for biometric comparison.

Show publication details

Damer, Naser; Nouak, Alexander

Weighted Integration of Neighbors Distance Ratio in Multi-biometric Fusion

2015

BIOSIG 2015

Annual International Conference of the Biometrics Special Interest Group (BIOSIG) <14, 2015, Darmstadt, Germany>

This work presents an approach to integrate biometric source weighting in the calculation of neighbors distance ratios to be used within a classification-based multi-biometric fusion process. The neighbors distance ratio represents the elevation of the top ranked identification match to the following ranks. Using biometric source weighing can help achieve more accurate initial identity ranking necessary for neighbors distance ratios. It also influences the effect of each biometric source on the ratios values. The proposed approach is developed and evaluated using the Biometric Scores Set BSSR1 database. The results are presented in the verification scenario as receiver operating curves (ROC). The achieved performance is compared to a number of baseline solutions and a satisfying and stable performance was achieved with a clear benefit of integrating the biometric source weights.

978-3-88579-639-8

Show publication details

Damer, Naser; Opel, Alexander; Nouak, Alexander

Biometric Source Weighting in Multi-Biometric Fusion: Towards a Generalized and Robust Solution

2014

2014 Proceedings of the 22nd European Signal Processing Conference (EUSIPCO)

European Signal Processing Conference (EUSIPCO) <22, 2014, Lisbon, Portugal>

This work presents a new weighting algorithm for biometric sources within a score-level multi-biometric system. Those weights are used in the effective and widely used weighted sum fusion rule to produce multi-biometric decisions. The presented solution is mainly based on the characteristic of the overlap region between the genuine and imposter scores distributions. It also integrates the performance of the biometric source represented by its equal error rate. This solution aims at avoiding the shortcomings of previously proposed solutions such as low generalization abilities and sensitiveness to outliers. The proposed solution is evaluated along with the state of the art and best practice techniques. The evaluation was performed on two databases, the Biometric Scores Set BSSR1 and the Extended Multi Modal Verification for Teleservices and Security applications database and a satisfying and stable performance was achieved.

Show publication details

Damer, Naser; Opel, Alexander; Nouak, Alexander

CMC Curve Properties and Biometric Source Weighting in Multi-Biometric Score-level Fusion

2014

FUSION 2014

International Conference on Information Fusion (FUSION) <17, 2014, Salamanca, Spain>

Multi-biometrics tries to build a unified biometric decision based on multiple biometric sources in an effort to gain more accuracy and robustness. Multi-biometric fusion aims at optimally combining the information produced by the multiple biometric sources, this usually requires assigning relative weights for the biometric sources to optimize their effect on the final decision. This work presents a new approach for biometric sources weighting within a score-level multi-biometric system. The presented solution tries to investigate the properties of the cumulative match characteristic (CMC) curve, which represents the biometric performance under the identification scenario, and extract biometric source weights based on those properties. The proposed solution is evaluated along with a set of state of the art and best practice weighting techniques. The evaluation was performed on the Biometric Scores Set BSSR1 database and a satisfying and stable performance was achieved.

Show publication details

Melzer, Marianne; Damer, Naser [Betreuer]

Gesichtserkennungssoftware in Bildarchiven - Eine Untersuchung zur Nutzung von Gesichtserkennungsprogrammen für die archivische Kernaufgabe "Erschließung"

2014

Potsdam, FH, Bachelor Thesis, 2014

Diese Arbeit betrachtet zunächst die theoretischen Grundlagen für die Gesichtserkennung. Das sind zum einen Erklärungen zur Archivale Foto, auch unter dem Aspekt der Fotoarchivierung und der archivischen Erschließung und die Biometrie bzw. Biometrik. Zum anderen ist es der Ablauf der Gesichtserkennung sowie die Algorithmen, welche häufig Anwendung hierbei finden. Mit Hilfe dieser Informationen werden verschiedene Softwareprogramme einer Untersuchung, die vorab ausführlich erläutert wird, zur Nutzung von Gesichtserkennungsprogrammen für die archivische Kernaufgabe "Erschließung" unterzogen. Die Ergebnisse dieser Untersuchung sollen aufzeigen, ob und inwiefern Gesichtserkennungsprogramme eine qualitative und quantitative Verbesserung in Bezug auf die Datensicherung (Personennennung) und Vereinfachung der Erschließung in Bildarchiven erzielen könnten. Eine Empfehlung resp. ein Ausblick zu diesem Thema vollenden die Arbeit.

Show publication details

Butt, Moazzam; Damer, Naser

Helper Data Scheme for 2D Cancelable Face Recognition Using Bloom Filters

2014

IWSSIP 2014. Proceedings

International Conference on Systems, Signals and Image Processing (IWSSIP) <21, 2014, Dubrovnik, Croatia>

Biometrics provide a source of automated recognition of individuals based on their physiological and behavioral characteristics. As per Directive 95/46/EC, biometric data is considered to be personal data. And according to article 8 of the European Convention on Human Rights, personal data needs to be privacy preserved. Biometric template protection mechanisms provide a privacy preserved biometric authentication. Such mechanisms assist irreversibility, revocability and unlinkability of biometric templates. Recently, a bloom filter based approach was proposed to generate irreversible iris template. In this paper, a helper data scheme for 2D cancelable face verification using bloom filters is proposed. The positions of most representative features (stable features) are used as helper data, which helps in the face recognition. The features used are extracted using Local Binary Linear Discriminant Analysis. The effect of stable features on recognition performance under scenarios of with and without using bloom filters is investigated. In addition, recognition performance after compressing multiple features into a single bloom filter is presented. The results are experimentally proved on two benchmark databases namely LFW and ORL datasets.

Show publication details

Damer, Naser; Opel, Alexander

Multi-biometric Score-Level Fusion and the Integration of the Neighbors Distance Ratio

2014

Image Analysis and Recognition. Proceedings Part II

International Conference on Image Analysis and Recognition (ICIAR) <11, 2014, Vilamoura, Portugal>

Multi-biometrics aims at building more accurate unified biometric decisions based on the information provided by multiple biometric sources. Information fusion is used to optimize the process of creating this unified decision. In previous works dealing with score-level multibiometric fusion, the scores of different biometric sources belonging to the comparison of interest are used to create the fused score. The novelty of this work focuses on integrating the relation of the fused scores to other comparisons within a 1:N comparison. This is performed by considering the neighbors distance ratio in the ranked comparisons set within a classification-based fusion approach. The evaluation was performed on the Biometric Scores Set BSSR1 database and the enhanced performance induced by the integration of neighbors distance ratio was clearly presented.

Show publication details

Aginako, Naiara; Irujo Arraiza, Juan; Cuadros, Montse; Raffaelli, Matteo; Kähm, Olga; Damer, Naser; Neto, Joao P. Neto

Multimedia Analysis of Video Sources

2014

SIGMAP 2014. Proceedings of the 11th International Conference on Signal Processing and Multimedia Applications

International Conference on Signal Processing and Multimedia Applications (SIGMAP) <11, 2014, Vienna, Austria>

Law Enforcement Agencies (LEAs) spend increasing efforts and resources on monitoring open sources, searching for suspicious behaviours and crime clues. The task of efficiently and effectively monitoring open sources is strongly linked to the capability of automatically retrieving and analyzing multimedia data. This paper presents a multimodal analytics system, created in cooperation with European LEAs. In particular it is described how the video analytics subsystem produces a workflow of multimedia data analysis processes. After a first analysis of video files, images are extracted in order to perform image comparison, classification and face recognition. In addition, audio content is extracted to perform speaker recognition and multilingual analysis of text transcripts. The integration of multimedia analysis results allows LEAs to extract pertinent knowledge from the gathered information.

Show publication details

Butt, Moazzam; Damer, Naser; Rathgeb, Christian

Privacy Preserved Duplicate Check using Multi-biometric Fusion

2014

FUSION 2014

International Conference on Information Fusion (FUSION) <17, 2014, Salamanca, Spain>

Automated recognition of individuals can be performed using biometrics without any requirement of explicit knowledge of a PIN or a password. On the one side biometrics has given convenience to citizens as they do not need to memorize a bunch of passwords, but on the other side intra (inter) class variations within (between) biometric features makes biometric authentications untrustworthy. Therefore, decisions based on biometric authentications are made more reliable by using several biometric authentications performed on single or multiple biometric modalities (i.e. multi-biometric fusion). This paper describes a method to identify if a person tries to re-enrol him/herself in a database, when he/she is already enrolled. This is referred to as duplicate check. In this work, duplicate check is performed using two modalities: face and iris. The templates used during the duplicate check are compliant to the ISO/IEC 24745 - Biometric information protection. Such templates are known as protected biometric templates. The protected biometric templates used in this work are generated using the recently published irreversible transformation based on Bloom filters. Scores are calculated from face and iris Bloom filters based templates by comparison with their respective enrolment templates using the normalized Hamming distance. As a decision of the duplicate check, these scores from both modalities are fused with appropriate weighting factors in order to get improved performance compared to using single individual modalities. The presented scheme is experimentally validated using two public benchmark databases namely the LFW and the CASIA-Iris-Thousand databases for face and iris respectively.

Show publication details

Siegmund, Dirk; Busch, Christoph [Supervisor]; Damer, Naser [Supervisor]

Prototypical Development of an In-Shop Advertisment System using Body Dimension Recognition

2014

Darmstadt, Hochschule, Master Thesis, 2014

This thesis outlines a system created to give consumers in the fashion industry an idea of how an item of clothing will look on them before trying it on. In the form of a short video, items of clothing are projected virtually onto an image of the user. Through the use of this system, retailers and manufacturers have the chance to immediately display their clothes on potential customers.

Show publication details

Samartzidis, Timotheos; Busch, Christoph [Supervisor]; Damer, Naser [Supervisor]

Three Dimensional Scanning of Clothes, for Simulation and Presentation Purposes in a Virtual Fitting Room

2014

Darmstadt, Hochschule, Master Thesis, 2014

The aim of this thesis is to develop a low-cost semi-professional automated 3D scanning and post-production system for digitizing clothing and apparel for in shop and online presentation purposes. Ultimately giving birth to a database of digitized 3d models of apparel to enable virtual-fitting rooms and real-time fitting feedback. In the first part different scanning methods are tested if they are suited for scanning apparel and if the quality is good enough for advertisement and presentation purposes. The cost of the system is also taken into account. The thesis then identifies the best and most cost effective approach and tries to develop and automate the method using state of the art consumer products. In the main section the thesis describes the functionality of the method and how it can be applied. Different algorithms and workflows are shown and combined to develop the automated system. In conclusion the thesis describes and summarizes the system and opens up how it could be implemented in a consumer oriented presentation like a virtual fitting room or an online shopping style advisor using the users body-metrics.

Show publication details

Siegmund, Dirk; Samartzidis, Timotheos; Damer, Naser; Nouak, Alexander; Busch, Christoph

Virtual Fitting Pipeline: Body Dimension Recognition, Cloth Modeling, and On-Body Simulation

2014

VRIPHYS 14: 11th Workshop in Virtual Reality Interactions and Physical Simulations

International Workshop in Virtual Reality Interaction and Physical Simulations (VRIPHYS) <11, 2014, Bremen, Germany>

This paper describes a solution for 3D clothes simulation on human avatars. The proposed approach consists of three parts, the collection of anthropometric human body dimensions, cloths scanning, and the simulation on 3D avatars. The simulation and human machine interaction has been designed for application in a passive In- Shop advertisement system. All parts have been evaluated and adapted under the aim of developing a low-cost automated scanning and post-production system. Human body dimension recognition was achieved by using a landmark detection based approach using both two 2D and 3D cameras for front and profile images. The human silhouettes extraction solution based on 2D images is expected to be more robust to multi-textured background surfaces than existing solutions. Eight measurements corresponding to the norm of body dimensions defined in the standard EN-13402 were used to reconstruct a 3D model of the human body. The performance is evaluated against the ground-truth of our newly acquired database. For 3D scanning of clothes, different scanning methods have been evaluated under apparel, quality and cost aspects. The chosen approach uses state of the art consumer products and describes how they can be combined to develop an automated system. The scanned cloths can be later simulated on the human avatars, which are created based on estimation of human body dimensions. This work concludes with software design suggestions for a consumer oriented solution such as a virtual fitting room using body metrics. A number of future challenges and an outlook for possible solutions are also discussed.

Show publication details

Damer, Naser; Opel, Alexander; Shahverdyan, Andreas

An Overview on Multi-biometric Score-level Fusion

2013

ICPRAM 2013

International Conference on Pattern Recognition Applications and Methods (ICPRAM) <2, 2013, Barcelona, Spain>

Multi-biometrics is the use of multiple biometric recognition sources to provide a more dependable verification or identification decision. Fusion of multi-biometric sources can be performed on different levels, such as the data, feature, or score level. This work presents an overview of the multi-biometric score-level fusion problem, along with the proposed solution in the literature. A discussion is made to provide a comparison between multi-biometric fusion in both scenarios. This discussion aims at providing a clearer view of future developments especially under the identification scenario where many related applications are rapidly growing such as forensics and ubiquitous surveillance.

Show publication details

Glaser, Christian; Kuijper, Arjan [Referent]; Kähm, Olga [Referentin]; Damer, Naser [Referent]

Face Liveness Detection Against Image and Video Spoofing Attacks

2013

Darmstadt, TU, Master Thesis, 2013

Many situations require users to log into a computer system. For example to perform private tasks like banking, social interaction or to get access to a secured area. Conventional security driven systems have the disadvantage that passwords or keycards are needed. These passwords or keycards can get lost or stolen resulting in a security risk. To overcome this drawback biometrics use the characteristics of the human body to grand access to a computer system. Beside fingerprint or iris recognition face detection is a popular biometric trait. The reason therefore is that it requires only a usual camera. Most of the current systems have a camera build in anyway. Also face recognition is not very intrusive to the user, which gives a high acceptability of face recognition is biometric trait. In the past though face recognition systems could easily be tricked due to spoofing attempts using pictures or videos of the authenticate user. This thesis analyses current algorithms to counter such spoofing attempts and presents a novel approach. The presented approach will use Machine Learning and Computer Vision to utilize an algorithms by Wu et al. [WRS_12] that can magnify subtle changes in videos to reveal the human pulse. An evaluation of the feasibility of this approach will be given.

Show publication details

Damer, Naser; Führer, Jan Benedikt; Kuijper, Arjan

Missing Data Estimation in Multi-biometric Identification and Verification

2013

2013 IEEE Workshop on Biometric Measurements and Systems for Security and Medical Applications

IEEE Workshop on Biometric Measurements and Systems for Security and Medical Applications (BioMS) <4, 2013, Naples, Italy>

In the practical use of multi-biometric solutions, biometric sources involved in producing the verification or identification decision do occasionally fail to produce results. This work discusses solutions for missing data in multi-biometric score-level fusion. A missing data estimation solution based on support vector regression was presented in this work and compared to four baseline solutions. The evaluation was carried under both the verification and the identification scenarios in an effort to show the effect of missing data estimation on the relatively understudied multi-biometric identification scenario. Evaluation was performed on the Biosecure DS2 score database and satisfying performance was achieved under both biometric scenarios.

Show publication details

Damer, Naser; Opel, Alexander; Nouak, Alexander

Performance Anchored Score Normalization for Multi-Biometric Fusion

2013

Advances in Visual Computing. 9th International Symposium, ISVC 2013

International Symposium on Visual Computing (ISVC) <9, 2013, Rethymnon, Crete, Greece>

This work presents a family of novel normalization techniques for score-level multi-biometric fusion. The proposed normalization is not only concerned to bring comparison scores to a common range and scale, it also focuses in bringing certain operational performance points in the distribution into alignment. The Performance Anchored Normalization (PAN) algorithms discussed here were tested on the extended Multi Modal Verification for Teleservices and Security applications database (XM2VTS) and proved to outperform conventional score normalization techniques in most tests. The tests were performed with combination fusion rules and presented as biometric verification performance measures.

Show publication details

Führer, Jan Benedikt; Damer, Naser [Betreuer]; Kuijper, Arjan [Prüfer]

Qualitätsbasierte Informationsfusion auf der Bewertungsebene innerhalb der multimodalen biometrischen Identifikation

2013

Darmstadt, TU, Master Thesis, 2013

Im Laufe der Zeit haben sich biometrische Erkennungsverfahren als zuverlässiges Mittel zum Zwecke der Zugangskontrolle zu physikalischen und virtuellen Bereichen entwickelt. Die Schwächen unimodaler Systeme werden dabei oft durch multimodale Ansätze verbessert, insbesondere ermöglichen diese einen robusteren Registrierungsprozess, erhöhte Sicherheit gegenüber gefälschten Identitäten und eine höhere Erkennungsgenauigkeit. Die vorliegende Arbeit beschäftigt sich mit der Frage, wie Qualitätsinformationen über die extrahierten Merkmale zu einer weiteren Verbesserung der multimodalen biometrischen Identifikation beitragen können. Dabei liegt die Idee zugrunde, dass Merkmale von höherer Qualität auch eine höhere Zuverlässigkeit im Hinblick auf deren Klassifikation zusichern, diese also stärker in den Entscheidungsprozess eingebunden werden sollten als Merkmale von geringerer Qualität. Zur Überprüfung dieser Annahme wurde ein auf dem Gradientenverfahren basierender Fusionsmechanismus um die Berücksichtigung von Qualitätsinformationen erweitert und dessen Erkennungsperformanz unter verschiedenen Konditionen, darunter im Besonderen das Vorhandensein fehlender Bewertungsmaße, mit dem ursprünglichen Algorithmus verglichen.

Show publication details

Chingovska, Ivana; Yang, Jinwei; Lei, Zhen; Yi, Dong; Li, Stan Z.; Kähm, Olga; Glaser, Christian; Damer, Naser; Kuijper, Arjan; Nouak, Alexander; Komulainen, Jukka; Pereira, Tiago de Freitas; Gupta, Shubham; Khandelwal, Shubham; Bansal, Shubham; Rai, Ayush; Krishna, Tarun; Goyal, Dushyant; Waris, Muhammad-Adeel; Zhang, Honglei; Ahmad, Iftikhar; Kiranyaz, Serkan; Gabbouj, Moncef; Tronci, Roberto; Pili, Maurizio; Sirena, Nicola; Roli, Fabio; Galbally, Javier; Fierrez, Julian; Pinto, Allan; Pedrini, Helio; Schwartz, William Robson; Rocha, Anderson; Anjos, André; Marcel, Sébastien

The 2nd Competition on Counter Measures to 2D Face Spoofing Attacks

2013

2013 International Conference on Biometrics (ICB)

IAPR International Conference on Biometrics (ICB) <6, 2013, Madrid, Spain>

As a crucial security problem, anti-spoofing in biometrics, and particularly for the face modality, has achieved great progress in the recent years. Still, new threats arrive in form of better, more realistic and more sophisticated spoofing attacks. The objective of the 2nd Competition on Counter Measures to 2D Face Spoofing Attacks is to challenge researchers to create counter measures effectively detecting a variety of attacks. The submitted propositions are evaluated on the Replay-Attack database and the achieved results are presented in this paper.

Show publication details

Maul, Fabian; Damer, Naser [Betreuer]

Untersuchung von Normalisierungs- und Kombinationsverfahren im Kontext von multibiometrischer score level Fusion

2013

Darmstadt, Hochschule, Bachelor Thesis, 2013

Diese Arbeit präsentiert eine Untersuchung von verschiedenen Normalisierungs- und Kombinationsverfahren im Kontext von multibiometrischer score level Fusion. Hierbei werden zur biometrischen Erkennung von Personen mehrere Informationen verwendet und zu einem eindeutigen Ergebnis verschmolzen. Die Effizienz dieser Verfahren wurde im Rahmen dieser Arbeit mittels einer Anwendung und einem konkretem Datensatz evaluiert. Die Evaluierung wurde für beide gängigen Modi einer biometrischen Erkennung, der Identifizierung und Verifikation, durchgeführt. Die Resultate zeigen, dass die Wahl der Verfahren einen großen Einfluss auf die Effizienz der biometrischen Erkennung ausübt. Im Verifikationsmodus wurde im schlechtesten Fall eine Erkennungsrate von 28.34% erzielt. Im besten Fall hingegen betrug die Erkennungsrate 95.54%. Die niedrigste Erkennungsrate im Identifikationsmodus betrug 51.92% und die höchste 98.71%.

Show publication details

Kähm, Olga; Damer, Naser

2D Face Liveness Detection: an Overview

2012

BIOSIG 2012

Annual International Conference of the Biometrics Special Interest Group (BIOSIG) <11, 2012, Darmstadt, Germany>

Face recognition based on 2D images is a widely used biometric approach. This is mainly due to the simplicity and high usability of this approach. Nonetheless, those solutions are vulnerable to spoof attacks made by non-real faces. In order to identify malicious attacks on such biometric systems, 2D face liveness detection approaches are developed. In this work, face liveness detection approaches are categorized based on the type of liveness indicator used. This categorization helps understanding different spoof attacks scenarios and their relation to the developed solutions. A review of the latest works dealing with face liveness detection works is presented. A discussion is made to link the state of the art solutions with the presented categorization along with the available and possible future datasets. All that aim to provide a clear path for the future development of innovative face liveness detection solutions.

Show publication details

Damer, Naser; Führer, Jan Benedikt

Ear Recognition Using Multi-Scale Histogram of Oriented Gradients

2012

Eighth International Conference on Intelligent Information Hiding and Multimedia Signal Processing. Proceedings

International Conference on Intelligent Information Hiding and Multimedia Signal Processing (IIH-MSP) <8, 2012, Piraeus-Athens, Greece>

Ear recognition is a promising biometric measure, especially with the growing interest in multi-modal biometrics. Histogram of Oriented Gradients (HOG) has been effectively and efficiently used solving the problems of object detection and recognition, especially when illumination variations are present. This work presents a robust approach for ear recognition using multi-scale dense HOG features as a descriptor of 2D ear images. The multi-scale features assure to capture the different and complicated structures of ear images. Dimensionality reduction was performed to avoid feature redundancy and provide a more efficient recognition process while being prone to over-fitting. Finally, a test was performed on a large and realistic database and the results were compared to the state of the art ear recognition approaches tested on the same dataset and under the same test procedure.