Detecting Face Morphing Attacks by Analyzing the Directed Distances of Facial Landmarks Shifts
German Conference on Pattern Recognition (GCPR) <40, 2018, Stuttgart, Germany>
Lecture Notes in Computer Science (LNCS), 11269
Face morphing attacks create face images that are verifiable to multiple identities. Associating such images to identity documents lead to building faulty identity links, causing attacks on operations like border crossing. Most of previously proposed morphing attack detection approaches directly classified features extracted from the investigated image. We discuss the operational opportunity of having a live face probe to support the morphing detection decision and propose a detection approach that take advantage of that. Our proposed solution considers the facial landmarks shifting patterns between reference and probe images. This is represented by the directed distances to avoid confusion with shifts caused by other variations. We validated our approach using a publicly available database, built on 549 identities. Our proposed detection concept is tested with three landmark detectors and proved to outperform the baseline concept based on handcrafted and transferable CNN features.
Humans vs Machines: A Comparison of Human and Machine Learning Performance in Inferring Professions from Facial Images
Frankfurt am Main, Univ., Master Thesis, 2019
Convolutional neural networks have seen a rise in the computer vision community in the recent years, even surpassing human accuracies in face recognition tasks. However, research regarding different face classification and face clustering experiments, which do not focus on the discrimination of face images according to unique individuals in the dataset, are still very limited. Many psychological studies indicate that humans are capable in inferring leaders, status and competence from facial images. This work focused on the question whether humans are indeed capable in telling profession – given by the branch of the profession and the career status – from face images and whether current state-of-the-art machine learning approaches are able to do the same. Furthermore, the performance of humans and machine learning systems were compared to give insights in the underlying processes. The results indicate that both human and machine learning models are capable to infer professions from facial images with better than chance accuracies. Both humans and machine learning systems perform almost equally well in these tasks, however, performance differences in individual tasks indicate that humans and machine learning algorithms solve the same tasks while relying on different cues in the face images.
Investigating the Use of Deeply Calculated Flows and Dynamic Routed Network Capsules in Face Presentation Attack Detection
Darmstadt, TU, Master Thesis, 2019
Identifying an individual based on their facial characteristics is convenient and is gaining more and more importance. Only a normal camera is needed for data capturing, the individual characteristic can’t be stolen or forgot like a password, and it’s a non-intrusive capturing method, which enables identification from distances. With increasing usage of face recognition systems, also the amount of sophisticated attacks increase. Therefore vulnerabilities of such a recognition system are used. Especially the attack directly at the sensor, by presenting fake biometrics at the sensor, can be performed with low efforts and costs, and without detailed knowledge about the biometric system. That’s why this kind of attacks have caused wide attention in the biometric community. The attack is called presentation attack and there are many detection algorithms developed to solve this issue. Most existing presentation attack detection algorithms work great, when deployed in the same environment as they are trained in. But different environmental conditions are still challenging to solve. This thesis deals with the presentation attack issue by developing an approach, which combines deep optical flows and a capsule neural network. Optical flows have already proven to be a good preprocessing step, because only motion information is kept and environmental conditions won’t have a big impact on it. Deep optical flows are produced by end-to-end trained convolutional neural networks for the optical flow calculation. Capsule neural networks train neurons in a more general way by forwarding information of found pattern’s poses in the network additionally to it’s probabilities. Forwarding is achieved by a so called routing-by-agreement method, which allows to take relationships between the found patterns into account. For presentation attack detection development not enough data is available to train a deep neural network from scratch. This could be overcome by using a capsule neural network for classification. A deep understanding can be learned, needing less data. The contribution of this thesis is the combination of deep optical flows and a capsule neural network, which hasn’t been tested so far. This is realized by stacking multiple optical flows over the time and downsample this information, thus that the capsule neural network is able to train a classification. This pipeline is structurally optimized by testing different optical flow sizes, data normalization and multiple class training. Also a new downsampling and time stacking method is developed to keep outliers in the reduced data and order a stack according to the contained data information. This has proven to help the network to classify. In the end, the behavior of the network on various attacks is analyzed. In this thesis, performance improvements are achieved by structure optimizations and a novel downsampling and time stacking method. The structural improvements increased the model performance up to 12% and the new downsampling and time stacking method leads to an average performance improvement of 7.56% for two databases. However, the achieved performance does not outperform recent state-of-the-art presentation attack detection methods and the analysis over different attacks reveals that the developed structure adapts weakly to changes in the attack performance or to new attacks. Nevertheless, stable results for different environmental conditions are achieved.
Minutiae-Based Gender Estimation for Full and Partial Fingerprints of Arbitrary Size and Shape
Computer Vision - ACCV 2018
Asian Conference on Computer Vision (ACCV) <14, 2018, Perth, Australia>
Lecture Notes in Computer Science (LNCS), 11361
Since fingerprints are one of the most widely deployed biometrics, accurate fingerprint gender estimation can positively affect several applications. For example, in criminal investigations, gender classification may significantly minimize the list of potential subjects. Previous work mainly offered solutions for the task of gender classification based on complete fingerprints. However, partial fingerprint captures are frequently occurring in many applications, including forensics and the fast growing field of consumer electronics. Due to its huge variability in size and shape, gender estimation on partial fingerprints is a challenging problem. Therefore, in this work we propose a flexible gender estimation scheme by building a gender classifier based on an ensemble of minutiae. The outputs of the single minutia gender predictions are combined by a novel adjusted score fusion approach to obtain an enhanced gender decision. Unlike classical solutions this allows to deal with unconstrained fingerprint parts of arbitrary size and shape. We performed investigations on a publicly available database and our proposed solution proved to significantly outperform state-of-the-art approaches on both full and partial fingerprints. The experiments indicate a reduction in the gender estimation error by 19.34% on full fingerprints and 28.33% on partial captures in comparison to previous work.
On the Detection of GAN-Based Face Morphs Using Established Morph Detectors
Image Analysis and Processing - ICIAP 2019
International Conference of Image Analysis and Processing (ICIAP) <20, 2019, Trento, Italy>
Lecture Notes in Computer Science (LNCS), 11752
Face recognition systems (FRS) have been found to be highly vulnerable to face morphing attacks. Due to this severe security risk, morph detection systems do not only need to be robust against classical landmark-based face morphing approach (LMA), but also future attacks such as neural network based morph generation techniques. The focus of this paper lies on an experimental evaluation of the morph detection capabilities of various state-of-the-art morph detectors with respect to a recently presented novel face morphing approach, MorGAN, which is based on Generative Adversarial Networks (GANs). In this work, existing detection algorithms are confronted with different attack scenarios: known and unknown attacks comprising different morph types (LMA and MorGAN). The detectors’ performance results are highly dependent on the features used by the detection algorithms. In addition, the image quality of the morphed face images produced with the MorGAN approach is assessed using well-established no-reference image quality metrics and compared to LMA morphs. The results indicate that the image quality of MorGAN morphs is more similar to bona fide images compared to classical LMA morphs.
Reducing Ethnic Bias of Face Recognition by Ethnic Augmentation
Darmstadt, TU, Master Thesis, 2019
Automated face recognition has gained wider deployment ground after the recent accuracy gains achieved by deep learning techniques. Despite the rapid performance gains, face recognition still suffers from very critical issues. One of the recently uncovered, and very sensitive, challenges is the ethnicity bias in face recognition decision. This is the case, unfortunately, even in the latest commercial and academic face recognition system. In 2018, the National Institute of Standards and Technology (NIST) published the latest report regarding the evaluation result of commercial face recognition solutions from several major biometric vendors. The report specifically evaluated and demonstrated the variance of the error rates of the evaluated solutions based on demographic variations. This thesis is one of the first research efforts in addressing the decision bias challenge in biometrics. It builds its hypothesis on the strong assumption that ethnicity bias is caused by the relative under-representation of certain ethnicities in training databases. This work introduces a novel ethnicity-driven data augmentation approach to reduce biometric bias. The proposed approach successfully utilize a generative image model to generate new face images that preserve the identity of their source images while partially transforming their ethnicity to the targeted ethnicity group. A large-scale ethnicity-labeled face images database is created in order to develop and evaluate the proposed approach. To achieve that, part of this thesis focused on creating an ethnicity classifier to annotate face images, achieving accuracy in the state-of-the-art range. The proposed ethnicity-driven face generative model is developed based on the ethnicity labeled images to generate realistically and high-resolution face images, depending on a limit amount of training data. More importantly, the thesis proves that the proposed augmentation approach strongly preserves the identity of the input images and partially transforms the ethnicity. The augmented images are used as part of the training data of a face recognition model. The achieved verification results prove that the proposed ethnicity augmentation methods significantly and consistently reduced the ethnicity bias of the face recognition model. For examples, the ERR was reduced from 0.159 to 0.130 when verifying inter-ethnicity samples of Black individuals on a model trained on Asian individual images, respectively before and after applying the proposed training data augmentation. Moreover, the overall performance of the face recognition model was improved. However, this improvement was more significant, as intended, in the targeted ethnicity groups.
Unsupervised Privacy-enhancement of Face Representations Using Similarity-sensitive Noise Transformations
Face images processed by a biometric system are expected to be used for recognition purposes only. However, recent work presented possibilities for automatically deducing additional information about an individual from their face data. By using soft-biometric estimators, information about gender, age, ethnicity, sexual orientation or the health state of a person can be obtained. This raises a major privacy issue. Previous works presented supervised solutions that require large amount of private data in order to suppress a single attribute. In this work, we propose a privacy-preserving solution that does not require these sensitive information and thus, works in an unsupervised manner. Further, our approach offers privacy protection that is not limited to a single known binary attribute or classifier. We do that by proposing similarity-sensitive noise transformations and investigate their effect and the effect of dimensionality reduction methods on the task of privacy preservation. Experiments are done on a publicly available database and contain analyses of the recognition performance, as well as investigations of the estimation performance of the binary attribute of gender and the continuous attribute of age. We further investigated the estimation performance of these attributes when the prior knowledge about the used privacy mechanism is explicitly utilized. The results show that using this information leads to significantly enhancement of the estimation quality. Finally, we proposed a metric to evaluate the trade-off between the privacy gain and the recognition loss for privacy-preservation techniques. Our experiments showed that the proposed cosine-sensitive noise transformation was successful in reducing the possibility of estimating the soft private information in the data, while having significantly smaller effect on the intended recognition performance.
Application-driven Advances in Multi-biometric Fusion
Darmstadt, TU, Diss., 2018
Biometric recognition is the automated recognition of individuals based on their behavioral or biological characteristics. Beside forensic applications, this technology aims at replacing the outdated and attack prone, physical and knowledge-based, proofs of identity. Choosing one biometric characteristic is a tradeoff between universality, acceptability, and permanence, among other factors. Moreover, the accuracy cap of the chosen characteristic may limit the scalability and usability for some applications. The use of multiple biometric sources within a unified frame, i.e. multi-biometrics, aspires to tackle the limitations of single source biometrics and thus enables a wider implementation of the technology. This work aims at presenting application-driven advances in multi-biometrics by addressing different elements of the multi-biometric system work-flow. At first, practical oriented pre-fusion issues regarding missing data imputation and score normalization are discussed. This includes presenting a novel performance anchored score normalization technique that aligns certain performance-related score values in the fused biometric sources leading to more accurate multi-biometric decisions when compared to conventional normalization approaches. Missing data imputation within scorelevel multi-biometric fusion is also addressed by analyzing the behavior of different approaches under different operational scenarios. Within the multi-biometric fusion process, different information sources can have different degrees of reliability. This is usually influenced in the fusion process by assigning relative weights to the fused sources. This work presents a number of weighting approaches aiming at optimizing the decision made by the multi-biometric system. First, weights that try to capture the overall performance of the biometric source, as well as an indication of its confidence, are proposed and proved to outperform the state-of-the-art weighting approaches. The work also introduces a set of weights derived from the identification performance representation, the cumulative match characteristics. The effect of these weights is analyzed under the verification and identification scenarios. To further optimize the multi-biometric process, information besides the similarity between two biometric captures can be considered. Previously, the quality measures of biometric captures were successfully integrated, which requires accessing and processing raw captures. In this work, supplementary information that can be reasoned from the comparison scores are in focus. First, the relative relation between different biometric comparisons is discussed and integrated in the fusion process resulting in a large reduction in the error rates. Secondly, the coherence between scores of multi-biometric sources in the same comparison is defined and integrated into the fusion process leading to a reduction in the error rates, especially when processing noisy data. Large-scale biometric deployments are faced by the huge computational costs of running biometric searches and duplicate enrollment checks. Data indexing can limit the search domain leading to faster searches. Multibiometrics provides richer information that can enhance the retrieval performance. This work provides an optimizable and configurable multi-biometric data retrieval solution that combines and enhances the robustness of rank-level solutions and the performance of feature-level solutions. Furthermore, this work presents biometric solutions that complement and utilize multi-biometric fusion. The first solution captures behavioral and physical biometric characteristics to assure a continuous user authentication. Later, the practical use of presentation attack detection is discussed by investigating the more realistic scenario of cross-database evaluation and presenting a state-of-the-art performance comparison. Finally, the use of multibiometric fusion to create face references from videos is addressed. Face selection, feature-level fusion, and score-level fusion approaches are evaluated under the scenario of face recognition in videos.
CrazyFaces: Unassisted Circumvention of Watchlist Face Identification
IEEE 9th International Conference on Biometrics: Theory, Applications and Systems
IEEE International Conference on Biometrics: Theory, Applications and Systems (BTAS) <9, 2018, Redondo Beach, CA, USA>
Once upon a time, there was a blacklisted criminal who usually avoided appearing in public. He was surfing the Web, when he noticed, what had to be a targeted advertisement announcing a concert of his favorite band. The concert was in a near town, and the only way to get there was by train. He was worried, because he heard in the news about the new face identification system installed at the train station. From his last stay with the police, he remembers that they took these special face images with the white background. He thought about what can he do to avoid being identified and an idea popped in his mind “what if I can make a crazy-face, as the kids call it, to make my face look different? What do I exactly have to do? And will it work?”. He called his childhood geeky friend and asked him if he can build him a face recognition application he can tinker with. The geeky friend was always interested in such small projects where he can use open-source resources and didn’t really care about the goal, as usual. The criminal tested the application and played around, trying to figure out how can he make a crazy-face that won’t be identified as himself. On the day of the concert, he took off to the train station with some doubt in his mind and fear in his soul. To know what happened next, you should read the rest of this paper.
Creating Face Morphing Attacks with Generative Adversarial Networks
Saint-Étienne, Univ., Master Thesis, 2018
Nowadays, the use of technologies related to biometrics is increasing significantly. The recent deep learning improvement in face recognition tasks, along with the relatively high social acceptance, have pushed automatic face recognition systems to be a key technology in identity verification in border controls. In this scenario, face recognition is used to link the identity of a passenger to their e-document. However, recent studies have highlighted the threat of morphing attacks against automatic face recognition systems. In this thesis, we present a novel morphed face attack, called MorGANA, created by using Generative Adversarial Networks. By creating a new morphing database, MorGAN dataset, we investigate the vulnerabilities of current face recognition systems against MorGANA attacks, alongside with baseline attacks. Moreover, the morphing detectability under face morphing attacks is further studied noticing an insufficient performance from common detectors. Examining the obtained results, we strongly consider that further studies need to be addressed in generating and detecting morphed face attacks in order to bring face recognition systems to real-case scenarios.
Cross-device Biometric Verification and the Benefit of Multi-algorithmic Biometric Fusion
Frankfurt am Main, Univ. of Applied Science, Master Thesis, 2018
Biometrics is a technology that aims to identify or verify people identities based on their physical characteristics or behavioral properties. Multi-biometrics is implemented to use a number of biometric information sources to create a unified decision to increase the accuracy of the biometric system. This aims at enhancing performance and avoiding the shortcomings of conventional single source biometrics such as sensitivity to noisy data or capture environment while maintaining high accuracy. One of these shortcomings are the negative effect of cross-device biometric verification, e.g. using different smart phones and capture devices for the enrolment and verification of face images. The previous work deals with multi-biometrics to obtain the fused score by incorporating the coherence information on the biometric sources to enhance the performance of the Biometric system. In this thesis the main purpose is to improve the performance of the Biometric system for multi-modality (two modalities) with each biometric source acquired from different sensor devices which includes both same sensor and different sensor combinations. This includes biometric sources, i.e. face and one of the periocular region fed to the biometric system with multi-algorithmic approach to extract the features and create unified decision at the score level. Analysis of different realistic scenario with varying static and dynamic weights and its effect on the unified decision on score level. This approach could be used in biometric system with more than two modalities.
Deep and Multi-algorithmic Gender Classification of Single Fingerprint Minutiae
International Conference on Information Fusion (FUSION) <21, 2018, Cambridge, UK>
Accurate fingerprint gender estimation can positively affect several applications, since fingerprints are one of the most widely deployed biometrics. For example, gender classification in criminal investigations may significantly minimize the list of potential subjects. Previous work mainly offered solutions for the task of gender classification based on complete fingerprints. However, partial fingerprint captures are frequently occurring in many applications, including forensics and the fast growing field of consumer electronics. Moreover, partial fingerprints are not well-defined. Therefore, this work improves the gender decision performance on a well-defined partition of the fingerprint. It enhances gender estimation on the level of a single minutia. Working on this level, we propose three main contributions that were evaluated on a publicly available database. First, a convolutional neural network model is offered that outperformed baseline solutions based on hand crafted features. Second, several multi-algorithmic fusion approaches were tested by combining the outputs of different gender estimators that help further increase the classification accuracy. Third, we propose including minutia detection reliability in the fusion process, which leads to enhancing the total gender decision performance. The achieved gender classification performance of a single minutia is comparable to the accuracy that previous work reported on a quarter of aligned fingerprints including more than 25 minutiae.
Deep Learning-based Face Recognition and the Robustness to Perspective Distortion
24th International Conference on Pattern Recognition. Proceedings
International Conference on Pattern Recognition (ICPR) <24, 2018, Beijing, China>
Face recognition technology is spreading into a wide range of applications. This is mainly driven by social acceptance and the performance boost achieved by the deep learningbased solutions in the recent years. Perspective distortion is an understudied distortion in face recognition that causes converging verticals when imaging 3D objects depending on the distance to the object. The effect of this distortion on face recognition was previously studied for algorithms based on hand-crafted features with a clear negative effect on verification performance. Possible solutions were proposed by compensating the distortion effect on the face image level, which requires knowing the camera settings and capturing a high quality image. This work investigates the effect of perspective distortion on the performance of a deep learning-based face recognition solution. It also provides a device parameter-independent solution to decrease this effect by creating more perspective-robust face representations. This was achieved by training the deep learning model on perspective-diverse data, without increasing the size of the training data. Experiments performed on the deep model in hand and a specifically collected database concluded that the perspective distortion effects face verification performance if not considered in the training process, and that this can be improved by our proposal of creating robust face representations by properly selecting the training data.
Fingerprint and Iris Multi-biometric Data Indexing and Retrieval
International Conference on Information Fusion (FUSION) <21, 2018, Cambridge, UK>
Indexing of multi-biometric data is required to facilitatefast search in large-scale biometric systems. Previous worksaddressing this issue in multi-biometric databases focused onmulti-instance indexing, mainly iris data. Few works addressedthe indexing in multi-modal databases, with basic candidate listfusion solutions limited to joining face and fingerprint data. Irisand fingerprint are widely used in large-scale biometric systemswhere fast retrieval is a significant issue. This work proposes jointmulti-biometric retrieval solution based on fingerprint and irisdata. This solution is evaluated under eight different candidatelist fusion approaches with variable complexity on a databaseof 10,000 reference and probe records of irises and fingerprints.Our proposed multi-biometric retrieval of fingerprint and irisdata resulted in a reduction of the miss rate (1- hit rate) at 0.1%penetration rate by 93% compared to fingerprint indexing and88% compared to iris indexing.
MorGAN: Recognition Vulnerability and Attack Detectability of Face Morphing Attacks Created by Generative Adversarial Network
IEEE 9th International Conference on Biometrics: Theory, Applications and Systems
IEEE International Conference on Biometrics: Theory, Applications and Systems (BTAS) <9, 2018, Redondo Beach, CA, USA>
Face morphing attacks aim at creating face images that are verifiable to be the face of multiple identities, which can lead to building faulty identity links in operations like border crossing. Research has been focused on creating more accurate attack detection approaches by considering different image properties. However, all the attacks considered so far are based on manipulating facial landmarks localized in the morphed face images. In contrast, this work presents novel face morphing attacks based on image generated by generative adversarial networks. We present the MorGAN structure that considers the representation loss to successfully create realistic morphing attacks. Based on that, we present a novel face morphing attacks database (MorGAN database) that contains 1000 morph images for both, the proposed MorGAN and landmark-based attacks. We present vulnerability analysis of two face recognition approaches facing the proposed attacks. Moreover, the detectability of the proposed MorGAN attacks is studied, in the scenarios where this type of attacks is know and unknown. We concluded with pointing out the challenge of detecting such unknown novel attacks and an analysis of detection performances of different features in detecting such attacks.
P-score: Performance Aligned Normalization and an Evaluation in Score-level Multi-biometric Fusion
2018 Proceedings of the 26th European Signal Processing Conference (EUSIPCO)
European Signal Processing Conference (EUSIPCO) <26, 2018, Rome, Italy>
Normalization is an important step for different fusion, classification, and decision making applications. Previous normalization approaches considered bringing values from different sources into a common range or distribution characteristics. In this work we propose a new normalization approach that transfers values into a normalized space where their relative performance in binary decision making is aligned across their whole range. Multi-biometric verification is a typical problem where information from different sources are normalized and fused to make a binary decision and therefore a good platform to evaluate the proposed normalization.We conducted an evaluation on two publicly available databases and showed that the normalization solution we are proposing consistently outperformed state-of-the-art and best practice approaches, e.g. by reducing the false rejection rate at 0.01% false acceptance rate by 60- 75% compared to the widely used z-score normalization under the sum-rule fusion.
The Dark Side of the Face: Exploring the Ultraviolet Spectrum for Face Biometrics
2018 International Conference on Biometrics (ICB)
IAPR International Conference on Biometrics (ICB) <11, 2018, Gold Coast, Australia>
Facial recognition in the visible spectrum is a widelyused application but it is also still a major field of research.In this paper we present melanin face pigmentation (MFP)as a new modality to be used to extend classical face biometrics. Melanin pigmentation are sun-damaged cells thatoccur as revealed and/or unrevealed pattern on human skin.Most MFP can be found in the faces of some people whenusing ultraviolet (UV) imaging. To proof the relevance ofthis feature for biometrics, we present a novel image datasetof 91 multiethnic subjects in both, the visible and the UVspectrum. We show a method to extract the MFP featuresfrom the UV images, using the well known SURF featuresand compare it with other techniques. In order to proof itsbenefits, we use weighted score-level fusion and evaluatethe performance in an one against all comparison. As a resultwe observed a significant amplification of performancewhere traditional face recognition in the visible spectrum isextended with MFP from UV images. We conclude with afuture perspective about the use of these features for futureresearch and discuss observed issues and limitations.
What Can a Single Minutia Tell about Gender?
2018 International Workshop on Biometrics and Forensics (IWBF)
International Workshop on Biometrics and Forensics (IWBF) <2018, Sassari, Italy>
Since fingerprints are one of the most widely deployed biometrics, several applications can benefit from an accurate fingerprint gender estimation. Previous work mainly tackled the task of gender estimation based on complete fingerprints. However, partial fingerprint captures are frequently occurring in many applications including forensics and consumer electronics, with the considered ratio of the fingerprint is variable. Therefore, this work investigates gender estimation on a small, detectable, and well-defined partition of a fingerprint. It investigates gender estimation on the level of a single minutia. Working on this level, we propose a feature extraction process that is able to deal with the rotation and translation invariance problems of fingerprints. This is evaluated on a publicly available database and with five different binary classifiers. As a result, the information of a single minutia achieves a comparable accuracy on the gender classification task as previous work using quarters of aligned fingerprints with an average of more than 25 minutiae.
Efficient, Accurate, and Rotation-Invariant Iris Code
IEEE Signal Processing Letters
The large scale of the recently demanded biometric systems has put a pressure on creating a more efficient, accurate, and private biometric solutions. Iris biometrics is one of the most distinctive and widely used biometric characteristics. High-performing iris representations suffer from the curse of rotation inconsistency. This is usually solved by assuming a range of rotational errors and performing a number of comparisons over this range, which results in a high computational effort and limits indexing and template protection. This work presents a generic and parameter-free transformation of binary iris representation into a rotation-invariant space. The goal is to perform accurate and efficient comparison and enable further indexing and template protection deployment. The proposed approach was tested on a database of 10 000 subjects of the ISYN1 iris database generated by CASIA. Besides providing a compact and rotational-invariant representation, the proposed approach reduced the equal error rate by more than 55% and the computational time by a factor of up to 44 compared to the original representation.
Exploring Deep Multi-biometric Fusion
Darmstadt, TU, Master Thesis, 2017
The field of biometrics aims at performing automatic recognition of individuals based on their biological traits and is, hence, increasingly applied in places with high security requirements. To make biometric systems truly robust and reliable, multiple biometric sources could be combined with a particular fusion scheme. Mainly due to its ease of access, score-level fusion is the most practical method of multi-biometric fusion and has, thus, received the most attention from the research community. Higher-level fusion schemes (e.g. data or feature), in contrast, are difficult to achieve in practice. Yet, they are expected to yield superior results, owing to the higher amount of information available at the point of fusion. A central problem with this type of fusion is the extraction of a discriminative joint feature set. Feature extractors have to be manually designed and typically require a great deal of technical knowledge; for many tasks it is also infeasible to find an appropriate solution. Deep learning offers the ability to automatically learn useful features from raw data for any particular task with minimal human intervention. As a result, it is considered a reasonable option for realizing fusion in a multi-biometric scenario. Within the scope of this work, several architectures, based on a convolutional neural network model, are considered. Their performance have been tested in a multi-modal and a multiinstance setup, respectively. The results have been compared against baseline score-level fusion solutions. It has been shown, that with comparable network structures and computational costs, the less sophisticated, score-level fusion approach performs better than utilizing deep learning for the multi-biometric fusion process. As a future outlook, this thesis proposes possible modifications to the deep fusion approach that might improve the overall performance.
General Borda Count for Multi-biometric Retrieval
2017 International Joint Conference on Biometrics
IEEE International Joint Conference on Biometrics (IJCB) <2017, Denver, CO, USA>
Indexing of multi-biometric data is required to facilitate fast search in large-scale biometric systems. Previous works addressing this issue were challenged by including biometric sources of different nature, utilizing the knowledge about the biometric sources, and optimizing and tuning the retrieval performance. This work presents a generalized multi-biometric retrieval approach that adapts the Borda count algorithm within an optimizable structure. The approach was tested on a database of 10k reference and probe instances of the left and the right irises. The experiments and comparisons to five baseline solutions proved to achieve advances in terms of general indexing performance, tunability to certain operating points, and response to missing data. A clear advantage of the proposed solution was noticed when faced by candidate lists of low quality.
Indexing of Multi-biometric Databases: Fast and Accurate Biometric Search
Darmstadt, TU, Master Thesis, 2017
Biometrics is a rapidly developing field of research and biometric-based identification systems experience a massive growth all around the world caused by the gaining industrial, government and citizen acceptance. The US-VISIT program uses biometric systems to enforce homeland and border security, whereas in the United Arab Emirates (UAE), biometric systems play a major role in the border control process. Similar, in India, biometrics have gained a great deal of attention, as the Unique Identification Authority of India (UIDAI) have already registered over one billion Indian citizens in the last 7 years (uidai.gov.in). Despite the rapid propagation of large-scale databases, the majority of researchers are still focusing on the matching accuracy of small databases, while neglecting scalability and speed issues. Identity association is usually determined by comparing input data against every entry in the database, which causes computational problems when it comes to large-scale databases. Biometric indexing aims to reduce the number of candidate identities to be considered by an identification system when searching for a match in large biometric databases. However, this is a challenging task since biometric data is fuzzy and does not exhibit any natural sorting order. Current indexing methods are mainly based on tree traversal (using kd-trees, B-trees, R-trees) which suffer from the curse of dimensionality, while other indexing methods are based on hashing, which suffer from pure key generation. The goal of this thesis is to develop an indexing scheme based on multiple biometric modalities. It aims to present the main results of research focusing on iris and fingerprint indexing. Fingerprints are undisputedly the most studied biometric modality that are extensive used in civil and forensic recognition systems. Together with the potential rise of iris recognition accurateness along with enhanced robustness, indexing of this modalities becomes a promising field of research. Different unimodal and multimodal identification approaches have already been proposed in past years. However, most of them trade fast identification rates at the cost of accuracy, while the remaining make use of complex indexing structures, which results in a complete restructuring if insertions or deletions are necessary. This work offers a framework for fast and accurate iris indexing as well as effective indexing schemes to combine multiple modalities. To achieve that, three main contributions are made: First, a new rotation invariant iris representation was developed, reducing the equal error rate by more than 55% and the computation time by a factor up to 44 compared to the original representation. Second, this representation was used to construct an indexing scheme, which reaches a hit rate of 99.7% at 0.1% penetration rate, outperforming state of the art algorithms. And third, a general rank-level indexing fusion scheme was developed to effectively combine multiple sources, achieving over 99.98% hit rate at same penetration rate of 0.1%.
Indexing of Single and Multi-instance Iris Data Based on LSH-Forest and Rotation Invariant Representation
Computer Analysis of Images and Patterns
International Conference on Computer Analysis of Images and Patterns (CAIP) <17, 2017, Ystad, Sweden>
Indexing of iris data is required to facilitate fast search in large-scale biometric systems. Previous works addressing this issue were challenged by the tradeoffs between accuracy, computational efficacy, storage costs, and maintainability. This work presents an iris indexing approach based on rotation invariant iris representation and LSH-Forest to produce an accurate and easily maintainable indexing structure. The complexity of insertion or deletion in the proposed method is limited to the same logarithmic complexity of a query and the required storage grows linearly with the database size. The proposed approach was extended into a multi-instance iris indexing scheme resulting in a clear performance improvement. Single iris indexing scored a hit rate of 99.7% at a 0.1% penetration rate while multi-instance indexing scored a 99.98% hit rate at the same penetration rate. The evaluation of the proposed approach was conducted on a large database of 50k references and 50k probes of the left and the right irises. The advantage of the proposed solution was put into prospective by comparing the achieved performance to the reported results in previous works.
Neighbor Distance Ratios and Dynamic Weighting in Multi-biometric Fusion
Pattern Recognition and Image Analysis
Iberian Conference on Pattern Recognition and Image Analysis (IbPRIA) <8, 2017, Faro, Portugal>
Multi-biometrics aims at building more accurate unified biometric decisions based on the information provided by multiple biometric sources. Information fusion is used to optimize the process of creating this unified decision. In previous works dealing with score-level multibiometric fusion, the scores of different biometric sources belonging to the comparison of interest are used to create the fused score. This is usually achieved by assigning static weights for the different biometric sources. In contrast, we focus on integrating the information imbedded in the relative relation between the comparison scores (within a 1:N comparison) in the biometric fusion process using a dynamic weighting scheme. This is performed by considering the neighbors distance ratio in the ranked comparisons to influence the dynamic weights of the fused scores. The evaluation was performed on the Biometric Scores Set BSSR1 database. The enhanced performance induced by including the neighbors distance ratio information within a dynamic weighting scheme in comparison to the baseline solution was shown by an average reduction of the equal error rate by more than 40% over the different test scenarios.
Opportunities for Biometric Technologies in Smart Environments
European Conference on Ambient Intelligence (AmI) <13, 2017, Malaga, Spain>
Smart environments describe spaces that are equipped with sensors, computing facilities and output systems that aim at providing their inhabitants with targeted services and supporting them in their tasks. Increasingly these are faced with challenges in differentiating multiple users and secure authentication. This paper outlines how biometric technologies can be applied in smart environments to overcome these challenges. We give an introduction to these domains and show various applications that can benefit from the combination of biometrics and smart environments.
Trust the Biometric Mainstream: Multi-biometric Fusion and Score Coherence
2017 Proceedings of the 25th European Signal Processing Conference (EUSIPCO)
European Signal Processing Conference (EUSIPCO) <25, 2017, Kos, Greece>
Multi-biometrics aims at building more accurate unified biometric decisions based on the information provided by multiple biometric sources. Information fusion is used to optimize the process of creating this unified decision. In previous works dealing with score-level multi-biometric fusion, the scores of different biometric sources belonging to the comparison of interest are used to create the fused score. This is usually achieved by assigning static weights for the different biometric sources with more advanced solutions considering supplementary dynamic information like sample quality and neighbours distance ratio. This work proposes embedding score coherence information in the fusion process. This is based on our assumption that a minority of biometric sources, which points out towards a different decision than the majority, might have faulty conclusions and should be given relatively smaller role in the final decision. The evaluation was performed on the BioSecure multimodal biometric database with different levels of simulated noise. The proposed solution incorporates, and was compared to, three baseline static weighting approaches. The enhanced performance induced by including the coherence information within a dynamic weighting scheme in comparison to the baseline solution was shown by the reduction of the equal error rate by 45% to 85% over the different test scenarios and proved to maintain high performance when dealing with noisy data.
Combining Low-level Features of Offline Questionnaires for Handwriting Identification
Image Analysis and Recognition
International Conference on Image Analysis and Recognition (ICIAR) <13, 2016, Póvoa de Varzim, Portugal>
When using anonymous offline questionnaires for reviewing services or products it is often not guaranteed that a reviewer does this only once as intended. In this paper an applied combination of different features of handwritten characteristics and its fusion is presented to expose such manipulations. The presented approach covers the aspects of alignment normalization, segmentation, feature extraction, classification and fusion. Nine features from handwritten text, numbers and checkboxes are extracted and used to recognize handwriter duplicates. The proposed method has been tested on a novel database containing pages of handwritten text produced by 1,734 writers. Furthermore we show that the unified biometric decision using a weighted sum combination rule can significantly improve writer identification performance even on low level features.
Multi-biometric Continuous Authentication: a Trust Model for an Asynchronous System
International Conference on Information Fusion (FUSION) <19, 2016, Heidelberg, Deutschland>
Biometric technologies are used to grant specific users access to services and data. The access control is usually performed at the start of a session that spans over a period of time. Continuous authentication aims at insuring the identity of the user over this period of time, and not only at its start. Multi-biometrics aims at increasing the accuracy, robustness and usability of biometrics systems. This work presents a multibiometric continuous authentication solution that includes information from the face images and the keystroke dynamics of the user. A database representing a realistic scenario was collected to develop and evaluate the presented solution. A multi-biometric trust model was designed to cope with the asynchronous nature induced by the different biometric characteristics. A set of performance metrics are discussed and a comparison is presented between the performances of the single characteristic solutions and the fused solution.
Practical View on Face Presentation Attack Detection
Proceedings of the British Machine Vision Conference 2016 [Online]
British Machine Vision Conference (BMVC) <27, 2016, York, UK>
Face recognition is one of the most socially accepted forms of biometric recognition. The recent availability of very accurate and efficient face recognition algorithms leaves the vulnerability to presentation attacks as the major challenge to face recognition solutions. Previous works have shown high preforming presentation attack detection PAD solutions under controlled evaluation scenarios. This work tried to analyze the practical use of PAD by investigating the more realistic scenario of cross-database evaluation and presenting a state-of-the-art performance comparison. The work also investigated the relation between the video duration and the PAD performance. This is done along with presenting an optical flow based approach that proves to outperform state-of-the-art solutions in most experiment settings.
Fuzzy Logic and Multi-biometric Fusion
International Conference on Pattern Recognition Applications and Methods (ICPRAM) <4, 2015, Lisbon, Portugal>
Fuzzy logic has been proposed to improve various aspects of multi-biometric applications including enhancements to the decision making of the application and the robustness to noisy data. This paper discusses recent work that utilized fuzzy logic techniques within the multi-biometric fusion problem. This discussion is presented under two categories, the type of authentication scenario and the nature of the fused data. The paper also presents an introduction to fuzzy logic and multi-biometric fusion. Based on the discussed works, this paper aims to establish current trends and research possibilities in this field.
Multi-Biometric Continuous Authentication
Darmstadt, Hochschule, Master Thesis, 2015
Biometric recognition can be used to secure the access to a system, by recognizing individuals seeking access, based on their behavioural and biological characteristics. In some scenarios, this level of security is not high enough, since it leaves room for attackers to gain access to the system after the initial recognition. Continuous authentication can be used to solve this problem by monitoring the current user during the work session. A genuine user with legitimate access should not be interrupted during the working session. Thus, biometric characteristics which require interaction with sensors are not suited for continuous authentication systems. As a consequence, research has been focused on behavioural biometric characteristics. A trust model defines the behaviour of the continuous authentication system by describing how actions of the user affect the trust value. Decisions are based on this trust value. This work aims to research whether a trust model can be used to combine a biological biometric characteristic and a behavioural characteristic, namely face recognition as the biological component and keystroke dynamics as the behavioural component. Face recognition was chosen because it does neither require additional interaction with a sensor, nor does it interrupt the work session of the genuine user. In order to lessen the impact on the privacy of the user, it was decided to use periodically taken pictures from a webcam instead of a permanent video surveillance. This added the challenge of the information collected by the system being asynchronous. The goal of this work is to develop and evaluate the feasibility and performance of such a system. In order to evaluate this proposed system a database of biometric data suitable for the application scenario was collected and a prototype of the system developed. Face recognition was implemented by using a Local Binary Linear Discriminant Analysis (LBLDA), for keystroke dynamics, a statistical method was implemented. Results show clear improvements in one metric, while the results in the other three measured metrics fell in a range between those of the unfused components. However, results can be further improved by using a more sophisticated fusion approach and tuning the sub components.
Personalized Face Reference from Video: Key-Face Selection and Feature-Level Fusion
Face and Facial Expression Recognition from Real World Videos
International Workshop on Face and Facial Expression Recognition from Real World Videos (FFER) <2014, Stockholm, Sweden>
Face recognition from video in uncontrolled environments is an active research field that received a growing attention recently. This was mainly driven by the wide range of applications and the availability of large databases. This work presents an approach to create a robust and discriminant reference face model from video enrollment data. The work focuses on two issues, first is the key faces selection from video sequences. The second is the feature-level fusion of the key faces. The proposed fusion approaches focus on inducing subject specific feature weighting in the reference face model. Quality based sample weighting is also considered in the fusion process. The proposed approach is evaluated under different sittings on the YouTube Faces data-base and the performance gained by the proposed approach is shown in the form of EER values and ROC curves.
Utilizing Fuzzy Distances in Biometric Comparison
Darmstadt, Hochschule, Bachelor Thesis, 2015
In recent years, the use of biometric systems has dramatically increased and offers substantial improvement in recognizing individuals. The field of biometrics aims at recognizing individuals using their biological and physical characteristics. These characteristics play an important role because they are not easily changed or misused. Furthermore, they cannot be forgotten as is the case for passwords, nor can they be lost in the same manner as identification cards. The increasing demand for biometrics has motivated a lot of research in this field of study, which aim at finding improved and more accurate methods of biometric recognition. All biometric systems use biometric comparison to build their decisions. This comparison process is usually done by calculating the distance between the two feature vectors, each represents the biometric characteristics of different acquisition. Euclidean distance is one of the most commonly used distance measure. Other distance measures such as Manhattan Distance, Canberra Distance, Chebyshev Distance and Cosine Metric was used as the baseline distances in this thesis. The aim of this thesis is to construct a fuzzy distance for biometric comparison and to compare its performance with the baseline distances mentioned above. The main purpose behind a fuzzy logic system is that it allows partial set membership instead of crisp membership. Fuzzy logic system has been successfully being implemented in a lot of appliances that require artificial intelligence. The dataset used in this thesis is called "Labeled Faces in the Wild" gotten from a database of face photographs designed for studying the problem of unconstrained face recognition. The database contains over 13,000 images of faces collected online. Face recognition can be seen as a process of recognizing individuals using their facial characteristics. After implementing the proposed fuzzy distance, its performance was compared with the baseline distances using EER values. Results show that the proposed fuzzy distance is also a good distance measure for biometric comparison.
Weighted Integration of Neighbors Distance Ratio in Multi-biometric Fusion
Annual International Conference of the Biometrics Special Interest Group (BIOSIG) <14, 2015, Darmstadt, Germany>
This work presents an approach to integrate biometric source weighting in the calculation of neighbors distance ratios to be used within a classification-based multi-biometric fusion process. The neighbors distance ratio represents the elevation of the top ranked identification match to the following ranks. Using biometric source weighing can help achieve more accurate initial identity ranking necessary for neighbors distance ratios. It also influences the effect of each biometric source on the ratios values. The proposed approach is developed and evaluated using the Biometric Scores Set BSSR1 database. The results are presented in the verification scenario as receiver operating curves (ROC). The achieved performance is compared to a number of baseline solutions and a satisfying and stable performance was achieved with a clear benefit of integrating the biometric source weights.
Biometric Source Weighting in Multi-Biometric Fusion: Towards a Generalized and Robust Solution
2014 Proceedings of the 22nd European Signal Processing Conference (EUSIPCO)
European Signal Processing Conference (EUSIPCO) <22, 2014, Lisbon, Portugal>
This work presents a new weighting algorithm for biometric sources within a score-level multi-biometric system. Those weights are used in the effective and widely used weighted sum fusion rule to produce multi-biometric decisions. The presented solution is mainly based on the characteristic of the overlap region between the genuine and imposter scores distributions. It also integrates the performance of the biometric source represented by its equal error rate. This solution aims at avoiding the shortcomings of previously proposed solutions such as low generalization abilities and sensitiveness to outliers. The proposed solution is evaluated along with the state of the art and best practice techniques. The evaluation was performed on two databases, the Biometric Scores Set BSSR1 and the Extended Multi Modal Verification for Teleservices and Security applications database and a satisfying and stable performance was achieved.
CMC Curve Properties and Biometric Source Weighting in Multi-Biometric Score-level Fusion
International Conference on Information Fusion (FUSION) <17, 2014, Salamanca, Spain>
Multi-biometrics tries to build a unified biometric decision based on multiple biometric sources in an effort to gain more accuracy and robustness. Multi-biometric fusion aims at optimally combining the information produced by the multiple biometric sources, this usually requires assigning relative weights for the biometric sources to optimize their effect on the final decision. This work presents a new approach for biometric sources weighting within a score-level multi-biometric system. The presented solution tries to investigate the properties of the cumulative match characteristic (CMC) curve, which represents the biometric performance under the identification scenario, and extract biometric source weights based on those properties. The proposed solution is evaluated along with a set of state of the art and best practice weighting techniques. The evaluation was performed on the Biometric Scores Set BSSR1 database and a satisfying and stable performance was achieved.
Gesichtserkennungssoftware in Bildarchiven - Eine Untersuchung zur Nutzung von Gesichtserkennungsprogrammen für die archivische Kernaufgabe "Erschließung"
Potsdam, FH, Bachelor Thesis, 2014
Diese Arbeit betrachtet zunächst die theoretischen Grundlagen für die Gesichtserkennung. Das sind zum einen Erklärungen zur Archivale Foto, auch unter dem Aspekt der Fotoarchivierung und der archivischen Erschließung und die Biometrie bzw. Biometrik. Zum anderen ist es der Ablauf der Gesichtserkennung sowie die Algorithmen, welche häufig Anwendung hierbei finden. Mit Hilfe dieser Informationen werden verschiedene Softwareprogramme einer Untersuchung, die vorab ausführlich erläutert wird, zur Nutzung von Gesichtserkennungsprogrammen für die archivische Kernaufgabe "Erschließung" unterzogen. Die Ergebnisse dieser Untersuchung sollen aufzeigen, ob und inwiefern Gesichtserkennungsprogramme eine qualitative und quantitative Verbesserung in Bezug auf die Datensicherung (Personennennung) und Vereinfachung der Erschließung in Bildarchiven erzielen könnten. Eine Empfehlung resp. ein Ausblick zu diesem Thema vollenden die Arbeit.
Helper Data Scheme for 2D Cancelable Face Recognition Using Bloom Filters
IWSSIP 2014. Proceedings
International Conference on Systems, Signals and Image Processing (IWSSIP) <21, 2014, Dubrovnik, Croatia>
Biometrics provide a source of automated recognition of individuals based on their physiological and behavioral characteristics. As per Directive 95/46/EC, biometric data is considered to be personal data. And according to article 8 of the European Convention on Human Rights, personal data needs to be privacy preserved. Biometric template protection mechanisms provide a privacy preserved biometric authentication. Such mechanisms assist irreversibility, revocability and unlinkability of biometric templates. Recently, a bloom filter based approach was proposed to generate irreversible iris template. In this paper, a helper data scheme for 2D cancelable face verification using bloom filters is proposed. The positions of most representative features (stable features) are used as helper data, which helps in the face recognition. The features used are extracted using Local Binary Linear Discriminant Analysis. The effect of stable features on recognition performance under scenarios of with and without using bloom filters is investigated. In addition, recognition performance after compressing multiple features into a single bloom filter is presented. The results are experimentally proved on two benchmark databases namely LFW and ORL datasets.
Multi-biometric Score-Level Fusion and the Integration of the Neighbors Distance Ratio
Image Analysis and Recognition. Proceedings Part II
International Conference on Image Analysis and Recognition (ICIAR) <11, 2014, Vilamoura, Portugal>
Multi-biometrics aims at building more accurate unified biometric decisions based on the information provided by multiple biometric sources. Information fusion is used to optimize the process of creating this unified decision. In previous works dealing with score-level multibiometric fusion, the scores of different biometric sources belonging to the comparison of interest are used to create the fused score. The novelty of this work focuses on integrating the relation of the fused scores to other comparisons within a 1:N comparison. This is performed by considering the neighbors distance ratio in the ranked comparisons set within a classification-based fusion approach. The evaluation was performed on the Biometric Scores Set BSSR1 database and the enhanced performance induced by the integration of neighbors distance ratio was clearly presented.
Multimedia Analysis of Video Sources
SIGMAP 2014. Proceedings of the 11th International Conference on Signal Processing and Multimedia Applications
International Conference on Signal Processing and Multimedia Applications (SIGMAP) <11, 2014, Vienna, Austria>
Law Enforcement Agencies (LEAs) spend increasing efforts and resources on monitoring open sources, searching for suspicious behaviours and crime clues. The task of efficiently and effectively monitoring open sources is strongly linked to the capability of automatically retrieving and analyzing multimedia data. This paper presents a multimodal analytics system, created in cooperation with European LEAs. In particular it is described how the video analytics subsystem produces a workflow of multimedia data analysis processes. After a first analysis of video files, images are extracted in order to perform image comparison, classification and face recognition. In addition, audio content is extracted to perform speaker recognition and multilingual analysis of text transcripts. The integration of multimedia analysis results allows LEAs to extract pertinent knowledge from the gathered information.
Privacy Preserved Duplicate Check using Multi-biometric Fusion
International Conference on Information Fusion (FUSION) <17, 2014, Salamanca, Spain>
Automated recognition of individuals can be performed using biometrics without any requirement of explicit knowledge of a PIN or a password. On the one side biometrics has given convenience to citizens as they do not need to memorize a bunch of passwords, but on the other side intra (inter) class variations within (between) biometric features makes biometric authentications untrustworthy. Therefore, decisions based on biometric authentications are made more reliable by using several biometric authentications performed on single or multiple biometric modalities (i.e. multi-biometric fusion). This paper describes a method to identify if a person tries to re-enrol him/herself in a database, when he/she is already enrolled. This is referred to as duplicate check. In this work, duplicate check is performed using two modalities: face and iris. The templates used during the duplicate check are compliant to the ISO/IEC 24745 - Biometric information protection. Such templates are known as protected biometric templates. The protected biometric templates used in this work are generated using the recently published irreversible transformation based on Bloom filters. Scores are calculated from face and iris Bloom filters based templates by comparison with their respective enrolment templates using the normalized Hamming distance. As a decision of the duplicate check, these scores from both modalities are fused with appropriate weighting factors in order to get improved performance compared to using single individual modalities. The presented scheme is experimentally validated using two public benchmark databases namely the LFW and the CASIA-Iris-Thousand databases for face and iris respectively.
Prototypical Development of an In-Shop Advertisment System using Body Dimension Recognition
Darmstadt, Hochschule, Master Thesis, 2014
This thesis outlines a system created to give consumers in the fashion industry an idea of how an item of clothing will look on them before trying it on. In the form of a short video, items of clothing are projected virtually onto an image of the user. Through the use of this system, retailers and manufacturers have the chance to immediately display their clothes on potential customers.
Three Dimensional Scanning of Clothes, for Simulation and Presentation Purposes in a Virtual Fitting Room
Darmstadt, Hochschule, Master Thesis, 2014
The aim of this thesis is to develop a low-cost semi-professional automated 3D scanning and post-production system for digitizing clothing and apparel for in shop and online presentation purposes. Ultimately giving birth to a database of digitized 3d models of apparel to enable virtual-fitting rooms and real-time fitting feedback. In the first part different scanning methods are tested if they are suited for scanning apparel and if the quality is good enough for advertisement and presentation purposes. The cost of the system is also taken into account. The thesis then identifies the best and most cost effective approach and tries to develop and automate the method using state of the art consumer products. In the main section the thesis describes the functionality of the method and how it can be applied. Different algorithms and workflows are shown and combined to develop the automated system. In conclusion the thesis describes and summarizes the system and opens up how it could be implemented in a consumer oriented presentation like a virtual fitting room or an online shopping style advisor using the users body-metrics.
Virtual Fitting Pipeline: Body Dimension Recognition, Cloth Modeling, and On-Body Simulation
VRIPHYS 14: 11th Workshop in Virtual Reality Interactions and Physical Simulations
International Workshop in Virtual Reality Interaction and Physical Simulations (VRIPHYS) <11, 2014, Bremen, Germany>
This paper describes a solution for 3D clothes simulation on human avatars. The proposed approach consists of three parts, the collection of anthropometric human body dimensions, cloths scanning, and the simulation on 3D avatars. The simulation and human machine interaction has been designed for application in a passive In- Shop advertisement system. All parts have been evaluated and adapted under the aim of developing a low-cost automated scanning and post-production system. Human body dimension recognition was achieved by using a landmark detection based approach using both two 2D and 3D cameras for front and profile images. The human silhouettes extraction solution based on 2D images is expected to be more robust to multi-textured background surfaces than existing solutions. Eight measurements corresponding to the norm of body dimensions defined in the standard EN-13402 were used to reconstruct a 3D model of the human body. The performance is evaluated against the ground-truth of our newly acquired database. For 3D scanning of clothes, different scanning methods have been evaluated under apparel, quality and cost aspects. The chosen approach uses state of the art consumer products and describes how they can be combined to develop an automated system. The scanned cloths can be later simulated on the human avatars, which are created based on estimation of human body dimensions. This work concludes with software design suggestions for a consumer oriented solution such as a virtual fitting room using body metrics. A number of future challenges and an outlook for possible solutions are also discussed.
An Overview on Multi-biometric Score-level Fusion
International Conference on Pattern Recognition Applications and Methods (ICPRAM) <2, 2013, Barcelona, Spain>
Multi-biometrics is the use of multiple biometric recognition sources to provide a more dependable verification or identification decision. Fusion of multi-biometric sources can be performed on different levels, such as the data, feature, or score level. This work presents an overview of the multi-biometric score-level fusion problem, along with the proposed solution in the literature. A discussion is made to provide a comparison between multi-biometric fusion in both scenarios. This discussion aims at providing a clearer view of future developments especially under the identification scenario where many related applications are rapidly growing such as forensics and ubiquitous surveillance.
Face Liveness Detection Against Image and Video Spoofing Attacks
Darmstadt, TU, Master Thesis, 2013
Many situations require users to log into a computer system. For example to perform private tasks like banking, social interaction or to get access to a secured area. Conventional security driven systems have the disadvantage that passwords or keycards are needed. These passwords or keycards can get lost or stolen resulting in a security risk. To overcome this drawback biometrics use the characteristics of the human body to grand access to a computer system. Beside fingerprint or iris recognition face detection is a popular biometric trait. The reason therefore is that it requires only a usual camera. Most of the current systems have a camera build in anyway. Also face recognition is not very intrusive to the user, which gives a high acceptability of face recognition is biometric trait. In the past though face recognition systems could easily be tricked due to spoofing attempts using pictures or videos of the authenticate user. This thesis analyses current algorithms to counter such spoofing attempts and presents a novel approach. The presented approach will use Machine Learning and Computer Vision to utilize an algorithms by Wu et al. [WRS_12] that can magnify subtle changes in videos to reveal the human pulse. An evaluation of the feasibility of this approach will be given.
Missing Data Estimation in Multi-biometric Identification and Verification
2013 IEEE Workshop on Biometric Measurements and Systems for Security and Medical Applications
IEEE Workshop on Biometric Measurements and Systems for Security and Medical Applications (BioMS) <4, 2013, Naples, Italy>
In the practical use of multi-biometric solutions, biometric sources involved in producing the verification or identification decision do occasionally fail to produce results. This work discusses solutions for missing data in multi-biometric score-level fusion. A missing data estimation solution based on support vector regression was presented in this work and compared to four baseline solutions. The evaluation was carried under both the verification and the identification scenarios in an effort to show the effect of missing data estimation on the relatively understudied multi-biometric identification scenario. Evaluation was performed on the Biosecure DS2 score database and satisfying performance was achieved under both biometric scenarios.
Performance Anchored Score Normalization for Multi-Biometric Fusion
Advances in Visual Computing. 9th International Symposium, ISVC 2013
International Symposium on Visual Computing (ISVC) <9, 2013, Rethymnon, Crete, Greece>
This work presents a family of novel normalization techniques for score-level multi-biometric fusion. The proposed normalization is not only concerned to bring comparison scores to a common range and scale, it also focuses in bringing certain operational performance points in the distribution into alignment. The Performance Anchored Normalization (PAN) algorithms discussed here were tested on the extended Multi Modal Verification for Teleservices and Security applications database (XM2VTS) and proved to outperform conventional score normalization techniques in most tests. The tests were performed with combination fusion rules and presented as biometric verification performance measures.
Qualitätsbasierte Informationsfusion auf der Bewertungsebene innerhalb der multimodalen biometrischen Identifikation
Darmstadt, TU, Master Thesis, 2013
Im Laufe der Zeit haben sich biometrische Erkennungsverfahren als zuverlässiges Mittel zum Zwecke der Zugangskontrolle zu physikalischen und virtuellen Bereichen entwickelt. Die Schwächen unimodaler Systeme werden dabei oft durch multimodale Ansätze verbessert, insbesondere ermöglichen diese einen robusteren Registrierungsprozess, erhöhte Sicherheit gegenüber gefälschten Identitäten und eine höhere Erkennungsgenauigkeit. Die vorliegende Arbeit beschäftigt sich mit der Frage, wie Qualitätsinformationen über die extrahierten Merkmale zu einer weiteren Verbesserung der multimodalen biometrischen Identifikation beitragen können. Dabei liegt die Idee zugrunde, dass Merkmale von höherer Qualität auch eine höhere Zuverlässigkeit im Hinblick auf deren Klassifikation zusichern, diese also stärker in den Entscheidungsprozess eingebunden werden sollten als Merkmale von geringerer Qualität. Zur Überprüfung dieser Annahme wurde ein auf dem Gradientenverfahren basierender Fusionsmechanismus um die Berücksichtigung von Qualitätsinformationen erweitert und dessen Erkennungsperformanz unter verschiedenen Konditionen, darunter im Besonderen das Vorhandensein fehlender Bewertungsmaße, mit dem ursprünglichen Algorithmus verglichen.
The 2nd Competition on Counter Measures to 2D Face Spoofing Attacks
2013 International Conference on Biometrics (ICB)
IAPR International Conference on Biometrics (ICB) <6, 2013, Madrid, Spain>
As a crucial security problem, anti-spoofing in biometrics, and particularly for the face modality, has achieved great progress in the recent years. Still, new threats arrive in form of better, more realistic and more sophisticated spoofing attacks. The objective of the 2nd Competition on Counter Measures to 2D Face Spoofing Attacks is to challenge researchers to create counter measures effectively detecting a variety of attacks. The submitted propositions are evaluated on the Replay-Attack database and the achieved results are presented in this paper.
Untersuchung von Normalisierungs- und Kombinationsverfahren im Kontext von multibiometrischer score level Fusion
Darmstadt, Hochschule, Bachelor Thesis, 2013
Diese Arbeit präsentiert eine Untersuchung von verschiedenen Normalisierungs- und Kombinationsverfahren im Kontext von multibiometrischer score level Fusion. Hierbei werden zur biometrischen Erkennung von Personen mehrere Informationen verwendet und zu einem eindeutigen Ergebnis verschmolzen. Die Effizienz dieser Verfahren wurde im Rahmen dieser Arbeit mittels einer Anwendung und einem konkretem Datensatz evaluiert. Die Evaluierung wurde für beide gängigen Modi einer biometrischen Erkennung, der Identifizierung und Verifikation, durchgeführt. Die Resultate zeigen, dass die Wahl der Verfahren einen großen Einfluss auf die Effizienz der biometrischen Erkennung ausübt. Im Verifikationsmodus wurde im schlechtesten Fall eine Erkennungsrate von 28.34% erzielt. Im besten Fall hingegen betrug die Erkennungsrate 95.54%. Die niedrigste Erkennungsrate im Identifikationsmodus betrug 51.92% und die höchste 98.71%.
2D Face Liveness Detection: an Overview
Annual International Conference of the Biometrics Special Interest Group (BIOSIG) <11, 2012, Darmstadt, Germany>
Face recognition based on 2D images is a widely used biometric approach. This is mainly due to the simplicity and high usability of this approach. Nonetheless, those solutions are vulnerable to spoof attacks made by non-real faces. In order to identify malicious attacks on such biometric systems, 2D face liveness detection approaches are developed. In this work, face liveness detection approaches are categorized based on the type of liveness indicator used. This categorization helps understanding different spoof attacks scenarios and their relation to the developed solutions. A review of the latest works dealing with face liveness detection works is presented. A discussion is made to link the state of the art solutions with the presented categorization along with the available and possible future datasets. All that aim to provide a clear path for the future development of innovative face liveness detection solutions.
Ear Recognition Using Multi-Scale Histogram of Oriented Gradients
Eighth International Conference on Intelligent Information Hiding and Multimedia Signal Processing. Proceedings
International Conference on Intelligent Information Hiding and Multimedia Signal Processing (IIH-MSP) <8, 2012, Piraeus-Athens, Greece>
Ear recognition is a promising biometric measure, especially with the growing interest in multi-modal biometrics. Histogram of Oriented Gradients (HOG) has been effectively and efficiently used solving the problems of object detection and recognition, especially when illumination variations are present. This work presents a robust approach for ear recognition using multi-scale dense HOG features as a descriptor of 2D ear images. The multi-scale features assure to capture the different and complicated structures of ear images. Dimensionality reduction was performed to avoid feature redundancy and provide a more efficient recognition process while being prone to over-fitting. Finally, a test was performed on a large and realistic database and the results were compared to the state of the art ear recognition approaches tested on the same dataset and under the same test procedure.