Application-driven Advances in Multi-biometric Fusion
Darmstadt, TU, Diss., 2018
Biometric recognition is the automated recognition of individuals based on their behavioral or biological characteristics. Beside forensic applications, this technology aims at replacing the outdated and attack prone, physical and knowledge-based, proofs of identity. Choosing one biometric characteristic is a tradeoff between universality, acceptability, and permanence, among other factors. Moreover, the accuracy cap of the chosen characteristic may limit the scalability and usability for some applications. The use of multiple biometric sources within a unified frame, i.e. multi-biometrics, aspires to tackle the limitations of single source biometrics and thus enables a wider implementation of the technology. This work aims at presenting application-driven advances in multi-biometrics by addressing different elements of the multi-biometric system work-flow. At first, practical oriented pre-fusion issues regarding missing data imputation and score normalization are discussed. This includes presenting a novel performance anchored score normalization technique that aligns certain performance-related score values in the fused biometric sources leading to more accurate multi-biometric decisions when compared to conventional normalization approaches. Missing data imputation within scorelevel multi-biometric fusion is also addressed by analyzing the behavior of different approaches under different operational scenarios. Within the multi-biometric fusion process, different information sources can have different degrees of reliability. This is usually influenced in the fusion process by assigning relative weights to the fused sources. This work presents a number of weighting approaches aiming at optimizing the decision made by the multi-biometric system. First, weights that try to capture the overall performance of the biometric source, as well as an indication of its confidence, are proposed and proved to outperform the state-of-the-art weighting approaches. The work also introduces a set of weights derived from the identification performance representation, the cumulative match characteristics. The effect of these weights is analyzed under the verification and identification scenarios. To further optimize the multi-biometric process, information besides the similarity between two biometric captures can be considered. Previously, the quality measures of biometric captures were successfully integrated, which requires accessing and processing raw captures. In this work, supplementary information that can be reasoned from the comparison scores are in focus. First, the relative relation between different biometric comparisons is discussed and integrated in the fusion process resulting in a large reduction in the error rates. Secondly, the coherence between scores of multi-biometric sources in the same comparison is defined and integrated into the fusion process leading to a reduction in the error rates, especially when processing noisy data. Large-scale biometric deployments are faced by the huge computational costs of running biometric searches and duplicate enrollment checks. Data indexing can limit the search domain leading to faster searches. Multibiometrics provides richer information that can enhance the retrieval performance. This work provides an optimizable and configurable multi-biometric data retrieval solution that combines and enhances the robustness of rank-level solutions and the performance of feature-level solutions. Furthermore, this work presents biometric solutions that complement and utilize multi-biometric fusion. The first solution captures behavioral and physical biometric characteristics to assure a continuous user authentication. Later, the practical use of presentation attack detection is discussed by investigating the more realistic scenario of cross-database evaluation and presenting a state-of-the-art performance comparison. Finally, the use of multibiometric fusion to create face references from videos is addressed. Face selection, feature-level fusion, and score-level fusion approaches are evaluated under the scenario of face recognition in videos.
Creating Face Morphing Attacks with Generative Adversarial Networks
Saint-Étienne, Univ., Master Thesis, 2018
Nowadays, the use of technologies related to biometrics is increasing significantly. The recent deep learning improvement in face recognition tasks, along with the relatively high social acceptance, have pushed automatic face recognition systems to be a key technology in identity verification in border controls. In this scenario, face recognition is used to link the identity of a passenger to their e-document. However, recent studies have highlighted the threat of morphing attacks against automatic face recognition systems. In this thesis, we present a novel morphed face attack, called MorGANA, created by using Generative Adversarial Networks. By creating a new morphing database, MorGAN dataset, we investigate the vulnerabilities of current face recognition systems against MorGANA attacks, alongside with baseline attacks. Moreover, the morphing detectability under face morphing attacks is further studied noticing an insufficient performance from common detectors. Examining the obtained results, we strongly consider that further studies need to be addressed in generating and detecting morphed face attacks in order to bring face recognition systems to real-case scenarios.
Cross-device Biometric Verification and the Benefit of Multi-algorithmic Biometric Fusion
Frankfurt am Main, Univ. of Applied Science, Master Thesis, 2018
Biometrics is a technology that aims to identify or verify people identities based on their physical characteristics or behavioral properties. Multi-biometrics is implemented to use a number of biometric information sources to create a unified decision to increase the accuracy of the biometric system. This aims at enhancing performance and avoiding the shortcomings of conventional single source biometrics such as sensitivity to noisy data or capture environment while maintaining high accuracy. One of these shortcomings are the negative effect of cross-device biometric verification, e.g. using different smart phones and capture devices for the enrolment and verification of face images. The previous work deals with multi-biometrics to obtain the fused score by incorporating the coherence information on the biometric sources to enhance the performance of the Biometric system. In this thesis the main purpose is to improve the performance of the Biometric system for multi-modality (two modalities) with each biometric source acquired from different sensor devices which includes both same sensor and different sensor combinations. This includes biometric sources, i.e. face and one of the periocular region fed to the biometric system with multi-algorithmic approach to extract the features and create unified decision at the score level. Analysis of different realistic scenario with varying static and dynamic weights and its effect on the unified decision on score level. This approach could be used in biometric system with more than two modalities.
Deep and Multi-algorithmic Gender Classification of Single Fingerprint Minutiae
International Conference on Information Fusion (FUSION) <21, 2018, Cambridge, UK>
Accurate fingerprint gender estimation can positively affect several applications, since fingerprints are one of the most widely deployed biometrics. For example, gender classification in criminal investigations may significantly minimize the list of potential subjects. Previous work mainly offered solutions for the task of gender classification based on complete fingerprints. However, partial fingerprint captures are frequently occurring in many applications, including forensics and the fast growing field of consumer electronics. Moreover, partial fingerprints are not well-defined. Therefore, this work improves the gender decision performance on a well-defined partition of the fingerprint. It enhances gender estimation on the level of a single minutia. Working on this level, we propose three main contributions that were evaluated on a publicly available database. First, a convolutional neural network model is offered that outperformed baseline solutions based on hand crafted features. Second, several multi-algorithmic fusion approaches were tested by combining the outputs of different gender estimators that help further increase the classification accuracy. Third, we propose including minutia detection reliability in the fusion process, which leads to enhancing the total gender decision performance. The achieved gender classification performance of a single minutia is comparable to the accuracy that previous work reported on a quarter of aligned fingerprints including more than 25 minutiae.
Deep Learning-based Face Recognition and the Robustness to Perspective Distortion
24th International Conference on Pattern Recognition. Proceedings
International Conference on Pattern Recognition (ICPR) <24, 2018, Beijing, China>
Face recognition technology is spreading into a wide range of applications. This is mainly driven by social acceptance and the performance boost achieved by the deep learningbased solutions in the recent years. Perspective distortion is an understudied distortion in face recognition that causes converging verticals when imaging 3D objects depending on the distance to the object. The effect of this distortion on face recognition was previously studied for algorithms based on hand-crafted features with a clear negative effect on verification performance. Possible solutions were proposed by compensating the distortion effect on the face image level, which requires knowing the camera settings and capturing a high quality image. This work investigates the effect of perspective distortion on the performance of a deep learning-based face recognition solution. It also provides a device parameter-independent solution to decrease this effect by creating more perspective-robust face representations. This was achieved by training the deep learning model on perspective-diverse data, without increasing the size of the training data. Experiments performed on the deep model in hand and a specifically collected database concluded that the perspective distortion effects face verification performance if not considered in the training process, and that this can be improved by our proposal of creating robust face representations by properly selecting the training data.
Fingerprint and Iris Multi-biometric Data Indexing and Retrieval
International Conference on Information Fusion (FUSION) <21, 2018, Cambridge, UK>
Indexing of multi-biometric data is required to facilitatefast search in large-scale biometric systems. Previous worksaddressing this issue in multi-biometric databases focused onmulti-instance indexing, mainly iris data. Few works addressedthe indexing in multi-modal databases, with basic candidate listfusion solutions limited to joining face and fingerprint data. Irisand fingerprint are widely used in large-scale biometric systemswhere fast retrieval is a significant issue. This work proposes jointmulti-biometric retrieval solution based on fingerprint and irisdata. This solution is evaluated under eight different candidatelist fusion approaches with variable complexity on a databaseof 10,000 reference and probe records of irises and fingerprints.Our proposed multi-biometric retrieval of fingerprint and irisdata resulted in a reduction of the miss rate (1- hit rate) at 0.1%penetration rate by 93% compared to fingerprint indexing and88% compared to iris indexing.
P-score: Performance Aligned Normalization and an Evaluation in Score-level Multi-biometric Fusion
2018 Proceedings of the 26th European Signal Processing Conference (EUSIPCO)
European Signal Processing Conference (EUSIPCO) <26, 2018, Rome, Italy>
Normalization is an important step for different fusion, classification, and decision making applications. Previous normalization approaches considered bringing values from different sources into a common range or distribution characteristics. In this work we propose a new normalization approach that transfers values into a normalized space where their relative performance in binary decision making is aligned across their whole range. Multi-biometric verification is a typical problem where information from different sources are normalized and fused to make a binary decision and therefore a good platform to evaluate the proposed normalization.We conducted an evaluation on two publicly available databases and showed that the normalization solution we are proposing consistently outperformed state-of-the-art and best practice approaches, e.g. by reducing the false rejection rate at 0.01% false acceptance rate by 60- 75% compared to the widely used z-score normalization under the sum-rule fusion.
The Dark Side of the Face: Exploring the Ultraviolet Spectrum for Face Biometrics
2018 International Conference on Biometrics (ICB)
IAPR International Conference on Biometrics (ICB) <11, 2018, Gold Coast, Australia>
Facial recognition in the visible spectrum is a widelyused application but it is also still a major field of research.In this paper we present melanin face pigmentation (MFP)as a new modality to be used to extend classical face biometrics. Melanin pigmentation are sun-damaged cells thatoccur as revealed and/or unrevealed pattern on human skin.Most MFP can be found in the faces of some people whenusing ultraviolet (UV) imaging. To proof the relevance ofthis feature for biometrics, we present a novel image datasetof 91 multiethnic subjects in both, the visible and the UVspectrum. We show a method to extract the MFP featuresfrom the UV images, using the well known SURF featuresand compare it with other techniques. In order to proof itsbenefits, we use weighted score-level fusion and evaluatethe performance in an one against all comparison. As a resultwe observed a significant amplification of performancewhere traditional face recognition in the visible spectrum isextended with MFP from UV images. We conclude with afuture perspective about the use of these features for futureresearch and discuss observed issues and limitations.
What Can a Single Minutia Tell about Gender?
2018 International Workshop on Biometrics and Forensics (IWBF)
Since fingerprints are one of the most widely deployed biometrics, several applications can benefit from an accurate fingerprint gender estimation. Previous work mainly tackled the task of gender estimation based on complete fingerprints. However, partial fingerprint captures are frequently occurring in many applications including forensics and consumer electronics, with the considered ratio of the fingerprint is variable. Therefore, this work investigates gender estimation on a small, detectable, and well-defined partition of a fingerprint. It investigates gender estimation on the level of a single minutia. Working on this level, we propose a feature extraction process that is able to deal with the rotation and translation invariance problems of fingerprints. This is evaluated on a publicly available database and with five different binary classifiers. As a result, the information of a single minutia achieves a comparable accuracy on the gender classification task as previous work using quarters of aligned fingerprints with an average of more than 25 minutiae.
Efficient, Accurate, and Rotation-Invariant Iris Code
IEEE Signal Processing Letters
The large scale of the recently demanded biometric systems has put a pressure on creating a more efficient, accurate, and private biometric solutions. Iris biometrics is one of the most distinctive and widely used biometric characteristics. High-performing iris representations suffer from the curse of rotation inconsistency. This is usually solved by assuming a range of rotational errors and performing a number of comparisons over this range, which results in a high computational effort and limits indexing and template protection. This work presents a generic and parameter-free transformation of binary iris representation into a rotation-invariant space. The goal is to perform accurate and efficient comparison and enable further indexing and template protection deployment. The proposed approach was tested on a database of 10 000 subjects of the ISYN1 iris database generated by CASIA. Besides providing a compact and rotational-invariant representation, the proposed approach reduced the equal error rate by more than 55% and the computational time by a factor of up to 44 compared to the original representation.
Exploring Deep Multi-biometric Fusion
Darmstadt, TU, Master Thesis, 2017
The field of biometrics aims at performing automatic recognition of individuals based on their biological traits and is, hence, increasingly applied in places with high security requirements. To make biometric systems truly robust and reliable, multiple biometric sources could be combined with a particular fusion scheme. Mainly due to its ease of access, score-level fusion is the most practical method of multi-biometric fusion and has, thus, received the most attention from the research community. Higher-level fusion schemes (e.g. data or feature), in contrast, are difficult to achieve in practice. Yet, they are expected to yield superior results, owing to the higher amount of information available at the point of fusion. A central problem with this type of fusion is the extraction of a discriminative joint feature set. Feature extractors have to be manually designed and typically require a great deal of technical knowledge; for many tasks it is also infeasible to find an appropriate solution. Deep learning offers the ability to automatically learn useful features from raw data for any particular task with minimal human intervention. As a result, it is considered a reasonable option for realizing fusion in a multi-biometric scenario. Within the scope of this work, several architectures, based on a convolutional neural network model, are considered. Their performance have been tested in a multi-modal and a multiinstance setup, respectively. The results have been compared against baseline score-level fusion solutions. It has been shown, that with comparable network structures and computational costs, the less sophisticated, score-level fusion approach performs better than utilizing deep learning for the multi-biometric fusion process. As a future outlook, this thesis proposes possible modifications to the deep fusion approach that might improve the overall performance.
General Borda Count for Multi-biometric Retrieval
2017 International Joint Conference on Biometrics
IEEE International Joint Conference on Biometrics (IJCB) <2017, Denver, CO, USA>
Indexing of multi-biometric data is required to facilitate fast search in large-scale biometric systems. Previous works addressing this issue were challenged by including biometric sources of different nature, utilizing the knowledge about the biometric sources, and optimizing and tuning the retrieval performance. This work presents a generalized multi-biometric retrieval approach that adapts the Borda count algorithm within an optimizable structure. The approach was tested on a database of 10k reference and probe instances of the left and the right irises. The experiments and comparisons to five baseline solutions proved to achieve advances in terms of general indexing performance, tunability to certain operating points, and response to missing data. A clear advantage of the proposed solution was noticed when faced by candidate lists of low quality.
Indexing of Multi-biometric Databases: Fast and Accurate Biometric Search
Darmstadt, TU, Master Thesis, 2017
Biometrics is a rapidly developing field of research and biometric-based identification systems experience a massive growth all around the world caused by the gaining industrial, government and citizen acceptance. The US-VISIT program uses biometric systems to enforce homeland and border security, whereas in the United Arab Emirates (UAE), biometric systems play a major role in the border control process. Similar, in India, biometrics have gained a great deal of attention, as the Unique Identification Authority of India (UIDAI) have already registered over one billion Indian citizens in the last 7 years (uidai.gov.in). Despite the rapid propagation of large-scale databases, the majority of researchers are still focusing on the matching accuracy of small databases, while neglecting scalability and speed issues. Identity association is usually determined by comparing input data against every entry in the database, which causes computational problems when it comes to large-scale databases. Biometric indexing aims to reduce the number of candidate identities to be considered by an identification system when searching for a match in large biometric databases. However, this is a challenging task since biometric data is fuzzy and does not exhibit any natural sorting order. Current indexing methods are mainly based on tree traversal (using kd-trees, B-trees, R-trees) which suffer from the curse of dimensionality, while other indexing methods are based on hashing, which suffer from pure key generation. The goal of this thesis is to develop an indexing scheme based on multiple biometric modalities. It aims to present the main results of research focusing on iris and fingerprint indexing. Fingerprints are undisputedly the most studied biometric modality that are extensive used in civil and forensic recognition systems. Together with the potential rise of iris recognition accurateness along with enhanced robustness, indexing of this modalities becomes a promising field of research. Different unimodal and multimodal identification approaches have already been proposed in past years. However, most of them trade fast identification rates at the cost of accuracy, while the remaining make use of complex indexing structures, which results in a complete restructuring if insertions or deletions are necessary. This work offers a framework for fast and accurate iris indexing as well as effective indexing schemes to combine multiple modalities. To achieve that, three main contributions are made: First, a new rotation invariant iris representation was developed, reducing the equal error rate by more than 55% and the computation time by a factor up to 44 compared to the original representation. Second, this representation was used to construct an indexing scheme, which reaches a hit rate of 99.7% at 0.1% penetration rate, outperforming state of the art algorithms. And third, a general rank-level indexing fusion scheme was developed to effectively combine multiple sources, achieving over 99.98% hit rate at same penetration rate of 0.1%.
Indexing of Single and Multi-instance Iris Data Based on LSH-Forest and Rotation Invariant Representation
Computer Analysis of Images and Patterns
International Conference on Computer Analysis of Images and Patterns (CAIP) <17, 2017, Ystad, Sweden>
Indexing of iris data is required to facilitate fast search in large-scale biometric systems. Previous works addressing this issue were challenged by the tradeoffs between accuracy, computational efficacy, storage costs, and maintainability. This work presents an iris indexing approach based on rotation invariant iris representation and LSH-Forest to produce an accurate and easily maintainable indexing structure. The complexity of insertion or deletion in the proposed method is limited to the same logarithmic complexity of a query and the required storage grows linearly with the database size. The proposed approach was extended into a multi-instance iris indexing scheme resulting in a clear performance improvement. Single iris indexing scored a hit rate of 99.7% at a 0.1% penetration rate while multi-instance indexing scored a 99.98% hit rate at the same penetration rate. The evaluation of the proposed approach was conducted on a large database of 50k references and 50k probes of the left and the right irises. The advantage of the proposed solution was put into prospective by comparing the achieved performance to the reported results in previous works.
Neighbor Distance Ratios and Dynamic Weighting in Multi-biometric Fusion
Pattern Recognition and Image Analysis
Iberian Conference on Pattern Recognition and Image Analysis (IbPRIA) <8, 2017, Faro, Portugal>
Multi-biometrics aims at building more accurate unified biometric decisions based on the information provided by multiple biometric sources. Information fusion is used to optimize the process of creating this unified decision. In previous works dealing with score-level multibiometric fusion, the scores of different biometric sources belonging to the comparison of interest are used to create the fused score. This is usually achieved by assigning static weights for the different biometric sources. In contrast, we focus on integrating the information imbedded in the relative relation between the comparison scores (within a 1:N comparison) in the biometric fusion process using a dynamic weighting scheme. This is performed by considering the neighbors distance ratio in the ranked comparisons to influence the dynamic weights of the fused scores. The evaluation was performed on the Biometric Scores Set BSSR1 database. The enhanced performance induced by including the neighbors distance ratio information within a dynamic weighting scheme in comparison to the baseline solution was shown by an average reduction of the equal error rate by more than 40% over the different test scenarios.
Opportunities for Biometric Technologies in Smart Environments
European Conference on Ambient Intelligence (AmI) <13, 2017, Malaga, Spain>
Smart environments describe spaces that are equipped with sensors, computing facilities and output systems that aim at providing their inhabitants with targeted services and supporting them in their tasks. Increasingly these are faced with challenges in differentiating multiple users and secure authentication. This paper outlines how biometric technologies can be applied in smart environments to overcome these challenges. We give an introduction to these domains and show various applications that can benefit from the combination of biometrics and smart environments.
Trust the Biometric Mainstream: Multi-biometric Fusion and Score Coherence
2017 Proceedings of the 25th European Signal Processing Conference (EUSIPCO)
European Signal Processing Conference (EUSIPCO) <25, 2017, Kos, Greece>
Multi-biometrics aims at building more accurate unified biometric decisions based on the information provided by multiple biometric sources. Information fusion is used to optimize the process of creating this unified decision. In previous works dealing with score-level multi-biometric fusion, the scores of different biometric sources belonging to the comparison of interest are used to create the fused score. This is usually achieved by assigning static weights for the different biometric sources with more advanced solutions considering supplementary dynamic information like sample quality and neighbours distance ratio. This work proposes embedding score coherence information in the fusion process. This is based on our assumption that a minority of biometric sources, which points out towards a different decision than the majority, might have faulty conclusions and should be given relatively smaller role in the final decision. The evaluation was performed on the BioSecure multimodal biometric database with different levels of simulated noise. The proposed solution incorporates, and was compared to, three baseline static weighting approaches. The enhanced performance induced by including the coherence information within a dynamic weighting scheme in comparison to the baseline solution was shown by the reduction of the equal error rate by 45% to 85% over the different test scenarios and proved to maintain high performance when dealing with noisy data.
Combining Low-level Features of Offline Questionnaires for Handwriting Identification
Image Analysis and Recognition
International Conference on Image Analysis and Recognition (ICIAR) <13, 2016, Póvoa de Varzim, Portugal>
When using anonymous offline questionnaires for reviewing services or products it is often not guaranteed that a reviewer does this only once as intended. In this paper an applied combination of different features of handwritten characteristics and its fusion is presented to expose such manipulations. The presented approach covers the aspects of alignment normalization, segmentation, feature extraction, classification and fusion. Nine features from handwritten text, numbers and checkboxes are extracted and used to recognize handwriter duplicates. The proposed method has been tested on a novel database containing pages of handwritten text produced by 1,734 writers. Furthermore we show that the unified biometric decision using a weighted sum combination rule can significantly improve writer identification performance even on low level features.
Multi-biometric Continuous Authentication: a Trust Model for an Asynchronous System
International Conference on Information Fusion (FUSION) <19, 2016, Heidelberg, Deutschland>
Biometric technologies are used to grant specific users access to services and data. The access control is usually performed at the start of a session that spans over a period of time. Continuous authentication aims at insuring the identity of the user over this period of time, and not only at its start. Multi-biometrics aims at increasing the accuracy, robustness and usability of biometrics systems. This work presents a multibiometric continuous authentication solution that includes information from the face images and the keystroke dynamics of the user. A database representing a realistic scenario was collected to develop and evaluate the presented solution. A multi-biometric trust model was designed to cope with the asynchronous nature induced by the different biometric characteristics. A set of performance metrics are discussed and a comparison is presented between the performances of the single characteristic solutions and the fused solution.
Practical View on Face Presentation Attack Detection
Proceedings of the British Machine Vision Conference 2016 [Online]
British Machine Vision Conference (BMVC) <27, 2016, York, UK>
Face recognition is one of the most socially accepted forms of biometric recognition. The recent availability of very accurate and efficient face recognition algorithms leaves the vulnerability to presentation attacks as the major challenge to face recognition solutions. Previous works have shown high preforming presentation attack detection PAD solutions under controlled evaluation scenarios. This work tried to analyze the practical use of PAD by investigating the more realistic scenario of cross-database evaluation and presenting a state-of-the-art performance comparison. The work also investigated the relation between the video duration and the PAD performance. This is done along with presenting an optical flow based approach that proves to outperform state-of-the-art solutions in most experiment settings.
Fuzzy Logic and Multi-biometric Fusion
International Conference on Pattern Recognition Applications and Methods (ICPRAM) <4, 2015, Lisbon, Portugal>
Fuzzy logic has been proposed to improve various aspects of multi-biometric applications including enhancements to the decision making of the application and the robustness to noisy data. This paper discusses recent work that utilized fuzzy logic techniques within the multi-biometric fusion problem. This discussion is presented under two categories, the type of authentication scenario and the nature of the fused data. The paper also presents an introduction to fuzzy logic and multi-biometric fusion. Based on the discussed works, this paper aims to establish current trends and research possibilities in this field.
Multi-Biometric Continuous Authentication
Darmstadt, Hochschule, Master Thesis, 2015
Biometric recognition can be used to secure the access to a system, by recognizing individuals seeking access, based on their behavioural and biological characteristics. In some scenarios, this level of security is not high enough, since it leaves room for attackers to gain access to the system after the initial recognition. Continuous authentication can be used to solve this problem by monitoring the current user during the work session. A genuine user with legitimate access should not be interrupted during the working session. Thus, biometric characteristics which require interaction with sensors are not suited for continuous authentication systems. As a consequence, research has been focused on behavioural biometric characteristics. A trust model defines the behaviour of the continuous authentication system by describing how actions of the user affect the trust value. Decisions are based on this trust value. This work aims to research whether a trust model can be used to combine a biological biometric characteristic and a behavioural characteristic, namely face recognition as the biological component and keystroke dynamics as the behavioural component. Face recognition was chosen because it does neither require additional interaction with a sensor, nor does it interrupt the work session of the genuine user. In order to lessen the impact on the privacy of the user, it was decided to use periodically taken pictures from a webcam instead of a permanent video surveillance. This added the challenge of the information collected by the system being asynchronous. The goal of this work is to develop and evaluate the feasibility and performance of such a system. In order to evaluate this proposed system a database of biometric data suitable for the application scenario was collected and a prototype of the system developed. Face recognition was implemented by using a Local Binary Linear Discriminant Analysis (LBLDA), for keystroke dynamics, a statistical method was implemented. Results show clear improvements in one metric, while the results in the other three measured metrics fell in a range between those of the unfused components. However, results can be further improved by using a more sophisticated fusion approach and tuning the sub components.
Personalized Face Reference from Video: Key-Face Selection and Feature-Level Fusion
Face and Facial Expression Recognition from Real World Videos
International Workshop on Face and Facial Expression Recognition from Real World Videos (FFER) <2014, Stockholm, Sweden>
Face recognition from video in uncontrolled environments is an active research field that received a growing attention recently. This was mainly driven by the wide range of applications and the availability of large databases. This work presents an approach to create a robust and discriminant reference face model from video enrollment data. The work focuses on two issues, first is the key faces selection from video sequences. The second is the feature-level fusion of the key faces. The proposed fusion approaches focus on inducing subject specific feature weighting in the reference face model. Quality based sample weighting is also considered in the fusion process. The proposed approach is evaluated under different sittings on the YouTube Faces data-base and the performance gained by the proposed approach is shown in the form of EER values and ROC curves.
Utilizing Fuzzy Distances in Biometric Comparison
Darmstadt, Hochschule, Bachelor Thesis, 2015
In recent years, the use of biometric systems has dramatically increased and offers substantial improvement in recognizing individuals. The field of biometrics aims at recognizing individuals using their biological and physical characteristics. These characteristics play an important role because they are not easily changed or misused. Furthermore, they cannot be forgotten as is the case for passwords, nor can they be lost in the same manner as identification cards. The increasing demand for biometrics has motivated a lot of research in this field of study, which aim at finding improved and more accurate methods of biometric recognition. All biometric systems use biometric comparison to build their decisions. This comparison process is usually done by calculating the distance between the two feature vectors, each represents the biometric characteristics of different acquisition. Euclidean distance is one of the most commonly used distance measure. Other distance measures such as Manhattan Distance, Canberra Distance, Chebyshev Distance and Cosine Metric was used as the baseline distances in this thesis. The aim of this thesis is to construct a fuzzy distance for biometric comparison and to compare its performance with the baseline distances mentioned above. The main purpose behind a fuzzy logic system is that it allows partial set membership instead of crisp membership. Fuzzy logic system has been successfully being implemented in a lot of appliances that require artificial intelligence. The dataset used in this thesis is called "Labeled Faces in the Wild" gotten from a database of face photographs designed for studying the problem of unconstrained face recognition. The database contains over 13,000 images of faces collected online. Face recognition can be seen as a process of recognizing individuals using their facial characteristics. After implementing the proposed fuzzy distance, its performance was compared with the baseline distances using EER values. Results show that the proposed fuzzy distance is also a good distance measure for biometric comparison.
Weighted Integration of Neighbors Distance Ratio in Multi-biometric Fusion
Annual International Conference of the Biometrics Special Interest Group (BIOSIG) <14, 2015, Darmstadt, Germany>
This work presents an approach to integrate biometric source weighting in the calculation of neighbors distance ratios to be used within a classification-based multi-biometric fusion process. The neighbors distance ratio represents the elevation of the top ranked identification match to the following ranks. Using biometric source weighing can help achieve more accurate initial identity ranking necessary for neighbors distance ratios. It also influences the effect of each biometric source on the ratios values. The proposed approach is developed and evaluated using the Biometric Scores Set BSSR1 database. The results are presented in the verification scenario as receiver operating curves (ROC). The achieved performance is compared to a number of baseline solutions and a satisfying and stable performance was achieved with a clear benefit of integrating the biometric source weights.
Biometric Source Weighting in Multi-Biometric Fusion: Towards a Generalized and Robust Solution
2014 Proceedings of the 22nd European Signal Processing Conference (EUSIPCO)
European Signal Processing Conference (EUSIPCO) <22, 2014, Lisbon, Portugal>
This work presents a new weighting algorithm for biometric sources within a score-level multi-biometric system. Those weights are used in the effective and widely used weighted sum fusion rule to produce multi-biometric decisions. The presented solution is mainly based on the characteristic of the overlap region between the genuine and imposter scores distributions. It also integrates the performance of the biometric source represented by its equal error rate. This solution aims at avoiding the shortcomings of previously proposed solutions such as low generalization abilities and sensitiveness to outliers. The proposed solution is evaluated along with the state of the art and best practice techniques. The evaluation was performed on two databases, the Biometric Scores Set BSSR1 and the Extended Multi Modal Verification for Teleservices and Security applications database and a satisfying and stable performance was achieved.
CMC Curve Properties and Biometric Source Weighting in Multi-Biometric Score-level Fusion
International Conference on Information Fusion (FUSION) <17, 2014, Salamanca, Spain>
Multi-biometrics tries to build a unified biometric decision based on multiple biometric sources in an effort to gain more accuracy and robustness. Multi-biometric fusion aims at optimally combining the information produced by the multiple biometric sources, this usually requires assigning relative weights for the biometric sources to optimize their effect on the final decision. This work presents a new approach for biometric sources weighting within a score-level multi-biometric system. The presented solution tries to investigate the properties of the cumulative match characteristic (CMC) curve, which represents the biometric performance under the identification scenario, and extract biometric source weights based on those properties. The proposed solution is evaluated along with a set of state of the art and best practice weighting techniques. The evaluation was performed on the Biometric Scores Set BSSR1 database and a satisfying and stable performance was achieved.
Gesichtserkennungssoftware in Bildarchiven - Eine Untersuchung zur Nutzung von Gesichtserkennungsprogrammen für die archivische Kernaufgabe "Erschließung"
Potsdam, FH, Bachelor Thesis, 2014
Diese Arbeit betrachtet zunächst die theoretischen Grundlagen für die Gesichtserkennung. Das sind zum einen Erklärungen zur Archivale Foto, auch unter dem Aspekt der Fotoarchivierung und der archivischen Erschließung und die Biometrie bzw. Biometrik. Zum anderen ist es der Ablauf der Gesichtserkennung sowie die Algorithmen, welche häufig Anwendung hierbei finden. Mit Hilfe dieser Informationen werden verschiedene Softwareprogramme einer Untersuchung, die vorab ausführlich erläutert wird, zur Nutzung von Gesichtserkennungsprogrammen für die archivische Kernaufgabe "Erschließung" unterzogen. Die Ergebnisse dieser Untersuchung sollen aufzeigen, ob und inwiefern Gesichtserkennungsprogramme eine qualitative und quantitative Verbesserung in Bezug auf die Datensicherung (Personennennung) und Vereinfachung der Erschließung in Bildarchiven erzielen könnten. Eine Empfehlung resp. ein Ausblick zu diesem Thema vollenden die Arbeit.
Helper Data Scheme for 2D Cancelable Face Recognition Using Bloom Filters
IWSSIP 2014. Proceedings
International Conference on Systems, Signals and Image Processing (IWSSIP) <21, 2014, Dubrovnik, Croatia>
Biometrics provide a source of automated recognition of individuals based on their physiological and behavioral characteristics. As per Directive 95/46/EC, biometric data is considered to be personal data. And according to article 8 of the European Convention on Human Rights, personal data needs to be privacy preserved. Biometric template protection mechanisms provide a privacy preserved biometric authentication. Such mechanisms assist irreversibility, revocability and unlinkability of biometric templates. Recently, a bloom filter based approach was proposed to generate irreversible iris template. In this paper, a helper data scheme for 2D cancelable face verification using bloom filters is proposed. The positions of most representative features (stable features) are used as helper data, which helps in the face recognition. The features used are extracted using Local Binary Linear Discriminant Analysis. The effect of stable features on recognition performance under scenarios of with and without using bloom filters is investigated. In addition, recognition performance after compressing multiple features into a single bloom filter is presented. The results are experimentally proved on two benchmark databases namely LFW and ORL datasets.
Multi-biometric Score-Level Fusion and the Integration of the Neighbors Distance Ratio
Image Analysis and Recognition. Proceedings Part II
International Conference on Image Analysis and Recognition (ICIAR) <11, 2014, Vilamoura, Portugal>
Multi-biometrics aims at building more accurate unified biometric decisions based on the information provided by multiple biometric sources. Information fusion is used to optimize the process of creating this unified decision. In previous works dealing with score-level multibiometric fusion, the scores of different biometric sources belonging to the comparison of interest are used to create the fused score. The novelty of this work focuses on integrating the relation of the fused scores to other comparisons within a 1:N comparison. This is performed by considering the neighbors distance ratio in the ranked comparisons set within a classification-based fusion approach. The evaluation was performed on the Biometric Scores Set BSSR1 database and the enhanced performance induced by the integration of neighbors distance ratio was clearly presented.
Multimedia Analysis of Video Sources
SIGMAP 2014. Proceedings of the 11th International Conference on Signal Processing and Multimedia Applications
International Conference on Signal Processing and Multimedia Applications (SIGMAP) <11, 2014, Vienna, Austria>
Law Enforcement Agencies (LEAs) spend increasing efforts and resources on monitoring open sources, searching for suspicious behaviours and crime clues. The task of efficiently and effectively monitoring open sources is strongly linked to the capability of automatically retrieving and analyzing multimedia data. This paper presents a multimodal analytics system, created in cooperation with European LEAs. In particular it is described how the video analytics subsystem produces a workflow of multimedia data analysis processes. After a first analysis of video files, images are extracted in order to perform image comparison, classification and face recognition. In addition, audio content is extracted to perform speaker recognition and multilingual analysis of text transcripts. The integration of multimedia analysis results allows LEAs to extract pertinent knowledge from the gathered information.
Privacy Preserved Duplicate Check using Multi-biometric Fusion
International Conference on Information Fusion (FUSION) <17, 2014, Salamanca, Spain>
Automated recognition of individuals can be performed using biometrics without any requirement of explicit knowledge of a PIN or a password. On the one side biometrics has given convenience to citizens as they do not need to memorize a bunch of passwords, but on the other side intra (inter) class variations within (between) biometric features makes biometric authentications untrustworthy. Therefore, decisions based on biometric authentications are made more reliable by using several biometric authentications performed on single or multiple biometric modalities (i.e. multi-biometric fusion). This paper describes a method to identify if a person tries to re-enrol him/herself in a database, when he/she is already enrolled. This is referred to as duplicate check. In this work, duplicate check is performed using two modalities: face and iris. The templates used during the duplicate check are compliant to the ISO/IEC 24745 - Biometric information protection. Such templates are known as protected biometric templates. The protected biometric templates used in this work are generated using the recently published irreversible transformation based on Bloom filters. Scores are calculated from face and iris Bloom filters based templates by comparison with their respective enrolment templates using the normalized Hamming distance. As a decision of the duplicate check, these scores from both modalities are fused with appropriate weighting factors in order to get improved performance compared to using single individual modalities. The presented scheme is experimentally validated using two public benchmark databases namely the LFW and the CASIA-Iris-Thousand databases for face and iris respectively.
Prototypical Development of an In-Shop Advertisment System using Body Dimension Recognition
Darmstadt, Hochschule, Master Thesis, 2014
This thesis outlines a system created to give consumers in the fashion industry an idea of how an item of clothing will look on them before trying it on. In the form of a short video, items of clothing are projected virtually onto an image of the user. Through the use of this system, retailers and manufacturers have the chance to immediately display their clothes on potential customers.
Three Dimensional Scanning of Clothes, for Simulation and Presentation Purposes in a Virtual Fitting Room
Darmstadt, Hochschule, Master Thesis, 2014
The aim of this thesis is to develop a low-cost semi-professional automated 3D scanning and post-production system for digitizing clothing and apparel for in shop and online presentation purposes. Ultimately giving birth to a database of digitized 3d models of apparel to enable virtual-fitting rooms and real-time fitting feedback. In the first part different scanning methods are tested if they are suited for scanning apparel and if the quality is good enough for advertisement and presentation purposes. The cost of the system is also taken into account. The thesis then identifies the best and most cost effective approach and tries to develop and automate the method using state of the art consumer products. In the main section the thesis describes the functionality of the method and how it can be applied. Different algorithms and workflows are shown and combined to develop the automated system. In conclusion the thesis describes and summarizes the system and opens up how it could be implemented in a consumer oriented presentation like a virtual fitting room or an online shopping style advisor using the users body-metrics.
Virtual Fitting Pipeline: Body Dimension Recognition, Cloth Modeling, and On-Body Simulation
VRIPHYS 14: 11th Workshop in Virtual Reality Interactions and Physical Simulations
International Workshop in Virtual Reality Interaction and Physical Simulations (VRIPHYS) <11, 2014, Bremen, Germany>
This paper describes a solution for 3D clothes simulation on human avatars. The proposed approach consists of three parts, the collection of anthropometric human body dimensions, cloths scanning, and the simulation on 3D avatars. The simulation and human machine interaction has been designed for application in a passive In- Shop advertisement system. All parts have been evaluated and adapted under the aim of developing a low-cost automated scanning and post-production system. Human body dimension recognition was achieved by using a landmark detection based approach using both two 2D and 3D cameras for front and profile images. The human silhouettes extraction solution based on 2D images is expected to be more robust to multi-textured background surfaces than existing solutions. Eight measurements corresponding to the norm of body dimensions defined in the standard EN-13402 were used to reconstruct a 3D model of the human body. The performance is evaluated against the ground-truth of our newly acquired database. For 3D scanning of clothes, different scanning methods have been evaluated under apparel, quality and cost aspects. The chosen approach uses state of the art consumer products and describes how they can be combined to develop an automated system. The scanned cloths can be later simulated on the human avatars, which are created based on estimation of human body dimensions. This work concludes with software design suggestions for a consumer oriented solution such as a virtual fitting room using body metrics. A number of future challenges and an outlook for possible solutions are also discussed.
An Overview on Multi-biometric Score-level Fusion
International Conference on Pattern Recognition Applications and Methods (ICPRAM) <2, 2013, Barcelona, Spain>
Multi-biometrics is the use of multiple biometric recognition sources to provide a more dependable verification or identification decision. Fusion of multi-biometric sources can be performed on different levels, such as the data, feature, or score level. This work presents an overview of the multi-biometric score-level fusion problem, along with the proposed solution in the literature. A discussion is made to provide a comparison between multi-biometric fusion in both scenarios. This discussion aims at providing a clearer view of future developments especially under the identification scenario where many related applications are rapidly growing such as forensics and ubiquitous surveillance.
Face Liveness Detection Against Image and Video Spoofing Attacks
Darmstadt, TU, Master Thesis, 2013
Many situations require users to log into a computer system. For example to perform private tasks like banking, social interaction or to get access to a secured area. Conventional security driven systems have the disadvantage that passwords or keycards are needed. These passwords or keycards can get lost or stolen resulting in a security risk. To overcome this drawback biometrics use the characteristics of the human body to grand access to a computer system. Beside fingerprint or iris recognition face detection is a popular biometric trait. The reason therefore is that it requires only a usual camera. Most of the current systems have a camera build in anyway. Also face recognition is not very intrusive to the user, which gives a high acceptability of face recognition is biometric trait. In the past though face recognition systems could easily be tricked due to spoofing attempts using pictures or videos of the authenticate user. This thesis analyses current algorithms to counter such spoofing attempts and presents a novel approach. The presented approach will use Machine Learning and Computer Vision to utilize an algorithms by Wu et al. [WRS_12] that can magnify subtle changes in videos to reveal the human pulse. An evaluation of the feasibility of this approach will be given.
Missing Data Estimation in Multi-biometric Identification and Verification
2013 IEEE Workshop on Biometric Measurements and Systems for Security and Medical Applications
IEEE Workshop on Biometric Measurements and Systems for Security and Medical Applications (BioMS) <4, 2013, Naples, Italy>
In the practical use of multi-biometric solutions, biometric sources involved in producing the verification or identification decision do occasionally fail to produce results. This work discusses solutions for missing data in multi-biometric score-level fusion. A missing data estimation solution based on support vector regression was presented in this work and compared to four baseline solutions. The evaluation was carried under both the verification and the identification scenarios in an effort to show the effect of missing data estimation on the relatively understudied multi-biometric identification scenario. Evaluation was performed on the Biosecure DS2 score database and satisfying performance was achieved under both biometric scenarios.
Performance Anchored Score Normalization for Multi-Biometric Fusion
Advances in Visual Computing. 9th International Symposium, ISVC 2013
International Symposium on Visual Computing (ISVC) <9, 2013, Rethymnon, Crete, Greece>
This work presents a family of novel normalization techniques for score-level multi-biometric fusion. The proposed normalization is not only concerned to bring comparison scores to a common range and scale, it also focuses in bringing certain operational performance points in the distribution into alignment. The Performance Anchored Normalization (PAN) algorithms discussed here were tested on the extended Multi Modal Verification for Teleservices and Security applications database (XM2VTS) and proved to outperform conventional score normalization techniques in most tests. The tests were performed with combination fusion rules and presented as biometric verification performance measures.
Qualitätsbasierte Informationsfusion auf der Bewertungsebene innerhalb der multimodalen biometrischen Identifikation
Darmstadt, TU, Master Thesis, 2013
Im Laufe der Zeit haben sich biometrische Erkennungsverfahren als zuverlässiges Mittel zum Zwecke der Zugangskontrolle zu physikalischen und virtuellen Bereichen entwickelt. Die Schwächen unimodaler Systeme werden dabei oft durch multimodale Ansätze verbessert, insbesondere ermöglichen diese einen robusteren Registrierungsprozess, erhöhte Sicherheit gegenüber gefälschten Identitäten und eine höhere Erkennungsgenauigkeit. Die vorliegende Arbeit beschäftigt sich mit der Frage, wie Qualitätsinformationen über die extrahierten Merkmale zu einer weiteren Verbesserung der multimodalen biometrischen Identifikation beitragen können. Dabei liegt die Idee zugrunde, dass Merkmale von höherer Qualität auch eine höhere Zuverlässigkeit im Hinblick auf deren Klassifikation zusichern, diese also stärker in den Entscheidungsprozess eingebunden werden sollten als Merkmale von geringerer Qualität. Zur Überprüfung dieser Annahme wurde ein auf dem Gradientenverfahren basierender Fusionsmechanismus um die Berücksichtigung von Qualitätsinformationen erweitert und dessen Erkennungsperformanz unter verschiedenen Konditionen, darunter im Besonderen das Vorhandensein fehlender Bewertungsmaße, mit dem ursprünglichen Algorithmus verglichen.
The 2nd Competition on Counter Measures to 2D Face Spoofing Attacks
2013 International Conference on Biometrics (ICB)
IAPR International Conference on Biometrics (ICB) <6, 2013, Madrid, Spain>
As a crucial security problem, anti-spoofing in biometrics, and particularly for the face modality, has achieved great progress in the recent years. Still, new threats arrive in form of better, more realistic and more sophisticated spoofing attacks. The objective of the 2nd Competition on Counter Measures to 2D Face Spoofing Attacks is to challenge researchers to create counter measures effectively detecting a variety of attacks. The submitted propositions are evaluated on the Replay-Attack database and the achieved results are presented in this paper.
Untersuchung von Normalisierungs- und Kombinationsverfahren im Kontext von multibiometrischer score level Fusion
Darmstadt, Hochschule, Bachelor Thesis, 2013
Diese Arbeit präsentiert eine Untersuchung von verschiedenen Normalisierungs- und Kombinationsverfahren im Kontext von multibiometrischer score level Fusion. Hierbei werden zur biometrischen Erkennung von Personen mehrere Informationen verwendet und zu einem eindeutigen Ergebnis verschmolzen. Die Effizienz dieser Verfahren wurde im Rahmen dieser Arbeit mittels einer Anwendung und einem konkretem Datensatz evaluiert. Die Evaluierung wurde für beide gängigen Modi einer biometrischen Erkennung, der Identifizierung und Verifikation, durchgeführt. Die Resultate zeigen, dass die Wahl der Verfahren einen großen Einfluss auf die Effizienz der biometrischen Erkennung ausübt. Im Verifikationsmodus wurde im schlechtesten Fall eine Erkennungsrate von 28.34% erzielt. Im besten Fall hingegen betrug die Erkennungsrate 95.54%. Die niedrigste Erkennungsrate im Identifikationsmodus betrug 51.92% und die höchste 98.71%.
2D Face Liveness Detection: an Overview
Annual International Conference of the Biometrics Special Interest Group (BIOSIG) <11, 2012, Darmstadt, Germany>
Face recognition based on 2D images is a widely used biometric approach. This is mainly due to the simplicity and high usability of this approach. Nonetheless, those solutions are vulnerable to spoof attacks made by non-real faces. In order to identify malicious attacks on such biometric systems, 2D face liveness detection approaches are developed. In this work, face liveness detection approaches are categorized based on the type of liveness indicator used. This categorization helps understanding different spoof attacks scenarios and their relation to the developed solutions. A review of the latest works dealing with face liveness detection works is presented. A discussion is made to link the state of the art solutions with the presented categorization along with the available and possible future datasets. All that aim to provide a clear path for the future development of innovative face liveness detection solutions.
Ear Recognition Using Multi-Scale Histogram of Oriented Gradients
Eighth International Conference on Intelligent Information Hiding and Multimedia Signal Processing. Proceedings
International Conference on Intelligent Information Hiding and Multimedia Signal Processing (IIH-MSP) <8, 2012, Piraeus-Athens, Greece>
Ear recognition is a promising biometric measure, especially with the growing interest in multi-modal biometrics. Histogram of Oriented Gradients (HOG) has been effectively and efficiently used solving the problems of object detection and recognition, especially when illumination variations are present. This work presents a robust approach for ear recognition using multi-scale dense HOG features as a descriptor of 2D ear images. The multi-scale features assure to capture the different and complicated structures of ear images. Dimensionality reduction was performed to avoid feature redundancy and provide a more efficient recognition process while being prone to over-fitting. Finally, a test was performed on a large and realistic database and the results were compared to the state of the art ear recognition approaches tested on the same dataset and under the same test procedure.