A Comprehensive Study on Face Recognition Biases Beyond Demographics
IEEE Transactions on Technology and Society
Face recognition (FR) systems have a growing effect on critical decision-making processes. Recent works have shown that FR solutions show strong performance differences based on the user’s demographics. However, to enable a trustworthy FR technology, it is essential to know the influence of an extended range of facial attributes on FR beyond demographics. Therefore, in this work, we analyse FR bias over a wide range of attributes. We investigate the influence of 47 attributes on the verification performance of two popular FR models. The experiments were performed on the publicly available MAAD-Face attribute database with over 120M high-quality attribute annotations. To prevent misleading statements about biased performances, we introduced control group based validity values to decide if unbalanced test data causes the performance differences. The results demonstrate that also many non-demographic attributes strongly affect the recognition performance, such as accessories, hair-styles and colors, face shapes, or facial anomalies. The observations of this work show the strong need for further advances in making FR system more robust, explainable, and fair. Moreover, our findings might help to a better understanding of how FR networks work, to enhance the robustness of these networks, and to develop more generalized bias-mitigating face recognition solutions.
Explainable Face Image Quality Assessment
Darmstadt, TU, Master Thesis, 2021
The high performance of today’s face recognition systems is driven by the quality of the used samples. To ensure a high quality of face images enrolled in a system, face image quality assessment is performed on these images. However, if an image is rejected due to low quality it is not obvious to the user why. Showing what led to the low quality is useful to increase trust and transparency in the system. Previous work has never addressed the explainability of their quality estimation, but only the explainability of their face recognition approaches. In this work, we propose a gradient-based method that explains which pixels contribute to the overall image quality and which do not. By adding a quality node to the end of the model, we can calculate quality-dependent gradients and visualize them. The qualitative experiments are conducted on three different datasets and we also propose a general framework for quantitative analysis of face image quality saliency maps. With our method, we can assign quality values to individual pixels and provide a meaningful explanation of how face recognition models work and how they respond to face image quality impairments. Our method provides pixel-level explanations, requires no training, applies to any face recognition model, and also takes model-specific behavior into account. By explaining how the poor quality is caused, concrete instructions can be given to people who take pictures suitable for face recognition, face image standards can be adapted, and low-quality areas can be inpainted.
Privacy Evaluation Protocols for the Evaluation of Soft-Biometric Privacy-Enhancing Technologies
Conference on Biometrics and Electronic Signatures (BIOSIG) <19, 2020, Online>
GI-Edition - Lecture Notes in Informatics (LNI), P-306
Biometric data includes privacy-sensitive information, such as soft-biometrics. Soft-biometric privacy enhancing technologies aim at limiting the possibility of deducing such information. Previous works proposed several solutions to this problem using several different evaluation processes, metrics, and attack scenarios. The absence of a standardized evaluation protocol makes a meaningful comparison of these solutions difficult. In this work, we propose privacy evaluation protocols (PEPs) for privacy-enhancing technologies (PETs) dealing with soft-biometric privacy. Our framework evaluates PETs in the most critical scenario of an attacker that knows and adapts to the systems privacy-mechanism. Moreover, our PEPs differentiate between PET of learning-based or training-free nature. To ensure that our protocol meets the highest standards in both cases, it is based on Kerckhoffs‘s principle of cryptography.
Multi-algorithmic Fusion for Reliable Age and Gender Estimation from Face Images
International Conference on Information Fusion (FUSION) <22, 2019, Ottawa, Canada>
Automated estimation of demographic attributes, such as gender and age, became of great importance for many potential applications ranging from forensics to social media. Although previous works reported performances that closely match human level. These solutions lack of human intuition that allows human beings to state the confidences of their predictions. While the human intuition subconsciously considers surrounding conditions or the lack of experience in a certain task, current algorithmic solutions tend to mispredict with high confidence scores. In this work, we propose a multi-algorithmic fusion approach for age and gender estimation that is able to accurately state the model’s prediction reliability. Our solution is based on stochastic forward passes through a dropout-reduced neural network ensemble. By utilizing multiple stochastic forward passes combined from the neural network ensemble, the centrality and dispersion of these predictions are used to derive a confidence statement about the prediction. Our experiments were conducted on the Adience benchmark.We showed that the proposed solution reached and exceeded state-of-the-art performance for the age and gender estimation tasks. Further, we demonstrated that the reliability statements of the predictions of our proposed solution capture challenging conditions and underrepresented training samples.
Reliable Age and Gender Estimation from Face Images: Stating the Confidence of Model Predictions
IEEE 10th International Conference on Biometrics: Theory, Applications and Systems
IEEE International Conference on Biometrics: Theory, Applications and Systems (BTAS) <10, 2019, Tampa, Florida, USA>
Automated age and gender estimation became of great importance for many potential applications ranging from forensics to social media. Although previous works reported high increased performances, these solutions tend to mispredict under challenging conditions or when the trained model faces a sample that was underrepresented in the training data. In this work, we propose an age and gender estimation model, as well as a novel reliability measure to quantify the confidence of the model’s prediction. Our solution is based on stochastic forward passes through dropout-reduced neural networks that were theoretically proven to approximate Gaussian processes. By utilizing multiple stochastic forward passes, the centrality and dispersion of these predictions are used to derive a confidence statement about the prediction. Experiments were conducted on the Adience benchmark. We showed that the proposed solution reached and exceeded state-ofthe-art performance. Further, we demonstrated that the proposed reliability measure correlates with the prediction performance and thus, is highly successful in quantifying the prediction reliability.