Kevin W. Bowyer

Face recognition systems (FRS) exhibit significant accuracy differences based on the user’s gender. Since such a gender gap reduces the trustworthiness of FRS, more recent efforts have tried to find the causes. However, these studies make use of manually selected, correlated, and small-sized sets of facial features to support their claims. In this work, we analyze gender bias in face recognition by successfully extending the search domain to decorrelated combinations of 40 non-demographic facial characteristics. First, we introduce a toolchain to effectively decorrelate and aggregate facial attributes to enable a lessbiased gender analysis on large-scale data. Second, we tailor two specialized metrics to quantify the effect of facial attributes on absolute and relative fairness. Based on these grounds, we thirdly present a novel unsupervised joint investigation framework capable of identifying attribute combinations leading to vanishing bias when used as filter predicates for balanced testing datasets. Experiments show the gender gap vanishing when images of male and female subjects share specific attributes, clearly indicating that the disparate performance is not a question of biology but of the social definition of appearance. These findings could reshape our understanding of fairness in face biometrics and provide insights into FRS, helping to address gender bias issues.

Biometric authentication systems face a critical security challenge: once a biometric template is compromised, it cannot be replaced. Revocable biometrics address this by enabling template cancellation and reissuance without degrading recognition accuracy. We systematically evaluate a revocable framework in which multiple independently trained instances of the same end-to-end model yield revocable templates. From a shared backbone architecture, training dataset, and identical hyperparameter configurations, we derive distinct face matchers that exhibit near-identical genuine and impostor score distributions. Notably, cross-model similarity scores (genuine scores) for the same individual are lower than same-model scores for different individuals (impostor scores), reducing the value of stolen templates to sub-impostor levels in cross-model scenarios. We assess ResNet-100, MobileFaceNet, and ViT backbones under ArcFace and AdaFace losses, verifying robustness across architectures and loss functions. t-SNE visualizations of embeddings from independently trained instances with the same backbone and loss show that features realize different coordinate systems on a shared face-identity manifold, achieving comparable validity and accuracy rather than representing trivial permutations or transforms. However, an adversary with paired feature vectors from two models for the same enrolled images can still approximate the inter-model transformation. To mitigate this, we propose a model-mixture deployment that randomly and uniformly partitions subjects across models, with subsequent updates following the same randomized partitioning to obstruct direct linear mappings while maintaining stable operation of the revocable biometric system.

Probe images used in one-to-many facial identification often deviate from ‘mugshot quality’ in various ways. This paper explores how the accuracy of one-to-many facial identification is degraded by the person in the probe image wearing sunglasses, and how this accuracy degradation can be ameliorated. We show that sunglasses degrade accuracy for mugshot-quality images by an amount similar to strong blur or noticeably lower resolution. Further, we demonstrate that the combination of sunglasses with blur or lower resolution has an additive effect in degrading accuracy. These results have important implications for developing criteria to qualify a probe image for the accuracy that can be expected if it is used for facial identification. We explore two approaches to reduce the accuracy loss. First, we show that by adding synthetic sunglasses to all gallery images, we can recover approximately 27% of the accuracy loss with no change to the face-matching model. Second, we demonstrate that training a sunglasses-aware model using augmented training data can achieve up to 33% higher accuracy for sunglasses-wearing probes while maintaining baseline accuracy for non-sunglasses probes. Moreover, combining both approaches achieves even greater accuracy recovery (up to 43% for males and 35% for females). Additionally, our results show that increasing the representation of sunglasses-wearing images evenly across all identities in training data is more effective than concentrating them on fewer identities. The effectiveness of our sunglasses-aware model is validated through t-SNE visualization and a 16% increase in feature similarity between sunglasses and no-sunglasses images of a given identity. Our investigation further reveals that baseball caps play a role similar to sunglasses in degrading facial identification accuracy. To support replication and further research, info to obtain the face image datasets used is available at https://cvrl.nd.edu/projects/data/

Post-mortem iris recognition brings both hope to the forensic community (a short-term but accurate and fast means of verifying identity) as well as concerns to society (its potential illicit use in post-mortem impersonation). These hopes and concerns have grown along with the volume of research in post-mortem iris recognition. Barriers to further progress in post-mortem iris recognition include the difficult nature of data collection, and the resulting small number of approaches designed specifically for comparing iris images of deceased subjects. This paper makes several unique contributions to mitigate these barriers. First, we have collected and we offer a new dataset of NIR (compliant with ISO/IEC 19794-6 where possible) and visible-light iris images collected after demise from 259 subjects, with the largest PMI (post-mortem interval) being 1,674 hours. For one subject, the data has been collected before and after death, the first such case ever published. Second, the collected dataset was combined with publicly-available post-mortem samples to assess the current state of the art in automatic forensic iris recognition with five iris recognition methods and data originating from 338 deceased subjects. These experiments include analyses of how selected demographic factors influence recognition performance. Thirdly, this study implements a model for detecting post-mortem iris images, which can be considered as presentation attacks. Finally, we offer an open-source forensic tool integrating three post-mortem iris recognition methods with explainability elements added to make the comparison process more human-interpretable.

Face recognition (FR) is one of the most widely used biometric methods for identity authentication. Although most of the recently proposed methods demonstrate remarkable performance on high-quality datasets, such as LFW, their effectiveness is limited when assessed on low-resolution images. To address this challenge, knowledge distillation and super-resolution techniques have been applied, primarily using the ResNet architecture. However, the performance of deep learning approaches depends in part on the architecture used. In this study, we use neuroevolution with a genetic algorithm (GA) to design Convolutional Neural Networks (CNNs) automatically for low-resolution (LR) FR. To reduce the search time, a binary classifier is used to identify which generated architectures should be trained and which should not. The selected architectures are then trained and evaluated using QUMUL-TinyFace (training partition), a native LR dataset, to obtain their fitness, while the remaining architectures are assessed using a performance predictor model to estimate their fitness, bypassing the training stage. The classifier and performance predictor are trained using the CNN architectures evaluated from previous generations, with the architecture encoding used as a feature vector. The proposed method was assessed on both the QMUL-TinyFace (for face identification) and QMUL-SurvFace (for face verification) datasets, achieving a rank-1 recognition rate of 74.7% and a mean verification accuracy rate of 85.1%, respectively, outperforming results from previously published methods.

State-of-the-art deep CNN face matchers are typically created using extensive training sets of color face images. Our study reveals that such matchers attain virtually identical accuracy when trained on either grayscale or color versions of the training set, even when the evaluation is done using color test images. Furthermore, we demonstrate that shallower models, lacking the capacity to model complex representations, rely more heavily on low-level features such as those associated with color. As a result, they display diminished accuracy when trained with grayscale images. We then consider possible causes for deeper CNN face matchers “not seeing color”. Popular Web-scraped face datasets actually have 30 to 60% of their identities with one or more grayscale images. We analyze whether this grayscale element in the training set impacts the accuracy achieved, and conclude that it does not. We demonstrate that using only grayscale images for both training and testing achieves accuracy comparable to that achieved using only color images for deeper models. This holds true for both real and synthetic training datasets. HSV color space, which separates chroma and luma information, does not improve the network’s learning about color any more than in the RGB color space. We then show that the skin region of an individual’s images in a Web-scraped training set exhibits significant variation in their mapping to color space. This suggests that color carries limited identity-specific information. We also show that when the first convolution layer is restricted to a single filter, models learn a grayscale conversion filter and pass a grayscale version of the input color image to the next layer. Finally, we demonstrate that leveraging the lower per-image storage for grayscale to increase the number of images in the training set can improve accuracy of face recognition.

Concerns related to the proliferation of automated face recognition technology with the intent of solving or preventing crimes continue to mount. The technology being implicated in wrongful arrest following 1-to-many face identification has heightened scrutiny in the media and spawned lawsuits. We analyze the accuracy of 1-to-many face identification across African-American and Caucasian demographic groups and in the presence of blur and reduced resolution in the probe image, as might occur in “surveillance camera quality” images. Because of the high accuracy of modern face recognition algorithms with “government ID quality” images and the size of available datasets, we use the following indirect metrics to assess the relative propensity for false positive identifications: (1) the traditional d-prime statistic between mated and non-mated score distributions, (2) absolute score difference between thresholds in the high-similarity tail of the non-mated distribution and the low-similarity tail of the mated distribution, and (3) distribution of (mated – non-mated rank-one scores) across the set of probe images. We find that demographic variation patterns in 1-to-many accuracy largely, but not perfectly, follow that observed in 1-to-1 accuracy. We show that, different from the case with 1-to-1 matching accuracy, demographic comparison of 1-to-many accuracy can be affected by different numbers of identities and images across demographics. We also show that increased blur or reduced resolution of the face in the probe image can significantly increase the false positive identification rate. This point is important because the reputation of modern face recognition algorithms for high accuracy in 1-to-many face identification comes from statistics based on analyzing “government ID quality” rather than “surveillance camera quality” images.