Dr. Damer, face morphing is more than just an innocent gimmick -- its being used more and more by criminals. Why? What exactly can happen when two faces are morphed in one photo?
Face morphing is when an image is created that, in principle, contains multiple identities. The deception is undetectable by both the human eye and machines. If this kind of photo is on a passport or ID, criminals who, say, are prohibited from entering a certain country, can use this photo to cross the border anyway. They use a passport with a fake name, fake details, and a morphed photo. Police and border control use this photo as a verification tool. A passport proves that the identity actually belongs to the person claiming to own the identity, and with face morphing, a person with a criminal background can simply generate another identity and use it to identify themselves.
At passport control in particular, identity is checked with more than the human eye. The passport is also scanned by a machine to verify the authenticity of the document, especially the photo. Why is this system so error-prone and unable to detect a morphed image?
The issue is with machine learning. These biometric systems were taught to think, more or less, like us humans. As a human, you need to put up with changes in the face of the person across from you on a daily basis. You recognize a family member, for example, even when their expression changes, like when they smile or cry. You recognize them even when they get their hair cut or dyed. You would recognize that person even in a photo that’s five years old. It’s exactly the same for machines: They were taught to accept certain changes in faces. If they weren’t, it would be a problem. Your passport, for example, is valid for 10 years and you expect to be identifiable by the border officials and the machine using this passport and corresponding image over this period.
So, the machine needs to master a task that isn’t exactly easy. They need to accept certain deviations in a person’s face while also being able to distinguish between different people. And as you said, this check is performed twice, so it’s not just the automated system that can be outwitted here. Even a police officer manually inspecting the document would let someone with a morphed photo through.
This is where the research of the Fraunhofer Institute for Computer Graphics Research IGD comes in. The institute has been addressing the issue of biometrics in depth and is pursuing various areas of research. One project, for example, is addressing qualitative image recognition, where low-quality passport photos are recognized at the outset and rejected. Is that correct?
That’s right. We’re also working on inspecting the quality of facial images. Part of this work also focuses on the question of what kinds of photos should be accepted on a passport.
Right now, the photos on a passport follow certain standards that determine what needs to be in the image. These standards include a white background, the person facing forward, no hair can be hanging over the face, and many others. But, in the end, these are just descriptive rules. An official checks the photo for these quality metrics, but, of course, these aren’t enough. A photo taken with a smartphone can definitely observe these rules, but the quality of the image itself is still poor¾or poorer than it should be.
Image quality can then, of course, also relate to the issue of face morphing attacks. The photo isn’t always morphed -- it can also be of low quality. In other words, there’s a correlation between these two research topics, but they aren’t the same thing.
Can you tell me more about the face morphing project?
Most of our work in the area of face morphing takes place as part of the ATHENE project. ATHENE is a research center of the Fraunhofer Gesellschaft, or, more accurately, the “National Research Center for Applied Cybersecurity”, in Darmstadt and Fraunhofer IGD, along with the Fraunhofer Institute for Secure Information Technology SIT, is a part of it. Within ATHENE, we focus on various levels of the field of biometrics. There’s a research project on identity management, for example, and this is where we’re working on face morphing. Above all, we consider one of our primary tasks to be detecting, uncovering and identifying face morphing attacks.
As with all other types of (cyber) attacks, face morphing attacks are increasing. Criminals are always finding new ways to set up these kinds of attacks -- and, accordingly, it’s difficult to uncover an attack without detecting it. So, we have two ways of addressing this problem and we use both: First, we’re working on detection algorithms designed to anticipate unknown attacks. This lets us be ready for an attack created using methods previously unknown to us, allowing us to create a generalized face morphing detection, so to speak. Second, we’re also trying simply to be quicker than the criminals themselves and develop possible attack scenarios and strategies. This lets us make sure our detection algorithm recognizes new, unused face morphing attack possibilities and, in case of emergency, expose it accordingly. You could say we come up with the attack ourselves.
How exactly does that work?
There are certain “interest points” on the human face, such as the corners of the mouth, the corners of the eyes, and so on. These points can vary. In general, there are 68 well-defined interest points. In the traditional method of morphing faces, two faces are used that, more or less, resemble one another. The locations of these interest points can be identified either manually or automatically, and then averaged by, say, image editing software to create a new face. The averaging of facial points is a key factor in face morphing. Some reworking is then necessary to hide certain features and the editing process. This type of face morphing is sometimes hard but also sometimes easy to detect. But you have to remember that hackers and attackers are often considerably more adept technically and develop other methods. This is why we’re also exploring a new method that could be used by hackers. We’re using generative adversarial networks to actually generate new faces. Deep learning and artificial intelligence play a critical role in this. So, now, we’re no longer averaging to produce a new face. The system learns from the images, uses the factors and properties, and generates a new face automatically based on these characteristic structures -- one that, naturally, has characteristics of both original faces that were used as the basis. With a face generated in this manner, we can then identify and verify the two original identities.
You mean you’re morphing images yourselves at the institute in order to reveal the error-proneness of biometric systems?
That’s correct. We’ve developed a method that shows how susceptible our system still is.
This will presumably be used as the basis of further research. Is there already a project focusing on the solution?
At the moment we’re working on solutions to that problem, although we can’t claim that these solutions are perfect. The reason for that is simple: There will be new types of attacks in the future and this is why our solution can’t be perfect. But we need to work on coming up with a solution that’s virtually perfect. We basically need to be one step ahead of criminals and also research new possibilities so we can teach and prepare our systems for them.
Can you give an example of how forgery proofing with regard to IDs with photos could be enhanced in general?
That’s hard to say. It doesn’t depend on just Germany, for example, on German research and German guidelines. We could say the passport issuing process needs to be changed. Even at this stage, the photos need to be examined to determine whether they’re real or morphed. Then we could trust German passports more, but what about those from other countries? Face morphing attacks will continue to be a major problem. It’s not enough to make German passports -- or even passports in general -- secure for the time being, we need to seek out the criminals and their methods. This is why I think the only realistic solution is really to develop methods that uncover this kind of attack. And that’s exactly what we’re working on at the ATHENE research center.
Is Fraunhofer IGD involved in other ATHENE projects about biometrics or face morphing?
There are three biometrics projects going on at ATHENE in which Fraunhofer IGD is also involved. These projects are, themselves, pretty large, however, which is why they will be divided into various project activities.
We’ve already talked about two of the three projects. The first is on quality control of images of faces. We measure the usability of a facial image within facial recognition systems. This is significant not only for passport control, but image recognition is also highly relevant for, say, forensic purposes. Ultimately, we need to know how much we can rely on a facial image in order to decide whether it’s of a specific person or not.
In the area of identity management, we’re focusing on variances and variabilities in biometric systems. This includes the morphing attacks we’ve mentioned, but the field itself is much broader. It also includes what are known as “presentation attacks”. Very roughly speaking, this is focusing on photos or videos of a person’s face or fingerprint that are submitted to a system in order to deceive it. We also focus on attacks on privacy that use these systems. If you, say, log into your bank using facial recognition, you want your bank only to use your facial data to verify your identity. In reality, however, facial recognition data can be used to determine your gender, race, health issues and other aspects. We want to make sure facial recognition data is protected and not used for other purposes.
The third project is on biometrics in embedded systems -- systems that don’t have as much variability, but rather have their own detection properties. We’re doing a lot of work on machine learning solutions in this area. The goal is to apply biometrics on, say, cellphones or augmented reality cameras in such a way that they’re really used to enhance security. We’re also working on augmented reality cameras that can also be used at passport control. To accomplish that, we need to develop machine learning solutions with low combination performance.
To wrap up, can you give us a quick look at where the research is going? What will your future research into biometrics and face morphing look like?
We continue to work on new solutions in the identity management project, of course. We’ve identified and defined the main issue. Now we need to use and develop new machine learning methods in order to come up with high-quality, generalizable solutions and be able to detect unknown attacks in time.
Fraunhofer IGD is an institute of applied research, so our focus isn’t on developing a theory that sounds interesting in a scientific paper. Our focus is on creating solutions that are generalizable and can be used practically to combat the problem.