�鶹ӰԺ

With the busiest travel season of the year in full swing, visitors to Denver International Airport and at least 24 other U.S. airports are facing a new method of checking their identity this summer.

Instead of handing their driver’s license to a security staff member,  passengers slip it into a card-reader, look into a camera and wait for facial recognition software to match their current face to their ID. In a few airports, select travelers may skip presenting a physical ID altogether, as the system matches their face to an existing gallery of their government-issued photos.

The pilot project aims to speed up lines and help Transportation Security Administration (TSA) agents more easily “determine that you are who you say you are,” according to TSA officials, who hope to roll out the technology at hundreds more airports within a few years.

Critics contend the technology is a threat to civil liberties and could boost discrimination against already marginalized communities. Several U.S. senators to stop using it, citing research showing that Asian and African American people are up to 100 times more likely to be misidentified than white men. Meanwhile, advocacy groups, frustrated that travelers are often unaware they can opt out, have begun to submit to the government.

Morgan Klaus Scheuerman, a postdoctoral researcher in the Department of Information Science in the College of Media, Communication and Information, studies racial and gender bias in artificial intelligence (AI) systems. CU �鶹ӰԺ Today spoke with him about the potential pitfalls of such systems and what tech companies and governments are doing to address them.

How often do we encounter facial recognition software in our daily lives?

It’s difficult to tell because it's invisible. It can be deployed in law enforcement surveillance scenarios, for example, in closed-circuit TV (CCTV). Some malls have experimented with using it in digital kiosks in the past. And, of course, people would be most familiar with its use on their personal devices. Every time you unlock your phone you use it.

Why are people concerned?

Generally speaking, the concerns are around areas of bias—that the system might perform poorly on certain groups of people while performing better on others. Another area of concern is around consent and agency. So often, people are not necessarily aware that this technology is being deployed on them. In the realm of policing, they are concerned they could be misclassified as a suspect because facial recognition software matched them with an image from CCTV, or some prior record in the system that the police are using to match faces.

Is this really happening?

Yes. There have been some pretty high-profile cases, including several in which Black men were wrongfully arrested. (In one case in Detroit, a man was handcuffed in front of his young daughters and taken to jail after a facial recognition system incorrectly matched his driver’s license photo with a still image from a security video of a shoplifting incident.) In some cities, like San Francisco, facial recognition use by police is now banned. But police sometimes try to get around those bans by sending the images to another jurisdiction that uses it.

How accurate is facial recognition software?

There have been papers released on commercial software systems that argue for really high accuracy rates—over 90%. But then you have to ask: ‘What were the metrics they were using and who was represented in the data?’

Maybe it was highly accurate because the majority of people in their data sets were white men, but then once it's deployed, in reality, it doesn't work super well on people who are not white men. Some studies have shown that facial recognition software misidentifies women of color more than one-third of the time.  have shown it doesn’t work as well for women, children or the elderly.

What about transgender or nonbinary people?

In my own research, I looked at how commercial computer vision systems do when classifying the gender of transgender or nonbinary people. Of course, for nonbinary people, the system can't accurately classify them, because all the commercially released systems have historically used a gender binary. So, those who identify as agender, genderqueer or nonbinary were mischaracterized 100% of the time. When it came to the accuracy levels of transgender people, it was much, much worse than for cisgender people. Transgender men were wrongly identified as women up to 38% of the time.

 

 

Morgan Klaus Scheuerman

How could it be problematic to wrongly identify people?

Because these systems are often deployed in really high-risk scenarios with real-world consequences: For example, in the airport, there have already been known issues for transgender people because maybe their appearance has changed over time. Then they're flagged for something like fraud or they have to interact with the TSA in a way that is really uncomfortable for them. The burden of proof is on the person and not the system, which is not fair.

Has progress been made to address concerns about this technology?

Yes. In the case of gender, a lot of the commercial computer vision systems that I looked at a few years ago, including Google, have decided to not use gender in their models anymore. With law enforcement, there have been a lot of bills introduced—including one in Colorado—to limit the use of facial recognition by government. 

What do you make of its use in airports

I think the push toward using any kind of artificial intelligence technology is to help automate jobs and make processes cheaper and more efficient. So, I'm not surprised that airports are turning to these technologies. But I am concerned how certain people will be treated by the systems. If they are going to use it, I think they should be very transparent about it and how it's used and the fact that you can opt-out, which a lot of people probably aren’t aware of. I also think they, and other makers of facial recognition systems, should be very transparent about how they are training the system.

Is there a way to develop more equitable AI systems?

This a hard question that a lot of people in the ethical AI space are really contending with.

People often view computer systems as objective. But in reality all of these different human actors, from engineers, to researchers, to human labelers (tasked with looking at pictures and determining who is white, Black, male, female, etc.) are actually helping to co- construct what that system is. We have labor laws and advertising laws and housing laws against racial and gender discrimination but there are no laws specific to embedding discrimination into algorithms.

Could this AI genie be put back in the bottle?

When it comes to computer vision and facial recognition, I think they probably are here to stay. But we can change the course of how they're deployed and developed by conducting research in this space, proving when and where bias is occurring, and pushing legislators to really care about how they are being used.