Amazon’s Facial Recognition Software Is Prone to Racial Profiling, Report Says

Amazon’s business practices have been under fire from multiple fronts in 2018. Consumer advocacy groups claim that it sells books, toys and clothing with racist messages and symbols that violate the company’s own anti-hate speech policies.

In June, Amazon employees signed an open letter demanding that CEO Jeff Bezos stop selling AWS Rekognition facial recognition software to Immigration and Customs Enforcement (ICE). They feared the product was being used to racially profile immigrants. In the letter, employees said they “refuse to contribute to tools that violate human rights.”

A new report from the ACLU shows that those workers may be right.

Using Amazon’s Rekognition software and 25,000 publicly available arrest records, researchers built a database and a face-searching tool. They then searched the database against all members of the U.S. House and Senate.

“The false matches,” the ACLU writes, “were disproportionately of people of color, including six members of the Congressional Black Caucus, among them civil rights legend Rep. John Lewis (D-Ga.).”

The report also points out that the Congressional Black Caucus (CBC) had already raised concerns about accuracy and racial profiling to Amazon:

In a recent letter to Amazon CEO Jeff Bezos, the Congressional Black Caucus expressed concern about the ‘profound negative unintended consequences’ face surveillance could have for Black people, undocumented immigrants, and protesters.

The ACLU researchers believe that their findings back up the CBC’s fears. They write, “Nearly 40 percent of Rekognition’s false matches in our test were of people of color, even though they make up only 20 percent of Congress.”

When BuzzFeed contacted Amazon for a comment, a spokesperson defended Rekognition, saying the ACLU’s test would be improved by “ ‘following best practices’ around setting the confidence thresholds — the percentage likelihood that Rekognition found a match — used in the test.”

The spokesperson continued, “While 80% confidence is an acceptable threshold for photos of hot dogs, chairs, animals, or other social media use cases, it wouldn’t be appropriate for identifying individuals with a reasonable level of certainty. … When using facial recognition for law enforcement activities, we guide customers to set a higher threshold of at least 95% or higher.”

The ACLU countered that Amazon doesn’t ask what their clients are using the facial recognition software for. What’s more, Amazon might recommend certain parameters for law enforcement use, but that doesn’t mean law enforcement will follow those parameters.

Jacob Snow, an attorney for the ACLU of Northern California, also points out that Amazon’s own website suggests an 80 percent confidence in matching human faces, not the 95 percent the spokesperson claimed.

Snow told BuzzFeed he believes “[t]hese results demonstrate why Congress should join the ACLU in calling for a moratorium on law enforcement use of face surveillance.”

Lawmakers are also alarmed. Rep. Luis Gutiérrez, D-Ill., who is among those falsely matched with an arrest record, asked in an email to BuzzFeed, “Are you sure Amazon isn’t just talking to Donald Trump, Tucker Carlson and Breitbart? Because they think all Latinos are criminals.”

He continued, “But on a more serious note: If this technology has not been proven effective and has a systematic bias against people of color, then it will hurt, not help, law enforcement and anyone else who uses it.”