Just How Dangerous Is Amazon’s Facial Recognition Program? (Audio and Transcript)

Listen to the full interview and read the transcript below:

Robert Scheer: Hi, this is Robert Scheer with another edition of Scheer Intelligence, where we get our intelligence from our guests. In this case, Jacob Snow, who is a technology and civil liberties attorney with the ACLU in San Francisco, a very important ACLU chapter, because you basically are looking at what Silicon Valley is doing, and what are the civil liberties issues involved in their technology. And the reason I was looking forward to having this discussion, you are dealing with a very important case involving Amazon’s face recognition software. And the fact is, this face recognition software is now being used increasingly by police departments, so we have a bleed-over into what governments do. And you folks did a test of the U.S. Congress, you used this software, and you discovered that using their standards, you falsely matched 28 members of Congress with mugshots of criminals. And had that been done by a police car or someone else, these people might have very well run into very serious difficulty. So why don’t you tell us, why Amazon, which has this wonderful reputation of servicing everyone, is expanding, is now a trillion-dollar company? Jeff Bezos owns the Washington Post personally, not Amazon, but Jeff Bezos who made all this money from Amazon, maybe, I guess arguably the richest man in the world right now. Why is the richest man in the world, and the company he has gotten rich off, a threat to our freedom?

Jacob Snow: Well, thanks, and it’s great to be here. Our work with respect to Amazon and facial recognition began when we saw that Amazon was marketing its facial recognition technology, which is called Rekognition, with a k, to law enforcement. And they were doing that in a way that indicated that they were encouraging law enforcement to use facial recognition as a tool of mass surveillance. So, they were telling law enforcement to use it to identify persons of interest in a crowd, or to identify individuals who would move around the city, to track faces across cameras, to use facial recognition in real time on the public. And that application of facial recognition was really concerning to us. And so, we sent Public Records Act requests to Washington County in Oregon, and to Orlando, Florida. And those Public Records Act requests revealed that Amazon was aggressively marketing its facial recognition technology to both of those places. And in Washington County in Oregon, the Public Records Act requests revealed that that sheriff’s office was using arrest photos as a matching database. And so, we decided to put Rekognition to the test in the same way that is being used by police departments in the United States right now. And so, we loaded a set of publicly available, 25,000 arrest photos, and we put that into recognition, and then we searched in that set for members of Congress, all current members of Congress. And the results that we found were pretty striking, and really concerning. Twenty-eight current members of Congress matched with an arrest photo. And legislators from both parties, from states around the country, were disproportionately and wrongly identified.

RS: Let’s explain that–wrongly identified.

JS: That’s right.

RS: Yeah, because people listening to this might [say], well, that’s a small percentage of members of Congress that might have mugshots–but these are people like John Lewis, and it was disproportionately people of color, I believe 40% were people of color, disproportionately women. But you know, famous civil rights hero, member of the Black Caucus, John Lewis from Atlanta, he was one of those identified as a criminal, falsely.

JS: That’s right, although I’d like to point out that arrest photos are not evidence that somebody is a criminal. People can be arrested, that just means they have been in contact with the police and they’ve been arrested, they haven’t been charged.

RS: Ah. So even by that dubious standard of mugshots, even that, they were misidentified.

JS: That’s right, and they haven’t been–arrest photos mean somebody hasn’t been charged or convicted of any crime or any wrongdoing. So, but as you point out, the members of Congress who were misidentified–and all of them were misidentified that we matched–were disproportionately people of color, and that included six members of the Congressional Black Caucus. And in Congress overall, 20 percent of members are people of color, but on our matches, 40 percent of those matches are people of color. So that indicates that face surveillance is really flawed, it’s biased, and it’s dangerous. And that’s why the ACLU is calling for a moratorium at the federal level for law enforcement use of facial recognition.

RS: So, if I could push you a little bit, this is the tip of a larger iceberg threatening our civil liberties, at least from my point of view. Because the assumption is that private corporations should be free to do what they do, if it’s convenient to their customers that they know everything about us, what we eat, what we read, our conversations–that’s in the private sector, and the U.S. Constitution, the Bill of Rights, was designed to protect us from government overreach. But in the case of Amazon and of a lot of these technology companies–certainly Palantir, which has a great surveillance; Google, which has a lot of contracts with the government, and so forth–they’re one and the same, in an uncomfortable way. Amazon is developing the big cloud for the intelligence agencies that has this great collection of all the data the government has on us. And here you have an example where Amazon is actively marketing this software to government agencies, police agencies, and others. So let me ask you, what are the constitutional implications of ostensibly private companies that have huge government contracts? We see that even with the NSA with Booz Allen and so forth. Could you sort of reflect on that?

JS: Sure. So, a few things. First of all, just to highlight that California has something called the California Electronic Communications Privacy Act, which protects information stored on service platforms and also in individuals’ devices. And in this day and age where a lot of our information is stored on cloud platforms, as you point out, and also our mobile phones represent a repository of some of the most private information in our lives, it’s important that that information is protected from government access. And CalECPA, the California Electronic Communications Privacy Act, requires a warrant for the government to access that information. But getting back to the question of how the government and private industry are collaborating in this way that’s concerning, you know, we’re seeing a lot of, a groundswell of opposition from the public and from members of Congress, looking at the interaction between platforms like Amazon and Microsoft and Google and the government. You saw recently with employee activism around the Google Project Maven, with automated weapons, where employees were saying that Google shouldn’t be engaging with and building weapons of war. And we also saw that around facial recognition, where Amazon employees, Amazon shareholders actually, wrote a letter to Jeff Bezos saying, it’s time to stop selling facial recognition to law enforcement.

RS: So but, again, address the constitutional question. Because the whole assumption of the American experiment, the American republic, was that private space is essential to freedom. And then in the case of the American Constitution, since we were a colony of England, the Founders were very conscious; they did not want even the government authority that they would be controlling with their new Constitution, to have unfettered right to your papers, your information. But now, you have a market economy and information age that depends on getting your information; that’s how they market to you. That’s why 90 percent of the money made on the internet goes to Google and Amazon and a few others, Facebook and so forth. So aren’t you really asking Amazon to stop being Amazon?

JS: No, we’re just asking them to stop selling facial recognition to governments. And one thing to be clear about is the Constitution doesn’t stop just because information is stored on a private platform. If the government is collecting information or accessing information, then the constitutional protections in the Fourth Amendment absolutely apply. And so there’s no exception just because Amazon and the government, whether it be state, local, or the federal government, are working together. But there is an important issue where it’s important for all of us to think about how information being stored on private platforms could be accessed by the government. And so some work that the ACLU of Northern California has done is to encourage those online platforms to have transparency reports, where they talk about government requests for information, whether those be for national security or law enforcement applications, and to disclose to the public when those applications are made. And that’s an important transparency mechanism, for the public to become informed about how the government is accessing information that’s stored on private platforms.

RS: In fact, there are a lot of liberal folks that I know–put myself in their own camp–are very harsh about some of the republicans on the Supreme Court. But Chief Justice John Roberts has been very strong on this very issue that we’re talking about, and particularly around protecting the data that’s in your cell phone. A couple of decisions where the Supreme Court, surprisingly, or maybe not–the conservatives, maybe they’re true conservatives and care about civil liberties–but where some strong decisions have come down saying local police cannot access your phone data, must have a warrant, and so forth. So the virtual internet wired world does not make the Constitution irrelevant–no, it makes it more relevant than ever. As an argument that Justice Roberts said, he said there’s more information on your phone than there is in your home, and it has to be protected. Well, you’re now talking about an advance in technology where people can be identified in a way that no secret police agency anywhere in the world ever dreamed about doing it. You know, your face, your–it’s gone way beyond fingerprints, it’s gone way beyond signature, right.  How did Amazon get into this business? Why are they doing this? Is it part of their marketing strategy?

JS: Well, it’s not clear why Amazon is doing it. And we are asking them to stop. But they have a cloud platform, as you point out; it’s called Amazon Web Services. And they offer a variety of services to all comers. And in the test that we did, we used the default settings; they’re available in Amazon Web Services for anyone, for any member of the public to use. And the fact that Amazon is deploying a tool which has these problems is really concerning for civil rights and civil liberties reasons. And it’s not clear to me why Amazon is doing it, and Amazon hasn’t been forthcoming in explaining their motivations, or why they’re enabling mass surveillance in the way that they are. But to get to your point about the recent Supreme Court decisions governing privacy, just this term in Carpenter v. United States, a case brought by the ACLU, the Supreme Court ruled five to four that the Constitution protects cell phone location data. So that’s not the device information, in the device, but the location data held by service providers. And actually, as you point out again, Chief Justice Roberts wrote the opinion in that case. And he wrote something that I think is really relevant to the discussion around facial recognition. He wrote, “A person does not surrender all Fourth Amendment protection by venturing into the public sphere. To the contrary, what one seeks to preserve as private, even in an area accessible to the public, may be constitutionally protected.” And you can’t leave the house without your face. And you move into the public sphere, and facial recognition really threatens to eliminate the ability for people to move around in public and go about their daily lives, whether they are attending a religious service, going to a medical clinic, associating with friends, going to a political protest–all of those things are subject to surveillance if the government can use facial recognition without restriction. And that’s why the ACLU is calling for strict prohibitions on facial recognition in Congress.

RS: Yeah. And let me just confess my own prejudice, thank God for the ACLU. And the great thing about, for people who don’t know about the ACLU, this is an organization that is often criticized by people who should know better. You know, like defending the rights of people to have free speech that you hate; you know, be they, the Nazis in Skokie was an infamous, or famous case. And what is bothersome now in the current climate is that many people, again, who should know better, want surveillance. They feel safer, they feel what’s the cost, if you don’t do anything–Eric Schmidt, famously, of Google once said just don’t do anything you would be embarrassed to have found out. But the point of the American Constitution and the need for private space is that you should be free to have thoughts others might find objectionable. You should be free to attend rallies that others don’t like. And the example you gave of the political space, we have a clear guarantee in our Constitution, the right of the people to peacefully assemble for a redress of grievances. But the assumption there is that they can do so without consequence, as long as they’re being peaceful, right? They–but if they can get their picture recorded, 20 years later someone says “you were at that demonstration”–perfectly legal demonstration, they didn’t do anything, maybe they just happened by and they were listening to a speaker. And that becomes part of their permanent record. That’s really the specter you’re holding out.

JS: That’s right. And you know, the notion of surveillance as a way to protect people and to encourage public safety is one that is very common, but the fact is that surveillance is used to target people who lack power in today’s society. You know, we’re in an environment where immigrants are being targeted and harmed, and where people of color are being targeted and are being persecuted. And the idea that facial recognition could become widespread as a tool of law enforcement is going to have disproportionate impacts on people of color, on political protesters, and also on immigrants. And once that infrastructure is built, once face surveillance is widespread in society, the damage to those communities can’t be undone. And so it’s really important for us all to be vigilant about the technology and the deployment of the technology. And more importantly, it’s important for Amazon to take responsibility for how its technology is being deployed in our communities. And Amazon, you know, should listen to the employees that have called for it to stop selling facial recognition to governments, and also from its shareholders and civil rights organizations, academics; 150,000 members of the public signed the petition that we distributed and dropped off at Amazon headquarters in Seattle. And Amazon needs to listen to all those voices calling for them to stop what they’re doing.

JS: I think that people are recognizing that in a time of crisis, where our communities are under attack, where immigrants are under attack, people of color are under attack, it’s, it comes down to all of us deciding that we will do what we can to stop that harm from happening. And the ACLU has a long history of doing that, but everyone else who works in technology companies or is a member of the public, a voting member of the public, can do a lot just by speaking up and trying to stop some of the harm that’s happening today.

RS: [omission for station break] We’re back with Jacob Snow. And I want to make this a little bit personal, because one of the reasons I’m doing these podcasts, I’m trying to find models of integrity and courage in an environment that is really incredibly seductive to selling out, to corruption, what have you.   And I suspect people working on the higher level for Palantir or Google or Amazon do very well. And I just want to introduce you a little, I know about you, but you came from small-town California. And you know, went to good schools, both on the local level and then got this great legal education. And in this community–and you’re not alone; at the Electronic Frontier Foundation there are people, there are a lot of people around the tech community that seem to have a great deal, sense of integrity. You mentioned employees at these companies, they must be risking something in their career if they speak out. And so let me ask you, why are you you? I mean, how come you decided to be with the ACLU rather than the people trying to justify invading our privacy, you’re out there protecting it?

JS: I think it’s important to have an avocation which helps the public. And I joined the ACLU fairly recently, and before that I was an attorney at the Federal Trade Commission here in San Francisco, doing consumer protection and antitrust work, mostly litigation. And I felt like that was worthwhile as well. I think the ACLU gets a lot of, gets a lot of attention, rightly so, and gets a lot of credit, but there are a lot of people working inside of government and outside of government to make their lives better, but also the lives of everyone we live with in our society better. And so you know, everyone is doing their part, and I certainly try.

RS: Well, it’s very encouraging, because frankly, we’ve had these big, you know, very, very important historically, dystopian novels; Brave New World, right, 1984 and so forth. And sometimes you can wake up in the morning and look at the news and listen to it and think, hey, we’ve already gone pretty far down that road, you know. If you think about, you know, what the powers that be–and I put private industry and its collaboration with government right there. And not only here; obviously in China, in Egypt, in Russia, right? All over the world–they can use the same technology. And in fact, there’s a big battle, will Google go into China and cooperate with that government now, and that means tailoring, I suppose, searches, and maybe handing over information. I mean, this is a–these companies are multinational, they operate throughout the world, so in a way, this battle you’re having here with Amazon over facial recognition, this is a battle that you’re really fighting for the entire world. Because if this, you know, you can do it here in the land of the free, in the U.S., then what’s to stop any dictator anywhere in the world saying, hey, I now have the ability, I can monitor where you’ve been, who you’ve been with, every crowd, every meeting, every restaurant–I got you nailed, you better shut up. And then you get into this dark world that 1984 and Brave New World described of, basically, self-censorship, and being deluded into thinking you’re free when you’re not free at all. When you raised this, I assume you raised some of these arguments, how do people on the other side, who are representing Amazon or some of these other companies, how did they respond?

JS: Well, I’d like to speak to the point you made about inevitability. Because I think a lot of people look at technology, they look at the pervasiveness of data-gathering in our lives, they look at Facebook and Google collecting information about all of us; and they look at facial recognition, and they think that that technology is inevitable, and that it will become pervasive, and that there’s nothing we can do about it. Just in the last few months, because of the work that the ACLU has done, you know, we’ve seen attention being drawn to this issue, not just in the press and not just in the public imagination, but in Congress itself. I mean, after we released the first results of our study in May, members of Congress wrote a letter to Amazon; in fact, the Congressional Black Caucus wrote a letter to Amazon calling for attention to the issues of civil rights that are implicated by facial recognition. And the same day as we released the study about lawmakers matching, incorrectly matching with arrest photos, Representative John Lewis and Representative Jimmy Gomez sent a letter to Amazon demanding an in-person meeting with Jeff Bezos, and saying that Jeff Bezos needs to take seriously the civil rights implications of its technology. Since then, Senator [Edward] Markey has sent a letter to Amazon asking for more information, and Senator [Cory] Booker and Senator Markey and Senator [Ron] Wyden have sent a letter asking 39 federal agencies for more information about how they use facial recognition.  It’s important for people to realize that there is a groundswell of opposition, not just to facial recognition, but to the pervasiveness of surveillance in our society, and that it is not inevitable; it’s just necessary for us all to stand up and oppose it. And we’re hoping that Congress will take action on this.

RS: It’s an interesting question of the spread of the power of these companies. I mean, here is Amazon, Jeff Bezos, who personally owns the Washington Post, which is a newspaper we have traditionally counted on to help monitor government in Washington. Well, suddenly, these people that you’re going to be appealing to in Congress, to get them to take a harder look at what Amazon’s doing, they have to respond to a, the major news organization in their town is owned by this person. And then this is a company that also owns Whole Foods, which is present in every community. And we used to have a great fear–and you mentioned the Federal Trade Commission–we used to have a great fear written into our laws against bigness, against monopoly, against cartels. And yet in the name of modernity, we accept that these industries are totally dominated by these larger companies than we’ve ever had. They make Standard Oil of old seem puny. Do you ever have the David and Goliath feeling here, living in the Bay Area close to Silicon Valley, that you’re up against some formidable odds?

JS: Well, companies are certainly powerful. And Amazon is a very wealthy company, and it’s run by Jeff Bezos, a very wealthy man. But the thing that I find is an antidote to what is the fact that our political process certainly has its problems, but it can work. And elected representatives can impose meaningful restrictions on these companies; we’ve seen it happen in the past with some efficacy, and I believe that it can happen in the future. So you know, the consolidation in the American economy is cause for real concern, but I guess I would say, I would agree with you in that I’m an optimist, in the sense that I think these, that consolidation can be fought through the democratic process and through the political process.

RS: Well, it’s good that you’re optimistic, or you might give up and go find another line of work. And let me just say, I want to, I don’t want to discount the positive, but it’s interesting. The very name that Amazon gave to this program, if people are listening and they want to write about it or think about it, it’s called Rekognition, with a k. And when I read that, I thought, this is out of some comic or something, right? Wasn’t that the old way of, you sort of said America was getting totalitarian, you put a k, a couple of k’s in the middle of America? Or the KKK or what have you? Are they really branding this as Rekognition with a k?

JS: They are branding it as Rekognition with a k. And I wouldn’t read into the marketing decisions that Amazon might have made in naming that product, but you know, there’s a lot of work showing that facial recognition has a race problem. It started with academic work coming out of MIT by Joy Buolamwini and Timnit Gebru, showing that facial recognition products are less accurate for faces, for darker-skinned faces and for faces of women. And then you also see similar work coming out of other academic sources as well. And so you know, no matter what the marketing decision in naming the product was, Amazon really needs to address the possibility that its product has a racial bias problem. And you know, we’ve asked that the company release records about how it tests its product internally for bias, how it tests it for accuracy. And members of Congress have asked those same questions. But to my knowledge, Amazon hasn’t responded. And so you know, it can name its products whatever it wants, but when it comes to the consequences for communities, it needs to be transparent, and we’re not seeing that.

RS: OK. We’re going to wrap this up, but just to be clear, we are talking about when a private company is marketing, not just to individual consumers and to use within, but as your report indicates, you mentioned two police departments, in Oregon and Florida–but essentially, as with Palantir, we have a company that has marketed its products, which is a surveillance, data-collecting, sifting through identification system–they seem to be in almost every jurisdiction now.  We talk about private business. But this is private business in the same way that Lockheed Martin and others in the defense industry were private business. They’re increasingly connected with what used to be called the military-industrial complex, to take General Eisenhower’s word, but now maybe it’s the military-industrial information complex. And they just move seamlessly with the intelligence agencies, with police departments. They are an integral part of a surveillance society, are they not?

JS: They are necessary for surveillance to be implemented on a modern and massive scale. And that’s why it’s important not to just focus on surveillance happening by the government, but how private industry is enabling that. And that’s a big motivation for our work. So, you know, Amazon’s deployment of face surveillance technology is certainly problematic, but we’re not seeing it being limited to Amazon. Microsoft has a similar product, other companies have a similar product, but some of those companies are taking a stand against using face surveillance in government and in law enforcement. There’s a company called Kairos, the CEO is a man named Brian Brackeen. And he has announced that his company won’t sell facial recognition to law enforcement. And that’s a decision that is not good for his company’s bottom line, but it’s an ethical stand that he’s taken. And that’s an example of an individual who has a deep understanding both of the business of using facial recognition, but also of the technology underlying facial recognition. And he’s taken a stand where his company won’t sell facial recognition to law enforcement, and other companies should be following Brian’s example.

RS: Thank you, Jacob Snow, technology and civil liberties attorney for the Northern California ACLU. And thank you, Kat Yore and Mario Diaz, our engineers at KCRW in Santa Monica. Our producers are Joshua Scheer and Isabel Carreon. And I want to have a special shout-out to Darren Peck at Sports Byline for providing these studios for us. Thank you.

JS: Thanks so much.