Amazon's facial recognition tech incorrectly identified 28 members of Congress as people who had been arrested for committing a crime, testing of the software discovered.
Analysis from the American Civil Liberties Union (ACLU) this week said a dataset of publicly available mugshots had been "disproportionately" matched with people of color, including civil rights pioneer Representative John Lewis and five other members of the Congressional Black Caucus. U.S. politicians are now demanding answers from Amazon CEO Jeff Bezos over the results.
The members of Congress—both Republican and Democrat—included men and women of differing ages from across the country, ACLU attorney Jacob Snow said in a blog post on Thursday. The test was built on a sample of 25,000 mugshots and, using the tool's default settings, compared those against a database of every current House and Senate member.
The facial recognition tool, called "Rekognition," is already used by U.S. law enforcement. Last month, the city of Orlando stopped using the software in the wake of the ACLU claiming its use lacked the public's consent and could be abused to "conduct mass location tracking of residents without criminal suspicion." The system, Amazon says, indexes 100 faces from a single image.
"Congress must take these threats seriously, hit the brakes, and enact a moratorium on law enforcement use of face recognition," Snow wrote. "This technology shouldn't be used until the harms are fully considered and all necessary steps are taken to prevent them from harming vulnerable communities." The full list of impacted members of Congress was posted online.
In a response to Business Insider, Amazon suggested the ACLU test results could have been improved by changing the "confidence thresholds," which alter the likelihood of a real match.
A statement said: "While 80 percent confidence is an acceptable threshold for photos of hot dogs, chairs, animals, or other social media use cases, it wouldn't be appropriate for identifying individuals with a reasonable level of certainty." It added: "When using facial recognition for law enforcement activities, we guide customers to set a threshold of at least 95% or higher."
The company said that it remained "excited" about the potential of image and video analysis across the public sector and law enforcement. Google and Microsoft also sell similar software.
Amazon did not immediately respond to a request for comment.
The high-level U.S. politicians did not appear convinced by Amazon's spin on the ACLU report. Several have sent letters—addressed to Bezos—requesting meetings with the billionaire CEO.
"We are alarmed about the deleterious impact...this tool—if left unchecked without proper, consistent, and rigorous calibration—will have on communities of color; immigrants; protestors…and others petitioning the government," Jimmy Gomez and John Lewis argued.
Amazon’s facial recognition misidentified 28 lawmakers, including me, as people arrested for a crime. It’s clearly inaccurate tech, but law enforcement may want to use it for surveillance. We’re demanding answers on the privacy and civil rights implications of “Rekognition”. pic.twitter.com/PVI9nUwfcu
— Ed Markey (@SenMarkey) July 26, 2018
"These lawmakers and others are standing up and holding Amazon's feet to the fire," commented ACLU legislative counsel Neema Singh Guliani after the letters emerged.
"It's disappointing that Amazon continues to bury its head in the sand in the face of concrete and legitimate concerns being raised about its technology," Guliani added. "In doing so, they ignore the real privacy, civil liberties, and safety risks of their technology. Congress should continue to demand Amazon take responsibility for the technology it sells to law enforcement."
The civil rights group alleged that Amazon makes little effort to audit how its software is used, once sold. In addition, ACLU attorney Snow rubbished the response from the e-retailing giant.
"Amazon seems to have missed, or refuses to acknowledge, the broader point: Face surveillance technology in the hands of government is primed for abuse and raises grave civil rights concerns," he wrote. "It could allow—and in some cases has already enabled—police to track protesters, ICE to continuously monitor immigrants, and cities to surveil their own residents. Changing the matching threshold doesn't change the danger. In fact, it could exacerbate it."
In June, Amazon staffers circulated a letter that asked Bezos to stop selling the face-scanning software to police. "Our company should not be in the surveillance business," they appealed.
Amazon asserted there "has been no reported law enforcement abuse of Amazon Rekognition."
