Twitter CDO Acknowledges 'Racist' Photo Algorithm: 'It's 100 Percent Our Fault'

Twitter's chief design officer, Dantley Davis, has acknowledged evidence of racial bias in how a neural network used by the platform generates photo previews.

Teams from the social media site will investigate an algorithm that is used to decide which sections of an image are cropped and displayed in tweet previews, after tests over the weekend indicated it favors white faces over black faces.

The neural network deployed by Twitter automatically crops images that are uploaded to the platform, deciding which "salient" parts are shown in a post.

A neural network—a fork of artificial intelligence (AI)—is a type of machine learning that is loosely modeled on the human brain. It enables computers to recognise patterns or make their own decisions based on analysis of vast quantities of data.

Davis conducted his own experiments into claims of bias on Saturday after a user flagged his Black colleague appeared to have been cropped out of a preview when a picture of them both was shared on mobile.

The user initially noted how the video software Zoom had been removing his colleague's head when adding a virtual background. His posts kicked off a series of informal tests focused on Twitter's own algorithm.

Geez...any guesses why @Twitter defaulted to show only the right side of the picture on mobile?

— colin should probably be reading or writing (@colinmadland) September 19, 2020

One widely-shared example suggested Twitter's algorithm repeatedly favored a picture of Sen. Mitch McConnell over former President Barack Obama, while the problem even appeared to exist when users uploaded images of characters from The Simpsons.

Trying a horrible experiment...

Which will the Twitter algorithm pick: Mitch McConnell or Barack Obama?

— Tony “Abolish (Pol)ICE” Arcieri 🦀 (@bascule) September 19, 2020

I wonder if Twitter does this to fictional characters too.

Lenny Carl

— Jordan Simonovski (@_jsimonovski) September 20, 2020

According to Davis, the algorithm had possibly favored the white user due to the way his facial hair had been contrasting with his skin. In a trial, the Black man was favored in a tweet preview after the white man's facial hair was edited out of the picture.

Davis said that the neural network does not explicitly use faces to make its decisions but "contrast can be problematic." Stressing that the team had previously tested for racial bias before releasing the algorithm, he tweeted: "Even facial hair seems to affect the model. We're looking into other effects based on the feedback we're getting."

Responding directly to a Twitter user who said the site needed to take responsibility for the effects of its actions, Davis wrote: "It's 100% our fault. No one should say otherwise. Now the next step is fixing it." He told another user there is "lots to improve" with the AI.

Based on some experiments I tried, I think @colinmadland's facial hair is affecting the model because of the contrast with his skin. I removed his facial hair and the Black man shows in the preview for me. Our team did test for racial bias before shipping the model.

— Dantley Davis (@dantley) September 19, 2020

It’s 100% our fault. No one should say otherwise. Now the next step is fixing it.

— Dantley Davis (@dantley) September 19, 2020

In a statement Sunday, Twitter said: "We tested for bias before shipping the model and didn't find evidence of racial or gender bias in our testing.

"But it's clear that we've got more analysis to do. We'll continue to share what we learn, what actions we take, & will open source it so others can review and replicate."

A similarly-worded statement was tweeted by Twitter communications chief Liz Kelley, who thanked everyone who raised the suspected issue to the site.

In a 2018 blog post detailing the inner-workings of the neural network, Twitter said that the algorithm crops images based on "salient" regions. It previously used face detection but said that approach had "obvious limitations" because not all pictures contain faces.

It is unclear how long it will take for Twitter to investigate the AI system. "This is a very important question," chief technology officer Parag Agrawal wrote Sunday.

"To address it, we did analysis on our model when we shipped it, but needs continuous improvement. Love this public, open, and rigorous test—and eager to learn from this."

Zoom has been contacted for comment by Newsweek.

Twitter logo
A stock image showing a Twitter logo is displayed on a mobile. The social media company said it will be investigating after its AI was accused of racial bias. OLIVIER DOULIERY/AFP/Getty