Twitter will label and remove deepfake videos under a new policy

Twitter Inc. will start labeling, and in some cases removing, doctored or manipulated media that users share on the platform beginning next month.

The policy includes what are known as deepfake videos, which have been edited using artificial intelligence or other advanced software to distort a person’s appearance or speech while appearing to be authentic, as well as other kinds of edited photos and videos that have been “substantially edited,” the company said Tuesday in a blog post.

In some cases, Twitter will label the video or photo so users know it has been doctored but will let it remain on the social media network. In other cases, when the content might “impact public safety or cause serious harm,” Twitter said the video or photo would likely be removed.

Identifying and responding to manipulated media have become matters of urgency for social media platforms as the 2020 U.S. presidential campaign is underway. There is concern that politicians, or bad actors from outside the established political system, may try to influence public opinion through misleading videos or photos shared on social media.

Twitter’s new “manipulated media” policy, which will take effect March 5, was released more than three months after the company announced it would research and ask the public about various options for handling doctored videos. The company said it settled on a system that looked at content in three ways:

• Was the video or photo “significantly” altered?

• Was it shared “in a deceptive manner”?

• Is it a threat to public safety, or could it cause user harm?

If the answer to either of the first two questions is “yes,” then the media could be labeled with a warning to those who view it, and those who like or retweet it. If Twitter deems the content could also pose real-world harm, it is likely to be removed, said Del Harvey, Twitter’s head of trust and safety, on a conference call with journalists.

Last May, a doctored video of House Speaker Nancy Pelosi was shared on Facebook that made it look as though the congresswoman was slurring her speech. Facebook Inc. elected to leave the video on the site despite calls from Pelosi and other politicians to take it down. Facebook eventually labeled the video as fake and cut its distribution but took more than a day to do so, a delay Chief Executive Mark Zuckerberg later called an “execution mistake.”

Under Twitter’s new policy, that same video would have been labeled because it was “significantly and deceptively altered,” said Yoel Roth, Twitter’s head of site integrity. It wouldn’t have been removed.

Facebook has released its own policy on manipulated content since the Pelosi video. In Facebook’s case, the company will only remove content if it’s misleading and was created using “artificial intelligence or machine learning” technology. Otherwise, it can be fact-checked by the company’s third-party fact checkers and may receive a label and limited distribution.

Twitter said its policy was written specifically to include doctored videos and photos regardless of the sophistication of the editing technology. That means Twitter may remove content that is harmful, even if it doesn’t fit the specific definition of a “deepfake.”

“Our focus on this policy is to look at the outcome,” Roth said, “not how it was achieved.”

Twitter’s and Facebook’s efforts to curb the spread of disinformation sometimes clash with their pledge to uphold free speech for global users. With the Pelosi video, both companies decided it didn’t cause a threat to user safety, but some critics argued it did and might have caused serious harm to Pelosi’s reputation. The congresswoman has since called Facebook “shameful” for its handling of misinformation.

“This new rule is not meant to be the final word on this issue,” Harvey said. “But it is an important new change that we will continue to iterate on and expand on as we identify other areas and issues that we need to make sure we can address.”