Manipulated Video and Audio Will Make Future Fake News Even More Believable

New technologies suggest that it's just a matter of time before we're truly in a world of special effects, and distrustful of everything we see and hear.

Published on 1/6/2017 at 6:39 AM

An explosion of fake news stories is being blamed for tilting the presidential election toward Donald J. Trump. On Thursday at a Senate hearing on Russian hacking, Director of National Intelligence James Clapper said that Russian's campaign included "propaganda, disinformation and fake news" - and that those efforts were ongoing.

According to false news articles shared by the millions over social media in 2016, Pope Francis backed Trump, an FBI agent involved with Hillary Clinton's leaked emails was found dead in his apartment, and Clinton was a member of a child pedophile ring operated out of the basement of a Washington, D.C. pizzeria.

Pablo Boczkowski studies media and technology at Northwestern University and recently wrote a fascinating essay for NiemanLab on fake news and the future of journalism.

"Fake news is as old as true news," Boczkowski told Seeker. "There's always been misinformation. What we have now is an information infrastructure that is very different, with a scale and a scope that we haven't seen before."

Nearly 1.2 billion people visit Facebook every day, and 65 percent of adults say they access news through social media. Studies show that false news stories are shared on Facebook at the same rate as true ones, making it increasingly difficult for the average citizen to separate fact from fiction. But just wait... it's about to get much worse.

A team of young German and American researchers have demonstrated a mind-blowing real-time motion capture technology that can transform any world leader into a virtual puppet. The experimental software transfers facial expressions from any actor's face into video clips of political figures like Trump and Vladimir Putin.

In their paper about the photo-realistic reenactment technology, the researchers don't mention fake news. In fact, their work aims to detect edits in videos to establish their credibility. They believe that the ability to faithfully reconstruct facial movements in real time will allow for "on-the-fly dubbing" of translated video clips to make the speaker's lips match their words.

Another revolutionary technology from Adobe Creative Cloud will soon allow video producers to easily and accurately replace someone's dialogue with whatever they want the person to say. In this sneak peek video from Adobe Max 2016, developer Zeyu Jin puts brand new words into comic Keegan-Michael Key's mouth by simply entering them into a computer. (He also notes that his team is working hard to make the application of this tech detectable.)

While both of these technologies have legitimate applications, they also present incredible opportunities for abuse. With nothing more than a webcam, YouTube and this next-generation motion capture and audio editing software, a future fake news producer could publish convincingly real video clips of world leaders making any statement - or threat - simply by speaking or typing.

Filippo Menczer runs the Center for Complex Networks and Systems Research at Indiana University and studies how misinformation spreads on social networks like Facebook and Twitter. Menczer compares the rise of newer and better fake news tools to the "spam wars" of the past decade. Not that long ago, spambots flooded our inboxes with junk, but eventually email providers built smarter filters, making it much harder and much more expensive to be in the spam business.

Today, most spam gets trapped in the junk folder, but when will we have the same technological tools to filter out fake news?

"This is the typical arms race that we observe with any abuse of technology," Menczer told Seeker. "The first fake news bots were really easy to recognize, but now we have smarter social bots that are much harder to detect and better fake news websites that look real. It's a little scary to think what kind of damage [this new video and audio editing technology] might create."

Alexios Mantzarlis leads the International Fact-Checking Network for the Poynter Institute and closely tracks the efforts of researchers like Menczer to develop automated tools that can spot fake news before it spreads. He feels that fact-checking organizations and the mainstream press are currently outgunned.

"An 'arms race' implies that you have the same weapons," Mantzarlis told Seeker. "I think that fake news has really capitalized on existing technologies like social media newsfeed algorithms in ways that fact checkers just can't."

Menczer at Indiana University reports that online sharing of fact-checked content - the debunking of a false story, for instance - trails fake news by 10 to 20 hours. A 2016 report from Full Fact, a U.K. fact-checking organization, said that we have existing automated tools that can greatly speed up the debunking process, but we need further investment and international cooperation to get them in the hands of journalists.

Mantzarlis at Poynter said these automated tools are nothing like filters that would block or censor online content. They simply help fact-checkers "publish stuff faster" by recognizing claims that have been debunked before or quickly comparing a politician's statement - "Unemployment is down 5 percent!" - with the best online databases.

As fake news technology evolves to include hyper-realistic video and audio clips, it will take more than automated fact-checking tools or advertising bans imposed by Facebook and Google. We the readers and consumers of social news will have to "upgrade" ourselves.

"What needs to change is the culture of interpretation," said Boczkowski of Northwestern. "When a story about Pope Francis endorsing the candidacy of Donald Trump gets more than one million shares, that tells us that we have a culture of critique that's not ready to distinguish this kind of misinformation."

In his native country of Argentina, which Boczkowski said has a long history of populist leaders and biased media organizations, Argentinians are savvy and healthily distrustful media critics. They know who owns which newspaper and which TV networks are puppets of the government.

"The U.S. has been blessed with a very different media culture and a much healthier democratic culture for most of 20th century," said Boczkowski. Until now. The recent explosion of fake news and its potential role in elections will hopefully force all of us to take a much closer look at that next article or video, before we click "share."

Photo: Face2Face real-time face capture. Credit: YouTube WATCH: Can You Spot a Fake Smile?