technologyreview.com

Think that AI will help put a stop to fake news? The US military isn’t so sure.

The Department of Defense is funding a project that will try to determine whether the increasingly real-looking fake video and audio generated by artificial intelligence might soon be impossible to distinguish from the real thing—even for another AI system.

This summer, under a project funded by the Defense Advanced Research Projects Agency (DARPA), the world’s leading digital forensics experts will gather for an AI fakery contest. They will compete to generate the most convincing AI-generated fake video, imagery, and audio—and they will also try to develop tools that can catch these counterfeits automatically.

The contest will include so-called “deepfakes,” videos in which one person’s face is stitched onto another person’s body. Rather predictably, the technology has already been used to generate a number of counterfeit celebrity porn videos. But the method could also be used to create a clip of a politician saying or doing something outrageous.

DARPA’s technologists are especially concerned about a relatively new AI technique that could make AI fakery almost impossible to spot automatically. Using what are known as generative adversarial networks, or GANs, it is possible to generate stunningly realistic artificial imagery.

“Theoretically, if you gave a GAN all the techniques we know to detect it, it could pass all of those techniques,” says David Gunning, the DARPA program manager in charge of the project. “We don’t know if there’s a limit. It’s unclear.”

A GAN consists of two components. The first, known as the “actor,” tries to learn the statistical patterns in a data set, such as a set of images or videos, and then generate convincing synthetic pieces of data. The second, called the “critic,” tries to distinguish between real and fake examples. Feedback from the critic enables the actor to produce ever-more-realistic examples. And because GANs are designed to outwit an AI system already, it is unclear if any automated system could catch them.

GANs are relatively new, but they have taken the machine-learning scene by storm (see “The GANfather: The man who’s given machines the gift of imagination”). They can already be used to dream up very realistic imaginary celebrities or to convincingly modify images by changing a frown into a smile or turning night into day.

Walter Scheirer, a digital forensics expert at the University of Notre Dame who is involved with the DARPA project, says that the technology has come a surprisingly long way since the initiative was launched a couple of years ago. “We are definitely in an arms race,” he says.

While it has long been possible for a skilled graphics expert to produce convincing-looking forgeries, AI will make the technology far more accessible. “It’s gone from state-sponsored actors and Hollywood to someone on Reddit,” says Hany Farid, a professor at Dartmouth who specializes in digital forensics. “The urgency we feel now is in protecting democracy.”