A Society of Clones: It Should No Longer Be On Us to Tell the Difference

An advert has been going around for several months. It looks like a television news report, and in it you see Cyprien presenting an app that supposedly makes you money. He never created that app and he never filmed that report. The edit is in fact a template: the same fake news segment circulates with Élise Lucet, with Nota Bene, with Squeezie, and all you have to change is the face and the voice. Cyprien eventually responded in a video and announced he wanted to press charges.

You could tell yourself this is the price of fame, that it comes with putting your face and your voice online every day. Except that is already no longer true.

Microsoft published a research paper twelve days ago called VASA-1. The method starts from a single image and an audio recording, and it generates that person speaking, on video, in real time. What is alarming here is that most people have a profile picture somewhere, and getting a recording of their voice is relatively easy. So you no longer need to be a public figure to have your image reused and made to say things. You just need to be someone that someone else wants to make speak.

What makes me smile, in a way, is that the platforms that started building this kind of model are the same ones now struggling to regulate it, even though they have every interest in doing so. Since March, YouTube has asked whoever uploads a video to declare for themselves whether it contains altered or synthetic content. That is better than nothing. But we built the technology that creates the problem, and we are asking the person using it to raise their hand.

I think something fairly simple needs saying here. It should no longer be on us to tell the difference. It should be on the platform showing us the image to tell us.

What I watched begin at Samsung

When I say these companies have no solution, that is not entirely true. Technical solutions exist, legislative solutions exist too, late but they exist. What I am saying, though, is that we could have seen all of this coming, and we could have put things in place a long time ago.

In 2019 I was at Samsung Research America, in the Think Tank Team, a group whose role was to push new uses and bring out new products. I was working on robotics myself. But I watched a project start up next to me whose goal was to create artificial humans. It became NEON, presented at CES the following year: avatars built from scratch, meant to carry a message and handle communication, in a controlled way.

And that is where the slide becomes very easy to make. Once you know how to build an avatar from scratch, you also know how to build someone’s avatar. One day people will create an avatar of themselves to represent them, which is rather good news. And on that same day, others will create one of a person who asked for nothing.

That question of consent, nobody asked it in the room I was in, and I do not think it was negligence. It was off topic. We were building a product, and a standard of proof fell outside the scope. Five years later the product exists, it is in everyone’s hands, and the standard of proof still has not been written.

Detecting a model’s signature

To understand what can be done, you have to look at what we are dealing with. A generated image usually comes out of a GAN or a diffusion model, and in both cases the reason it is hard to detect comes down to the method itself. A GAN explicitly aims to produce an image you cannot distinguish from a real one, and a diffusion model generates by probabilistically following the distribution of what it has observed. So we have built systems whose function is to not be recognisable.

Five years ago you spotted a generated image immediately. Then came that fairly interesting period where everything was nearly perfect, with small details that still gave it away. That is still somewhat true today, artefacts remain, but it takes an effort on every image, and that is exactly the problem: nobody is going to make that effort forty times a day.

Traces do remain, though. A recent paper, Mastering Deepfake Detection, shows that running a Fourier analysis over a large number of generated images reveals patterns specific to each family of models. Each model has a kind of signature, and that signature lets you say which generator produced the image in front of you.

That approach has a limit I find disqualifying on its own, though: the detector has to be readapted to every new generation method, since each method has its own signature. And it only takes an open-source model being fine-tuned for a particular case, or even fine-tuned explicitly to change its signature, for the detector to fall out of calibration. You end up running behind.

The watermark

Hence the second solution, the one DeepMind proposes with SynthID, and which is nothing new in itself: hiding a signal inside the image. We have known how to add watermarks for a long time, they are imperceptible to the naked eye and they can be read by decomposing the image signal. What DeepMind proposes is that any content generated by their models carries that watermark automatically, and above all that it be robust: that it survive a crop, a filter, a compression.

I think this is a big idea, and something important to hold on to. What is missing at this stage is less the technique than an agreement among the people publishing these models, on the form of the watermark, on what it should contain and on how to make it as hard as possible to remove. Today most generative models put online carry none. They do have a signature by construction, but an unintended one, and therefore a fragile one.

What it would give us, concretely

Say both mechanisms are in place at the companies displaying videos uploaded by users.

At upload time, the platform checks whether the content was generated, via the watermark or via automatic detection. If it was, it checks whether there is a human face. If there is, it asks the person uploading a question: is this your face, is this an avatar, and if not, do you actually have the right to use it?

That has three consequences, and I find all three more interesting than the detection debate.

The first is the label. Whoever is watching knows what they are watching, and they know it without having to go looking.

The second is legal. Whoever uploaded the video ticked a box knowingly, so they had all the information needed to understand what they were doing. That completely changes the position of the person whose face was used, on the day they want to defend themselves.

The third is the most operational: the platform can spot this kind of content very early, before it goes viral. Today we react to a complaint, once the video has already spread. Here, there was a doubt from the start, and it can be cut.

Open source obviously punches through all of this. A model published with a watermark can be retrained to stop carrying it, and it will be. But this is a security argument like any other: there are always people who get past the barriers, and as you raise the cost of entry, getting past becomes far more expensive. Statistically it still reduces the number of people who will do it, especially when what they want to do is fraudulent.

There is one area where I have nothing to offer, and I believe it is harder than this one: autonomous agents publishing text. Watermarking text is possible when it is long, it gets very difficult when it is short, and text is also what gets produced in the largest quantity. That is another problem.

For images and video, at least, the technical part is roughly there. What is missing is a decision, the decision to say that recognising a copy is not the job of the person looking at it. And that decision, unlike the watermark, could have been made five years ago.