Company
Published
14 May 2024
Announcing our novel watermarking method for AI-generated text and video, and how we’re bringing SynthID to key Google productsGenerative AI tools — and the large language model technologies behind them — have captured the public imagination. From helping with work tasks to enhancing creativity, these tools are quickly becoming part of products that are used by millions of people in their daily lives.These technologies can be hugely beneficial but as they become increasingly popular to use, the risk increases of people causing accidental or intentional harms, like spreading misinformation and phishing, if AI-generated content isn’t properly identified. That’s why last year, we launched SynthID, our novel digital toolkit for watermarking AI-generated content.Today, we’re expanding SynthID’s capabilities to watermarking AI-generated text in the Gemini app and web experience, and video in Veo, our most capable generative video model.SynthID for text is designed to complement most widely-available AI text generation models and for deploying at scale, while SynthID for video builds upon our image and audio watermarking method to include all frames in generated videos. This innovative method embeds an imperceptible watermark without impacting the quality, accuracy, creativity or speed of the text or video generation process.SynthID isn’t a silver bullet for identifying AI generated content, but is an important building block for developing more reliable AI identification tools and can help millions of people make informed decisions about how they interact with AI-generated content. Later this summer, we’re planning to open-source SynthID for text watermarking, so developers can build with this technology and incorporate it into their models.
How text watermarking worksLarge language models generate sequences of text when given a prompt like, “Explain quantum mechanics to me like I’m five” or “What’s your favorite fruit?”. LLMs predict which token most likely follows another, one token at a time.Tokens are the building blocks a generative model uses for processing information. In this case, they can be a single character, word or part of a phrase. Each possible token is assigned a score, which is the percentage chance of it being the right one. Tokens with higher scores are more likely to be used. LLMs repeat these steps to build a coherent response.SynthID is designed to embed imperceptible watermarks directly into the text generation process. It does this by introducing additional information in the token distribution at the point of generation by modulating the likelihood of tokens being generated — all without compromising the quality, accuracy, creativity or speed of the text generation.
SynthID adjusts the probability score of tokens generated by a large language model.
The final pattern of scores for both the model’s word choices combined with the adjusted probability scores are considered the watermark. This pattern of scores is compared with the expected pattern of scores for watermarked and unwatermarked text, helping SynthID detect if an AI tool generated the text or if it might come from other sources.
A piece of text generated by Gemini with the watermark highlighted in blue.
The benefits and limitations of this techniqueSynthID for text watermarking works best when a language model generates longer responses, and in diverse ways — like when it’s prompted to generate an essay, a theater script or variations on an email.It performs well even under some transformations, such as cropping pieces of text, modifying a few words and mild paraphrasing. However, its confidence scores can be greatly reduced when an AI-generated text is thoroughly rewritten or translated to another language.SynthID text watermarking is less effective on responses to factual prompts because there are fewer opportunities to adjust the token distribution without affecting the factual accuracy. This includes prompts like “What is the capital of France?” or queries where little or no variation is expected like “recite a William Wordsworth poem”.Many currently available AI detection tools use algorithms for labeling and sorting data, known as classifiers. These classifiers often only perform well on particular tasks, which makes them less flexible. When the same classifier is applied across different types of platforms and content, its performance isn’t always reliable or consistent. This can lead to a text being mislabeled, which can cause problems, for example, where text might be incorrectly identified as AI-generated.SynthID works effectively on its own, but it can also be combined with other AI detection approaches to give better coverage across content types and platforms. While this technique isn’t built to directly stop motivated adversaries like cyberattackers or hackers from causing harm, it can make it harder to use AI-generated content for malicious purposes.
How video watermarking worksAt this year’s I/O we announced Veo, our most capable generative video model. While video generation technologies aren’t as widely available as image generation technologies, they’re rapidly evolving and it’ll become increasingly important to help people know if a video is generated by an AI or not.Videos are composed of individual frames or still images. So we developed a watermarking technique inspired by our SynthID for image tool. This technique embeds a watermark directly into the pixels of every video frame, making it imperceptible to the human eye, but detectable for identification.Empowering people with knowledge of when they’re interacting with AI-generated media can play an important role in helping prevent the spread of misinformation. Starting today, all videos generated by Veo on VideoFX will be watermarked by SynthID.
SynthID for video watermarking marks every frame of a generated video
Bringing SynthID to the broader AI ecosystemSynthID’s text watermarking technology is designed to be compatible with most AI text generation models and for scaling across different content types and platforms. To help prevent widespread misuse of AI-generated content, we’re working on bringing this technology to the broader AI ecosystem.This summer, we’re planning to publish more about our text watermarking technology in a detailed research paper, and we’ll open-source SynthID text watermarking through our updated Responsible Generative AI Toolkit, which provides guidance and essential tools for creating safer AI applications, so developers can build with this technology and incorporate it into their models.
AcknowledgementsThe SynthID text watermarking project was led by Sumanth Dathathri and Pushmeet Kohli, with key research and engineering contributions from (listed alphabetically): Vandana Bachani, Sumedh Ghaisas, Po-Sen Huang, Rob McAdam, Abi See and Johannes Welbl.Thanks to Po-Sen Huang and Johannes Welbl for helping initiate the project. Thanks to Brad Hekman, Cip Baetu, Nir Shabat, Niccolò Dal Santo, Valentin Anklin and Majd Al Merey for collaborating on product integration; Borja de Balle Pigem, Rudy Bunel, Taylan Cemgil, Sven Gowal, Jamie Hayes, Alex Kaskasoli, Ilia Shumailov, Tatiana Matejovicova and Robert Stanforth for technical input and feedback. Thanks also to many others who contributed across Google DeepMind and Google, including our partners at Gemini and CoreML.The SynthID video watermarking project was led by Sven Gowal and Pushmeet Kohli, with key contributions from (listed alphabetically): Rudy Bunel, Christina Kouridi, Guillermo Ortiz-Jimenez, Sylvestre-Alvise Rebuffi, Florian Stimberg and David Stutz. Additional thanks to Jamie Hayes and others listed above.Thanks to Nidhi Vyas and Zahra Ahmed for driving SynthID product delivery.