How AI Text Watermarking Works (And How to Evade It)
Anthropic is adding text watermarks to Claude-developed content. Here's how AI text watermarking works, and how to evade it.
Earlier this month, Anthropic announced that it would be “working to include machine-readable marks in content that Claude generates.” It took this action to comply with the “EU AI Act’s Article 50 Code of Practice on Transparency of AI-Generated Content.” The watermarking will take two forms: “ (1) watermarks embedded in text, and (2) signed provenance metadata attached to files.”
Importantly, the Act specifies that the “marking obligation should not cover AI systems performing primarily an assistive function for standard editing or AI systems not substantially altering the input data provided by the deployer or the semantics thereof.” In plain language: There is absolutely no requirement to mark content that is only used for editing content or if the AI did not meaningfully change content produced by a human.
Anthropic has no obligation to introduce a blanket watermark on all Claude-generated content. However, it’s not practical for Anthropic to determine when the line is being crossed, so all content is being watermarked. According to Anthropic, when Claude “generates text, it weaves an imperceptible watermark directly into the text itself. You won’t see it, and it doesn’t change the meaning, quality, or readability of Claude’s response.” Importantly, “[b]ecause the watermark is part of the text, it will travel with the text when it’s copied and pasted elsewhere, and may persist through some editing.” The last part is critical, which I’ll discuss later.
Anthropic isn’t the only organization to commit to marking AI-generated content. According to the European Commission: “Several well-established and prominent AI companies have committed to adhering to the code. Examples ... include: Aleph Alpha, Anthropic, Black Forest Labs, Cohere, Google, Meta, Microsoft, Mistral, Open AI, Synthesia.”
This means watermarking of AI-generated image, video audio and text content will be widespread. There are real benefits to AI content identification. For example, it can be used to prevent deep fakes and criminal activity using voice cloning. Another benefit is, when done properly, it can improve other AI-text detection techniques.
Some of these systems use machine learning to identify whether a text has been AI generated. However, they aren’t perfect and have high false positive rates (occasions when human-generated text is mistakenly identified as AI-written and vice-versa). Watermarking may be much more reliable. It can also be defeated.
The purpose of this essay is to help you understand the mechanics of AI text watermarking from a first principles perspective. How:
Watermarking is implemented
AI text watermarks are detected
The ways it can be defeated
How the scale of AI assistance can be measured using watermarks (this directly addreses concerns about watermarking being applied to light or minimal AI changes to text, e.g., for spelling corrections or light edits)
How LLMs Generate Text
Because AI text watermarking is conducted at the model level, we’ll begin with a high-level overview of how AI models actually generate text. For many, Large Language Models (or LLMs) are magical black boxes that can generate text or answer questions on-demand, with distinct personalities. LLMs don’t have personalities, but that’s the topic of a future essay.
The important thing to understand is that there is a standard pipeline used in LLMs to ‘ingest’ text and output a response. Watermarking involves modifying how LLMs generate text.
Text Conversion
Let’s say you decide to ask an LLM to tell you what words should be follow the phrase “The quick brown fox”. You input the text into a chatbox and the LLM provides a response. But how did the LLM do this? Let’s go through the steps.
First, a system called a tokenizer takes the words “the quick brown fox” and converts them into small units represented as a list of numbers, or vectors. A vector can look like this, for example: Token: ‘The’, vector [-0.12, 0.45, 0.03, ...] (768 numbers in the list).
This process of translating a text string into numbers is called embedding. This is required so that the LLM can process text strings.
Token Matching
Each token is matched to a large table of vectors that are in the model’s vocabulary. Because word ordering is important, the model also assigns a position id to the vectors. For example if the word “quick” was represented by a vector consisting of 768 numbers, it would be assigned a position id of 2 because it is the 2nd number in the phrase.
At their core, vectors are numerical representations. They tell the model that ‘the’ equals a specific list of numbers that represent a word. Words that are related are assigned vectors with similar numbers. This allows models to make associations between words. These associations are provided to LLMs during model training.
Transformation: Or Token Enrichment
After embedding, the vectors flow through the LLM’s transformer layers. During transformation, individual tokens are enriched with contextual information (e.g., a fox is an animal). An important step is that the model determines token relevance and semantic meaning, or the context of the tokens. Information about token definitions and relevance is provided to the model during its training.
After the transformation step, the model uses all this information to output a series of tokens with a score, or logit. This score represents the model answering the question: “based on what it knows about the input tokens, what are the most relevant tokens it should output?”
Token Scoring and Probability Matching
The model outputs a list of potential candidate tokens or logits. For example:
Jumps: 8.2
Lazy: 5.2
Over: 1.8
a: 1.1
Dog: 1.05
Brown: 0.4
The next stage is the crucial one: A calculation called the softmax function takes the logits provided by the model and converts them to probabilities, from 0 to 100%. This conversion answers the question: “what are the most likely relevant tokens related to this input?”
Each token is assigned a probability. For example:
Jumps: 0.91 (or 91%)
Lazy: 0.05
Over: 0.02
a: 0.01
Dog: 0.005
Brown: 0.005
It’s easy to assume that the model always picks the token with the highest probability. But, this isn’t the case. Instead a few different modifiers can be used to weight the probability of token selection in different ways to make the output more creative.
After this additional weighting process tokens (and their positions) are picked at random from the list of high-probability tokens provided by the model. This is what the model outputs to answer the question: “What comes after “A quick brown fox?” The model’s answer: ‘jumps over a lazy dog.’”
How Text Watermarking is Added and Detected
The specific method Anthropic and other AI labs will be using to watermark text has not yet been revealed. However, we have some prior work to guide us on some potential methods. In In 2024, John Kirchenbauer and colleagues published A Watermark for Large Language Models.
The paper outlined a process that could be used to reliably mark LLM-generated text in a way that survived copying, pasting and even re-wording. It is possible that Anthropic and other labs may utilize this foundational technique, so it will be the focus of my discussion.
Watermark Definition and Application
If you recall, Anthropic said: Claude “weaves an imperceptible watermark directly into the text itself.” Here’s how the weaving might happen.
At its core, a watermark is a statistical signal. It provides information about whether words were produced by a human or a model.
In the last section, I discussed how the model:
Outputs a list of tokens along with a score (logits)
The softmax function is then used to assign a probability of whether a specific set of tokens will be selected
The watermarking happens at the pre-softmax function stage. After the tokens are generated, but before they are assigned a probability, the set of tokens outputted by the model are assigned a color: red or green. The logits of tokens in the green list are modified (an additional number is produced), which is added to the token score.
For example:
Green - Jumps: 8.2 + modifier: .02
Red - Lazy: 5.2 (no modifier)
Green - Over: 1.8 + modifier: .02
Green - a: 1.1 (+ modifier: .02)
Green - Dog: 1.05 + modifier: .02
Red - Brown: 0.4 (no modifier)
What happens is that this slightly increases the probability of certain tokens being selected. This process works best if there are a large number of potential relevant next tokens rather than a smaller list. If the answer of what token should be selected for a sentence is obvious, it’s more difficult to determine whether a model produced the output.
Detecting the Watermark
The key to the detection method is answering another question: “Given an LLM or human, how likely is it that the human would have picked words from the green list in their content?”
If the list of tokens that could be selected is small, meaning the answer is obvious, detection is unreliable. If the list of potential tokens is very large, detection is more reliable.
If a text has a large number of tokens from the green list, it is less likely that a human develpoed it.
Importantly, the sensitivity of detection does not rely on producing a very long text:
The list of green tokens has to be sufficiently large (the list can be large even with a short text)
The percentage of green tokens in the text is higher than random chance would allow
The specific method used to determine what tokens are on the red versus green list must be kept secret. Otherwise it would be easy for others to avoid detection.
Defeating the Text Watermark
Now that you know the watermarking process, you can understand why Anthropic said: “the watermark is part of the text, it will travel with the text when it’s copied and pasted elsewhere, and may persist through some editing.” Specifically:
The watermark is part of the text: Anthropic knows what tokens are on the green lists. Others do not, so it’s hard to determine what text to omit to avoid the watermark
The watermark will travel with the text: When the text is copied and pasted the fact that it is on the list of green tokens does not change
May persist through some editing: Without knowledge of the green list, one can’t target the tokens and delete them easily
Despite the difficulty of defeating the watermark, it is possible. Anthropic admits this in its announcement:
“Content generated by Claude may not carry a detectable mark if, for example:
The text has been heavily edited, paraphrased, translated, or mixed into other writing;
The passage is very short, leaving too little text for a reliable signal;”
The key to defeating the watermark comes from understanding that the strength of the method comes from the fact that each token (individual word unit) seeds the next red/green list.
For example, the phrase: “A quick brown fox” results in a list of red/green tokens associated with that set of tokens.
But what happens if the text is changed from “A quick brown fox” to “A rapidly moving fox”? or “A quick red fox”? The first example is a major edit. It’s unlikely that watermarking would survive that edit. The second is a minor edit. The list of tokens to select may still overlap with the red/green token list. In that case, the watermark may survive.
If you change enough tokens, the signal disappears. Minor edits may not make much of a difference.
There’s also the challenge associated with a large body of AI-generated text. Changing a few sentences in a 1000 word essay, largely generated with AI, won’t make much of a difference to the detection algorithm. Completely re-writing or modifying the text would work. This is the strategy some people in the open source community are using to attack AI text watermarking.
Methods for Stripping Text Watermarks
A new popular open source tool tool as been developed to defeat AI text watermarking. Here’s what the model does.
Text Cleaning, Backtranslation and Re-writing
First, a script can be run on the text to remove invisible markers in the text inserted by model providers. This process deletes these types of hidden marks, but has no impact on the text watermarking I’ve discussed in this essay.
The second stage removal process can involve:
Rewording the text: The watermark can’t be removed by re-organizing the text. The text has to be completely re-written line by line.
Backtranslation: This involves translating the text to another language and translating it back, which breaks the watermark.
Working to maintain text quality: Re-writing the text completely can have a negative impact on quality. Correcting this requires using another high-capability model to re-write the text, ensuring quality is retained.
This process is complex and can be expensive depending on the model. Also, with many model providers signing on to EU AI Act, using an OpenAI model to re-write content produced by Claude will just result in a mix of marks in the content.
Human Content Creation from an Outline and Editing
Another way to defeat watermarking is to:
Have a model generate an outline and use the outline to develop the text: No signal
Re-write text the model produces line by line: Reduced signal
Measuring the Unexpected Benefits of Watermarking: The AI Text Watermark Density Metric (AITDM)
A common concern people have is that the mere presence of an AI text watermark in their copy will brand it as AI-generated. For the percentage of people who are anti-AI, that might be the case. However, for everyone else, there might be an unexpected benefit: Being able to reliably determine how much of the copy was generated with AI assistance. To illustrate this I have created a metric called: The AI Text Density Metric or AITDM
Imagine documents A and B:
Document A: Alice has an AI generate a financial analysis using a company dataset, assumptions outlined in a presentation and Slack messages. Her content is 100% AI generated, but she has restructured it. A watermark analysis is run on the content and finds that 5% of the content is human-generated and 95% is AI-created. It gets a AITDM of 95.
Document B: Bob has created a different financial analysis using a similar data set. Bob has the AI generate an outline, but develops the analysis himself. He then asks the AI to reviw the content, but only correct grammatical errors and spelling mistakes. After analysis, it receives an AITDM of 15.
Clearly, Document A and B are in different classes. This is because AITDM allows us to measure the percentage of content developed by an LLM versus human.
AITDM analysis has a few additioal implications:
Content with a high AITDM should warrant additional scrutiny. Did the LLM hallucinate data? Were the assumptions correct, etc?
Content with the low AITDM should also be scrutinized because the author used an AI-generated outline to develop the content. However, it is less likely that the content suffers from hallucinations than fully AI generated material.
A Low AITDM Does Not Signal Quality
Importantly, a low AITDM does not mean the content is high-quality. Humans are prone to mistakes, misinterpretations and bad writing.
A high AITDM does not mean the user has out-sourced their strategic insights to a model. Instead, they may have carefully developed a prompt, checked the sources, corrected the model during development and taken other steps to ensure high-quality output. Judgement and expertise play a role here. In this case, the additional scrutiny is about accounting for LLM limitations rather than the overall quality of the output.
Where We Go From Here
Over the last few years the coding world has been embroiled in an argument about whether AI-assisted code is useful, relevant and high-quality. There are still many who don’t trust AI-developed code, but they understand that AI code generation is widespread. It’s becoming accepted in some corners that just because an AI developed code, does not mean that the producer is not responsible for the outputs, or that it is automatically low quality.
The writing world is now engaged in a similar debate. What makes AI-assisted writing different from coding is that it is easier to evaluate the outputs of AI-generated code. Scripts can be well-structured, secure, bug free and well-architected. There are software design principles that can be applied to code and testing can be done for correctness.
Doing this is harder with writing. People generally can’t tell the difference between well-written AI content and content produced by a human. A strategy is harder to evaluate for ‘correctness.’ It depends on a lot of factors.
Over time, I expect the debate about whether to use AI for writing will cool. Mainly because it will be so widespread. And, the presence of AI watermarking may prompt people to change their writing habits to get low AI text density scores, if only to demonstrate that they maintained significant control over the content development process, even if AI was used to aid their thinking. We’ll see.
References
This newsletter is part of the Doing AI Efficiently Operating System, built on five operational layers: Grasp, Discern, Ward, Execute, and Honor. This essay is part of the Grasp layer, which is focused on helping you understand how AI works from first principles, including tokens, context windows, model architecture, how LLM content generation happens, and more.