Text watermark lab
How text
watermarks work
A text watermark steers a model’s choices so a detector can recognize a pattern. Explore the sampling rule with four words, then test how length, edits, and keys affect the signal.
Try the four-word experiment ↓An offline lab with two teaching models and a recorded model trace.
01 / ONE DECISION · EXACT TOY MODEL
A watermark in four words
A model assigns odds to its next token, a small piece of text. We use four whole words here. Their probabilities are set by hand.
The trail was …
Draw two candidates independently from the base odds. Keep the one with score 1. If the scores tie, keep the first.
In tournament mode, a score change keeps the same candidate pair. The key and recent tokens determine the scores in a real system.
Compare the probabilities
Map 7 of 16Each word has a score of 0 or 1. Flip a score to change this map; the last column averages all 16 equally likely maps.
| Word / score | Base odds | This map | All 16 maps |
|---|
WHAT THE DETECTOR CAN COUNT
How often does the chosen word score 1?
Averaged over all 16 maps, the word probabilities can stay the same while score-1 choices become more common. A detector with the maps can count those choices.
Show the calculation and its assumptions
For word i, let pᵢ be its base probability and gᵢ its score, either 0 or 1. Let G = Σ pᵢgᵢ.
One tournament layer: qᵢ = pᵢ(1 + gᵢ − G). Each score bit is an independent fair bit. Averaging all maps gives E[gᵢ] = E[G] = ½, so E[qᵢ] = pᵢ.
The winner’s expected score is ¾ − ¼Σpᵢ². It exceeds ½ when at least two words are possible. The green-list comparison instead uses qᵢ ∝ pᵢ exp(1.4gᵢ).
This is one decision with idealized score bits. A fixed map generally changes its odds. Reusing a context can repeat a map, so repeated text does not supply fresh independent evidence.
The tournament is a small model of a public SynthID idea. A full implementation adds multiple layers and context handling. The rest of this page uses a separate green-list model.
SynthID implementation notes · Research paper · This lab’s code
02 / COUNT THE SIGNAL · GREEN-LIST TOY
How much text is enough?
The next experiments use a different rule: a key and the previous token label some candidates as favored, or “green.” The sampler increases their chance. Both groups below come from the same synthetic model; change the length to compare their scores.
ONE WATERMARKED RUN
Count the extra matches
The score uses 50% as its reference rate. The simulations below show how much the count varies without a watermark.
The first token is skipped. Each distinct adjacent token pair counts once. This reduces evidence from repetition; it does not make all remaining observations independent.
A score is not the probability that a person used AI. The cutoff at z = 3 is an example for this toy setup. Zero false positives in a finite run would not establish a zero false-positive rate.
03 / EDIT THE TEXT · GREEN-LIST TOY
How much of the signal survives?
Replace a token. The score rule checks the token before it, so the next transition can change too. Insert or delete a token to test where the rule lines up again.
Click a token to replace it. Outlines mark edits. These controls make mechanical changes to synthetic text. They do not simulate a fluent rewrite. The score can rise after an edit.
04 / CHANGE ONLY THE TRIAL KEY
Try another detector key
This check uses the original, unedited passage. The generation key stays fixed. Change the detector key and compare the scores.
OPTIONAL · A REAL MODEL TRACEInspect the stored word choices
RECORDED OUTPUT · GREEN-LIST SAMPLER
Which sample used the boost?
These passages came from SmolLM2-135M-Instruct. Two used ordinary sampling. One used the green-list boost. The stored samples show token choices; three passages cannot establish a quality difference.
Reveal the sample, then inspect a token choice.
Trace the revealed passage →RECORDED OUTPUT · ONE DECISION
Inspect one recorded decision
Choose a position in the recorded passage. The table shows the candidate probabilities at that position. The toggle changes the table; the recorded passage stays fixed.
The highlight is the token the model actually chose. The table below shows other plausible choices at that exact moment.
Recorded pipeline: divide logits by temperature, add the boost, apply top-k and top-p, then sample. Only the stored candidate subset appears here.
No token is permanently green.
The class depends on the key and recent context.
Generation sees logits.
The bias is applied before the token is sampled.
The detector uses token IDs and the key.
A real text detector also needs the matching tokenizer.
RECORDED OUTPUT · SCORE
Count the favored tokens
The same rule labels the observed tokens. Count the favored transitions, then compare that count with the 50% reference used by this toy score.
OPTIONAL · REPRODUCE THE COUNTSChange the synthetic model
GREEN-LIST MODEL · PROBABILITY SHIFT
Concentrate the next-token probabilities
Increase sharpness to concentrate the probability on fewer tokens. See how the boosted distribution changes.
KL measures a change in this probability distribution. It does not measure writing quality. The bars show the most likely tokens in the current context.
SOURCE CHECK · 16 OCT 2026
Claude’s text watermark
Anthropic names a version of SynthID-Text as its method. This lab demonstrates the sampling idea. Read the explanation ↗
- Text detection
- Anthropic describes its detector API as a private preview. Access and current model coverage ↗
- File checking
- The public Content Checker checks file credentials. It does not detect text watermarks.
- Identity
- Anthropic says its watermark carries no user, organization, or chat identity. Source ↗
- This lab
- The toy keys are public. The examples cannot test Claude output. A low score here cannot establish human authorship.
Use the live model-coverage table for rollout status. A provider-specific signal can indicate involvement in producing or editing text; it does not settle who supplied the ideas.