Invisible characters in AI text

Seeing them at all

The fastest check needs no tools. Copy a paragraph of AI output, then run it through a hex dump:

$ pbpaste | hexdump -C | grep -E 'c2 a0|e2 80 8b|ef bb bf'
00000040  61 6e 64 c2 a0 74 68 65  6e 20 74 68 65 20 6d 6f  |and..then the mo|

Those two bytes, c2 a0, are a non-breaking space wearing the costume of a normal one. In a text editor the line reads and then the mo with nothing out of place. To grep, to a CSV parser, and to a string equality check, that word boundary is a different character than the one you typed.

What tends to be in there

Not every invisible character is equally likely, and they are not equally harmful. These are the ones worth knowing by sight:

Where they actually come from

The common explanation is that the model inserts them as a watermark. That claim is durable, widely repeated, and almost certainly wrong.

It fails an easy test. Ask a model the same question twice and diff the invisible characters in both answers. A watermark has to survive light editing and has to encode something, so it would need a stable, recoverable structure. What you find instead is non-breaking spaces clustered around line wraps and punctuation, in positions that change between runs. That is not a signal. That is typography.

The real sources are duller, and knowing them tells you how to avoid the problem:

• The copy path, not the model. Chat interfaces render answers as HTML. When you select and copy, you are copying rendered markup, and the browser hands the clipboard whatever the layout used for spacing. That is where the bulk of U+00A0 comes from. The same text pulled from an API response is usually clean.
• Training data. Models learn punctuation spacing from text that included it. French typography puts a narrow no-break space before a colon or a question mark, so U+202F shows up in output that has nothing to do with French.
• Upstream documents. Soft hyphens and byte order marks come from PDFs and exported files. If you pasted a source document into the prompt, the model may reproduce what it was given.
• Emoji. Zero-width joiners and variation selectors are how multi-person and skin-tone emoji are assembled. They are supposed to be there.

What breaks

The failures share a shape: something compares two strings that look identical and concludes they are different.

• Spreadsheet imports. A byte order mark at the start of a CSV attaches itself to the first column header, so a lookup for id misses a column displayed as id.
• Deduplication. Two rows that differ only by a zero-width space are distinct keys. A unique constraint accepts both, and the duplicate is invisible in every report.
• Search. Exact-match search fails on a word that contains a zero-width character. So does find-and-replace, which is why manual cleanup tends to miss it.
• Code. A non-breaking space in indentation is a syntax error in Python and a silent style violation nearly everywhere else. The error message points at a line that looks fine.
• Form validation. A trailing non-breaking space in an email field passes a naive trim, which only strips U+0020, and then fails the regex.

The part that is a security problem

Two categories in that table are not typography. They are attack surface.

Directional overrides

Bidirectional control characters tell a renderer to display text in a different order than it is stored. Source code is stored as one sequence and displayed as another, which means a reviewer and a compiler can disagree about what a program does. This was published in 2021 as Trojan Source and assigned CVE-2021-42574; compilers and code hosting platforms now warn on these characters, but plain text pipelines generally do not.

Tag characters

The block from U+E0000 to U+E007F mirrors printable ASCII. It was deprecated for its original purpose, renders as nothing, and survives most copy operations intact. An arbitrary ASCII string can be encoded into it and hidden inside ordinary-looking text.

This matters specifically because of AI. If a model reads a document that carries a hidden instruction in tag characters, a human reviewing that document sees nothing unusual, while the model receives text it may treat as an instruction. That is a prompt injection channel that passes visual review. Any pipeline that feeds user-supplied text to a model should be stripping this range before the text ever reaches a prompt.

When stripping is the wrong move

Most advice on this topic stops at delete everything invisible. That advice will corrupt valid content.

• Emoji sequences. Remove the zero-width joiners from a family emoji and you get three separate people. Remove U+FE0F and some emoji revert to a monochrome text glyph.
• Right-to-left text. Arabic and Hebrew mixed with Latin script or numbers rely on directional marks to display correctly. Stripping them mangles the text for the people who can actually read it.
• Indic and Persian script. Zero-width non-joiners are orthographically required in Persian and several Indic scripts. They are not decoration.

A defensible default is to delete the characters that carry no meaning in your content, normalize the fixed-width spaces to a plain U+0020, and leave the rest alone. That is the distinction between delete and normalize in the table above, and it is why a blanket regex is not the right instrument.

Practical detection

Ordered by how quickly they answer the question:

• Hex dump. hexdump -C or xxd. Definitive, because it shows bytes rather than glyphs.
• Editor settings. VS Code flags invisible and ambiguous characters in source files by default and can render whitespace explicitly.
• A Unicode-aware regex. In JavaScript, /[\u200B-\u200F\u2060\uFEFF\u00AD]/gu catches the common zero-width set, though a complete rule also needs the fixed-width space range \u2000-\u200A and the tag block \u{E0000}-\u{E007F}.
• A visual scan. The invisible character tool on this site shows each code point in place with a count, so you can see what is there before deciding what to remove. It runs in the browser, which matters if the text is something you would not paste into a server you do not control.

The short version

The invisible characters in AI output are mostly non-breaking spaces picked up from the browser during copy, not a watermark. They are harmless to read and reliably destructive to anything that compares strings. A small subset, the directional overrides and the tag character block, is a genuine security concern and deserves stripping on principle when text crosses a trust boundary. And a different small subset is load-bearing, so the correct operation is rarely delete everything.

If the text came from a chat interface and is headed somewhere strict, checking it takes a few seconds and saves a debugging session that starts with the words but it looks fine.