How AI Detectors Work — and How to Read the Score Without Getting Burned

Paste a paragraph into an AI detector and a number comes back: 78% likely AI. It looks precise. It is not. The tool has not checked who wrote the text, when, or with what. It has measured one thing — how much the writing resembles the average output of a language model — and turned that into a percentage.

That gap between what a detector measures and what people think it measures is where the damage happens: a student marked down for an essay they wrote alone, a freelancer's article rejected, a job applicant's cover letter binned. This guide explains what is actually being scored, why detectors fail in predictable ways, and how to use one without fooling yourself or wronging someone else.

What a detector actually measures

Language models write by predicting the most likely next word. Text that a model produces therefore tends to be the text a model would have predicted — smooth, even, unsurprising. Detectors exploit this. Most of them look at two signals:

• Perplexity — how surprising each word is, given the words before it. Low perplexity means every word was the obvious next word. Human writing is usually more surprising, because people have odd word choices, private references and opinions.
• Burstiness — how much sentence length and rhythm vary. People write a long sentence, then a short one. Then a fragment. Models tend to keep an even 15–25 words, sentence after sentence.

Newer detectors add a trained classifier on top: a model shown millions of human and machine samples that learns other tells — stock transitions, tidy summaries, a fondness for words like delve and crucial. But every one of those signals is a property of the text, not of its author. That distinction is the whole story.

A worked example

Here are two sentences that say the same thing. Read them and guess which a detector flags.

Version A scores as machine-like on almost every detector: every word is the expected word, the sentence is a textbook length, and it contains no information that only its author could know. Version B scores as human: a specific month, a doctor, a small joke. Now the uncomfortable part — Version A might have been written by a nurse with twenty years' experience, and Version B can be produced by any model if you ask it to "add a personal anecdote". The detector cannot tell. It can only tell you which one reads more like a model.

The four ways detectors get it wrong

1. False positives on careful, plain writing

Clear, well-edited prose is low-perplexity by design. Technical documentation, legal text, lab reports and anything written to a template score "AI" at high rates because the genre demands predictable language. The better you follow a style guide, the more suspicious you look.

2. Bias against non-native English writers

Writers working in a second language tend to use a smaller vocabulary and safer sentence structures — exactly the profile detectors associate with machines. A widely cited Stanford study in 2023 ran essays by non-native English speakers through seven popular detectors: more than half were labelled AI-generated, while essays by native speakers were almost all cleared. Nothing about the tools has fixed this since, because the bias is baked into what they measure.

3. False negatives after light editing

The mirror image: a generated draft with a few sentences reworded, a personal detail dropped in and the rhythm broken up will often pass. So will output from a model prompted to "write casually" or to imitate a sample. Detectors catch lazy AI use. They rarely catch deliberate AI use.

4. Short texts and mixed texts

Below roughly 150 words there is not enough signal to say anything, and most honest tools will tell you so. Documents that mix a human introduction with a model-drafted middle confuse a single overall score entirely — which is why sentence-level highlighting matters more than the headline number.

How to read a detector result

A good detector gives you more than one number. Read them in this order:

If a tool gives you only a percentage and no range, no confidence and no explanation, it is asking you to trust it blindly. Don't.

Using a detector fairly

What not to do

The honest summary

AI detectors are useful the way a smoke detector is useful: they tell you where to look, and they go off when someone burns toast. As a screening tool with a human decision behind it, they help. As a verdict, they harm real people — disproportionately the careful and the non-native. Read the range, read the highlighted sentences, then use your own judgement.

Frequently asked questions

How accurate are AI detectors?

Vendors quote accuracy of 95–99%, measured on their own test sets of clearly human and clearly machine text. On real-world writing — edited drafts, non-native English, technical prose, short passages — independent tests find far higher error rates in both directions. Treat any score as a probability estimate with a wide margin, not a measurement.

Can a detector prove I used AI?

No. It has no access to how the text was produced; it only scores the text's predictability. It cannot distinguish a human who writes plainly from a model, or a model prompted to write with personality from a human. Proof of authorship comes from process — drafts, notes, version history — not from a percentage.

Why does my own writing get flagged as AI?

Usually because it is clear and well organised: consistent sentence length, standard vocabulary, no digressions. Writers working in a second language and writers following a strict style guide are flagged most often. It is a limitation of the method, not a judgement of you.

Does Google penalise AI-detected content?

Google has said it rewards helpful content regardless of how it was produced and does not use AI detectors to rank pages. What gets penalised is thin, generic, unhelpful content — which AI produces easily when left unedited. Edit for the reader and the detector question takes care of itself.

What is the difference between an AI detector and a plagiarism checker?

A plagiarism checker compares your text against existing sources and shows exact matches — it deals in facts. An AI detector estimates the statistical style of your text — it deals in probabilities. One tells you where a sentence came from; the other guesses what kind of thing might have produced it.