Run a document through an AI-writing detector and you get a number — say, “74% likely AI-generated.” The number looks precise, which is exactly why it gets misused. Behind it sits a genuinely interesting piece of statistics, a set of well-documented failure modes, and a hard truth every integrity office should internalize: a detector measures how much a text resembles what language models tend to produce. It does not, and cannot, observe who typed it.
This article walks through the mechanics — perplexity, burstiness, classifier ensembles — then the places where those mechanics break, and finally what a defensible review workflow looks like once you accept that the output is a probability, not a verdict.
The statistical fingerprints: perplexity and burstiness
Perplexity: how “surprising” is each next word?
A language model is, at its core, a machine for predicting the next word. Given “The mitochondria is the powerhouse of the…”, it assigns a high probability to cell and a low one to harpsichord. Perplexity turns this around: score an existing text with a model and measure how surprised the model is, word by word. Text that a model itself generated tends to score low — the generator preferred high-probability continuations, so a similar model finds every word unsurprising. Human writing is messier. We reach for odd verbs, bury asides mid-sentence, make choices a model rates unlikely. That statistical roughness is the single strongest signal detectors have.
Burstiness: how much does that surprise vary?
The second fingerprint is the variance of perplexity across a document. Human prose is bursty: a long, winding sentence, then a short one. A dense technical claim, then a plain-language aside. Model output is characteristically uniform — sentence after sentence of similar length, rhythm and predictability. Low average perplexity combined with low burstiness is the classic machine-text profile; either signal alone is much weaker evidence.
From fingerprints to a score: classifier ensembles
No serious detector ships a raw perplexity threshold. Production systems combine families of evidence, because each family fails in different places:
- Zero-shot statistical tests compare how differently scoring models react to the text — no training on labeled examples, which makes them robust to new generators but noisy on short input.
- Trained classifiers — typically fine-tuned transformers — learn the boundary between human and machine text from large labeled corpora. Sharper where the training data is representative, weaker on genres and generators they never saw.
- Stylometric features — sentence-length distributions, vocabulary diversity, punctuation habits, discourse structure — add a signal that survives light paraphrasing better than raw token probabilities do.
An ensemble votes these signals into one calibrated estimate. The engineering goal isn't a magic feature; it's making sure no single family's blind spot decides the outcome — and knowing when the honest answer is “not enough signal to say.”
Why the output is a probability, not a verdict
Here is the part marketing pages tend to skip. The distribution of human writing and the distribution of machine writing overlap. Plenty of human prose is smooth and predictable; plenty of edited model output is rough. Ordinary generated text carries no hidden watermark a detector can decode — so detection is statistical inference over overlapping populations, and every score is a likelihood, not an observation. Any threshold you draw trades false positives against false negatives; there is no line that eliminates both.
Where detectors fail — and why you should know the list
Every failure mode below follows directly from the mechanics above. A reviewer who knows them reads reports better than one who trusts the number:
- Short texts. Perplexity statistics need tokens to stabilize. A two-paragraph answer simply doesn't contain enough signal for a confident call in either direction.
- Lists, tables and references. Bullet points, headings and bibliographies are structurally predictable for everyone. Scoring them as prose inflates machine-likeness.
- Formulaic academic prose. Methods sections, lab reports and legal boilerplate are written to genre conventions that compress exactly the variance detectors rely on. Disciplined human writing can look statistically machine-like.
- Non-native and highly-trained writers. Writers working in a second language often lean on conventional, learned phrasing — smoother, more predictable text. This is a known elevated false-positive risk for perplexity-based methods, and it is the strongest single argument against automated penalties.
- Paraphrased or “humanized” AI output. Light human editing, synonym passes and paraphrasing tools raise perplexity and can drag generated text back under thresholds. False negatives are real, which is another reason a low score isn't an exoneration any more than a high score is a conviction.
Why sentence-level beats document-level
The most common real case isn't “fully human” or “fully generated.” It's mixed authorship: a student drafts the analysis, generates the literature review, and polishes the conclusion with a model. A single document-level percentage averages all of that into mush. Sentence- and segment-level scoring keeps the signal where it occurred:
| Document-level score | Sentence-level scores |
|---|---|
| One number for the whole file | A probability per sentence or segment |
| Mixed authorship averages into an ambiguous mid-range | Machine-like passages stay visible even in mostly-human work |
| Reviewer argues about a percentage | Reviewer reads specific flagged passages in context |
| Hard to discuss with a student | Grounds a concrete conversation: “walk me through this section” |
This is how iOriginally's AI-writing analysis is built: probabilities are reported per sentence and segment, kept strictly separate from the similarity score (they measure different things and should never be summed), and every AI screen and PDF carries a printed policy note stating that the analysis supports human judgment rather than replacing it.
What a responsible workflow looks like
Institutions that handle AI-writing signals well tend to converge on the same sequence:
- Treat the score as triage, not testimony. A flagged submission earns attention, nothing more. Set policy bands in advance so “attention” means the same thing across departments.
- Have a human read the flagged segments in context. Do the passages fit the failure modes above? Is the writing consistent with the student's previous work, drafts and version history? An experienced marker's read is part of the evidence.
- Talk to the student before concluding anything. Ask about process: notes, drafts, sources, how a key passage came to be. A writer who did the work can usually explain it; the conversation is where cases actually resolve.
- Never automate the penalty. No threshold should trigger a sanction by itself. Given the documented false-positive risks — especially for non-native writers — an auto-punishment pipeline is indefensible on both fairness and procedural grounds.
None of this makes detection useless. It makes it what it honestly is: a screening instrument that directs scarce human attention to where the statistical evidence is strongest. Used that way — signal, then review, then conversation — it protects students from false accusation exactly as much as it protects assessment from unearned credit. Used as an oracle, it fails both.
