AI text detectors can miss machine-generated writing that has been altered to look human, and they can also struggle as writing habits and language models change. Researchers at MIT and Cornell University tested a way to make a detector adjust to those changing conditions after deployment.
The technique, called test-time adaptation (TTA), updates a deployed model using new examples without requiring a complete retraining. For a text detector, those examples are user-submitted passages whose human or AI origins are not supplied to the model. Given enough examples, the detector can learn patterns that help distinguish the two.
In a test involving writing deliberately altered to evade detection, the adaptive model caught more than 93 percent of disguised text, compared with about 27 percent for existing methods. In another experiment simulating changes in human language over time, the adaptive detector achieved 85 percent accuracy; traditional methods reached 73 percent.
Changing writing and deliberate evasion
Detectors are generally trained on internet text whose origins are known. Their performance can suffer when later inputs differ substantially from those examples, a problem called distribution shift. Changes in how people write or use AI, as well as the arrival of new language models, can create such differences.
For example, a detector trained on writing from 2010 might encounter unfamiliar topics and analogies in human writing from 2030 and mistakenly label it AI-generated. Deliberate evasion presents another challenge: automated humanizer tools can alter AI output so that a detector sees it as human-written.
Lead author Kevin Ren described a recurring contest in which developers add humanized text to their training material and users then seek ways around the updated detector. He said TTA could help deployed models respond proactively to repeated attempts to game them.
How the researchers tested adaptation
The team assembled datasets with specified amounts of human- and AI-generated writing but withheld those labels from the model. The researchers also used large language models to produce examples intended to evade a detector. After adaptation, they evaluated the model against data with known origins.
Ren said shifts in human writing are harder for TTA to address because they occur more slowly and subtly. The researchers also cautioned that the method may not always work if future language models produce more natural-sounding text and verifiable human writing becomes harder to obtain for training.
Study co-author Manish Raghavan said the work showed that TTA can make detectors more robust despite continuing adversarial behavior. He also said it remains unclear whether distinguishing human from AI content will be a solvable problem, raising open questions for science, writing and art.
Raghavan is the Drew Houston (2005) Career Development Professor at the MIT Sloan School of Management, holds a shared position in MIT’s Department of Electrical Engineering and Computer Science through the Schwarzman College of Computing, and is a principal investigator in the Laboratory for Information and Decision Systems. Ren is a Cornell Tech graduate student; senior author Nikhil Garg is an assistant professor there.
The research will be presented at the Conference on Neural Information Processing Systems. The team plans to investigate other strategic settings for TTA and develop better benchmarks for improving detectors as conditions change. Pangram provided API credits for the project’s use of its AI text detectors.




Discussion