How we test HumanizerGPT against AI detectors

Status: this is the method for our first public benchmark run, which is being prepared. Until it is published, the figures on the benchmark page are sample data.

1. The sample set

Each run uses a fixed set of 500 texts written by several current AI models, including models from OpenAI, Anthropic, Google and DeepSeek. The set is split evenly across five content types, because detectors behave differently on each:

• Academic essays and essay sections
• Blog posts and SEO articles
• Emails and business messages
• Marketing copy and product descriptions
• Cover letters and personal statements

Samples range from roughly 150 to 600 words, since very short texts are hard for any detector to judge. The prompts and the raw drafts are frozen, so every month tests the same inputs and changes in results come from the tool or the detectors, not from new samples. When we refresh the set, we say so and keep the old results available.

2. Humanizing

Every sample is run once through HumanizerGPT in Balanced mode with default settings, with no manual editing and no cherry-picking of the best of several attempts. We record the model version used, so results can be tied to a specific release.

3. Detectors and how we access them

Turnitin has no public API. Its AI writing indicator is only available to institutions, so Turnitin results come from manual checks on a smaller, randomly chosen subset of the sample set. We report the subset size next to every Turnitin number, and its margin of error is wider than for the other detectors.

We use each detector's default settings and its current public version on the day of the run. Where a detector returns a probability rather than a label, we use the detector's own threshold for "human" or "AI".

4. Metrics

Pass rate

A sample passes when the detector classifies it as human-written. The pass rate is the share of samples that pass, reported for the raw draft and for the humanized version.

Meaning similarity

We compare each rewrite with its draft using a semantic similarity score from 0 to 1, and spot-check a random subset by hand for dropped facts, added claims or changed numbers. A rewrite that loses meaning is flagged, whether or not it passes.

Readability

We report a standard reading-grade score for the draft and the rewrite, to show whether the text became easier or harder to read.

Processing time

The median time from submitting a sample to receiving the full rewrite.

5. Versioning and publishing

• Each run is labelled with its date, the HumanizerGPT model version and the detector versions where they are published.
• Results from earlier months stay available, so changes over time are visible.
• We publish every detector we test, including the ones where our results are weakest.
• If we find an error in a published run, we correct it and note the correction on the page.

6. Limitations

• Detectors change. Vendors update their models without notice; a result from one month may not hold the next.
• Your text is not our text. Topic, length, language and how much you edit afterwards all affect the outcome.
• English first. The sample set is in English. Results in other languages may differ.
• No guarantees. No tool can guarantee that a text will pass a detector, and this benchmark does not claim otherwise.
• Detectors make mistakes. A detector verdict is not proof of how a text was written, in either direction.

7. Requesting raw data

Researchers, educators and journalists can request per-sample results for published runs, including drafts, rewrites and detector scores. Email contact@nowaymobile.com with your name, organisation and how you plan to use the data.