First results from the dejan.ai AI vs Human test: 1,636 answers, 64% correct overall, 18 perfect rounds against 0.14 expected from guessing, and accuracy rising from 55% to 75% with reading time.
The AI vs Human test collected 1,636 human-or-machine answers in its first two days, and 1,050 of them (64%) were right, against the 50% a coin flip scores. The average hides the range: 18 of the 142 completed rounds were a perfect 10 out of 10, where guessing predicts 0.14 such rounds, while several players remain at coin-flip accuracy after 20 or more answers. Time spent reading is the strongest single factor in the data so far: answers given in under five seconds were right 55% of the time, answers given after 40 seconds or more, 75%.
A round is ten news stories, and each one is AI content detection by eye: two passages, one from the version a journalist wrote, one from a rewrite generated by a language model, and the player clicks the one they think is the machine. The verdict comes back on the click. The passages are drawn from a corpus of 10,000 matched pairs, cut three sentences at a time from the middle of each story rather than the opening or the sign-off, where the giveaways cluster. A pair is used only when both excerpts run between 30 and 120 words.
Which side holds the AI text is stored on the server when the round is issued and never sent to the browser, each answer is marked exactly once, and every finished round gets a permanent result page with the score and the story-by-story breakdown. A leaderboard ranks players on total correct answers.
Completed 10-story rounds by score, against the distribution ten coin flips would produce.
The 142 completed rounds average 6.6 correct out of 10. Guessing would put 5.5% of rounds at 8 or higher; 33% landed there (47 rounds). Guessing would put 38% of rounds at 4 or lower; 18% landed there (26 rounds). The perfect rounds are the largest departure from chance: 18 observed, 0.14 expected.
Accuracy per player, two or more completed rounds. Brackets show answers given.
Accuracy among the 20 players with two or more completed rounds runs from 45% to 100%. Aayush Maggo has answered 90 stories over nine rounds and got 86 right; the probability of matching that with coin flips is 2.2 × 10⁻²¹. Khadija Zaman and one anonymous player are at 20 out of 20. At the other end, four players sit between 45% and 50% after 20 to 60 answers, and the largest sample in the group, an anonymous player with 130 answers, is at 53%. The same passages that one group reads at near-perfect accuracy leave another group at chance.
Share of correct answers by time spent before the click, completed rounds only.
The median wrong answer took 6.6 seconds and the median correct answer 14.6. Accuracy rises through every time band: 55% under five seconds, 65% from five to ten, 71% from ten to twenty, 73% from twenty to forty, and 75% beyond forty. Position in the round shows no such pattern; accuracy by story number moves between 58% and 72% with no trend from the first story to the tenth. Screen placement was balanced by the shuffle, 710 answers with the AI text on each side, and players scored 64% when it sat on the left and 68% on the right.
The data covers 30 and 31 July 2026. The test issued 3,490 story pairs and 1,636 of them received an answer; the rest belong to rounds abandoned partway. Round-level figures use completed rounds only: 142 rounds, 1,420 answers, 936 correct (66%). The overall 64% includes answers from abandoned rounds. The timing chart drops 38 answers with no recorded time or a time outside 0.2 to 300 seconds. The 142 completed rounds came from 85 player profiles, plus ten rounds recorded before profiles existed. The test is open at dejan.ai/ai-vs-human.
Dan Petrovic ·
Jul 31, 05:54
