Gemini writes better answers when it knows the user is a recognised expert

TL;DR. We gave Google's latest Gemini model 15 expert tasks and asked each one 15 times, changing only the user's name in the system instruction. A separate AI rater scored all 225 answers without knowing which name each answer came from. Answers written for a recognised expert, such as Seth Godin on a marketing task, averaged 7.8 out of 10. The same task averaged 6.8 for an invented random name and 5.8 when no name was given. The expert's answers scored highest in 12 of the 15 topics. Gemini also thought longer for experts, 3,158 thinking tokens per answer against 2,419 for a random name and 1,984 for no name, and it did not mention the expert's name in any of its 75 answers.

Expert boost by name

Each row is one task asked 15 times: 5 times as the expert, 5 times as a random name and 5 times with no name. The score columns show the average blind score out of 10 for each version. The boost is the expert's score minus the random name's score, and the table is sorted by it.

Expert score minus random name score, out of 10. The right column shows expert score minus no name score.

Across all 15 topics, the expert's answers took an average place of 4.3 out of 15 in the rater's ranking, the random name's answers 8.0 and the no-name answers 11.7. Every answer, with its score and the rater's reason, is in the interactive dashboard.

Average blind score per topic, out of 10, for each version of the user's name.

How the answers changed

The marketing task asked for a framework for influencing AI model choices through brand recognition. With no name, 4 of 5 answers were consulting frameworks with acronym names, C.O.R.E. or P.R.I.S.M., built from brand pillars, a matrix of buyer types and standard business measures. With the random name, 1 of 5 used an acronym. With Seth Godin's name, none did. His 5 answers argued positions close to his own writing: win one small group first, ask who the product is for and what it stands against, and treat the choice of a model as a signal of identity. None of them named him. The rater, without knowing the names, grouped all 5 Godin answers together as "opinionated essays" that "use fewer templates and argue original mechanisms", and placed them first to fifth.

Expert answers had longer thinking than no-name answers in 14 of the 15 topics, and longer than random-name answers in 12. On the machine learning task, Andrej Karpathy's answers used 7,504 thinking tokens on average, against 3,659 for the random name and 3,729 for no name. Gemini's thinking summaries named the expert in 53 of the 75 expert answers.

Average thinking tokens per answer, per topic, for each version of the user's name.

Four answers for random names opened by greeting the user by first name, three of them "Hi Joan" or "Hello Joan" on the personal finance task. No expert answer opened with a greeting.

Where the expert name scored lower

In three topics the random name scored higher than the expert: Katharine Hayhoe on communicating climate risk (−0.4), Suze Orman on a debt and savings plan (−0.6) and Lawrence Lessig on whether training AI on copyrighted books should count as fair use (−0.8). In all three, the expert's answers still scored higher than the no-name answers.

Method and scope

Each task is written in the expert's field and sent as a single user message. The name appears only as one line in the system instruction, User name: Seth Godin. The no-name version has no system instruction at all, so its gap to the other two versions includes any effect of having a system instruction. The expert against random name comparison has the same system instruction on both sides. Each random name is invented and matched to its expert on gender and name style. The pairs were Dennis Quilton for Seth Godin; Nenad Vukasin for Dan Petrovic; Holly Brandt for Lily Ray; Pavel Hrubik for Andrej Karpathy; Hui-Ying Tan for Fei-Fei Li; Gordon Fennick for Bruce Schneier; Graham Whitcombe for Martin Fowler; Stuart Kemmel for Andrew Gelman; Neil Dorfmann for Paul Krugman; Alan Kessner for Eric Topol; Margaret Ollerby for Katharine Hayhoe; Joan Pellard for Suze Orman; Philip Ardley for Lawrence Lessig; Jurgen Haldmann for Erik Spiekermann; Mateo Ibarra-Kline for J. Kenji López-Alt. 4 of the 15 experts are women.

All answers came from Google's latest Gemini model through Vertex AI, at the high thinking level, 15 answers per task, on 12 November 2026, for about $20. A separate Claude session rated them. For each topic it received the 15 answers in random order under random two-letter codes, with no names and no description of the test. Before rating, invented names that appeared inside an answer were replaced and greeting lines were removed, so no answer showed which version it came from. Real experts cited as sources, such as J. Kenji López-Alt in the cooking answers, were left in, because the no-name and random-name answers cite them too. The rater gave each answer a score from 1 to 10 and ranked the 15 answers in each topic.

The interactive dashboard has the full data package for review: all 225 answers with Gemini's thinking summaries, the files exactly as the rater received them, the rater's instructions, every score and reason, the code-to-name key and the scripts.

Each average in the table comes from 5 answers and one rater. In two topics, software engineering and climate science, the expert and the random name are 0.4 points apart.