Read the articleDoes Gemini give better answers when it knows who it is talking to?
15 topics. Each task was asked 15 times on Google's latest Gemini model through Vertex AI, with high thinking: 5 times with no name, 5 times with an invented random name, and 5 times with a real expert's name. The name was only passed as "User name: …" in the system instruction. A separate Claude session scored all 225 answers from 1 to 10 without knowing which name each came from.
Expert boost
How much higher each expert's answers scored than the random name's answers on the same task (average blind score, out of 10). Sorted by boost. The last column shows the gap against no name.
Blind quality score by topic
Thinking length by topic (average tokens)
Table view
Download the data
- expert-users-data.zip: everything below plus the rater files, rater instructions, SQLite database and scripts
- answers.csv: all 225 answers with thinking summaries, blind codes, scores and the rater's reasons
- tasks.csv: the 15 tasks, experts and random names
- README.md: method, file list and how to reproduce