Listen: From Free-Text to Likert Distributions: A Practical Guide to SSR for Purchase Intent
Semantic Similarity Rating (SSR) maps LLM free-text responses to Likert distributions to improve purchase intent realism and match human response patterns.
Transcript
When we ask large language models to rate things on a scale of one to five, they tend to fail. Instead of mimicking human diversity, their answers cluster in the middle or lean far too positive.
A better approach is to let the models speak like real people. This is called Semantic Similarity Rating. Instead of forcing a number, you ask the model, prompted with a specific demographic persona, to write a short, natural response about how likely they are to buy a product.
Next, you convert that text into an embedding and compare it to five simple anchor statements, ranging from "definitely not" to "definitely yes." By measuring the similarity to each anchor, you can map the text back to a highly realistic one-to-five scale.
In tests across dozens of concept surveys, this method reproduced human purchase intent with about ninety percent of the reliability of actual human test-retest results. It fixes the unrealistic clustering of direct rating, matches the true spread of human opinions, and gives you the qualitative explanation for free. For researchers, it means getting the best of both worlds: realistic statistics and the natural language behind them.