Sign in
← back
From Free-Text to Likert Distributions: A Practical Guide to SSR for Purchase Intent

From Free-Text to Likert Distributions: A Practical Guide to SSR for Purchase Intent

IdeaProcess

Semantic Similarity Rating (SSR) maps LLM free-text responses to Likert distributions to improve purchase intent realism and match human response patterns.

Instead of forcing LLMs to pick a number on a 1–5 scale, ask them to speak like a person and map the text to a Likert distribution via Semantic Similarity Rating (SSR). In benchmarks across 57 personal-care concept surveys (9.3k human responses), SSR reproduced human purchase intent signals with ~90% of human test–retest reliability and much closer response distributions than direct Likert elicitation. Plus, you get qualitative rationale for free.

Why you should care

  1. Directly asking an LLM for 1..5 tends to collapse into the middle (regression to mean) and over-positivity. Distributions end up unrealistic.
  2. SSR fixes this by: (1) eliciting a short free-text PI statement; (2) embedding it; (3) computing cosine similarity to five anchor statements (one per Likert level) and normalising into a probability mass function across 1–5.
  3. On 57 real surveys, SSR closed the realism gap: distribution similarity (KS) jumped to ≈0.8–0.9, while correlation of mean PI vs. humans improved or matched FLR.

A bar chart displays an illustrative Likert rating probability distribution generated from Semantic Similarity Rating, peaking at rating four.

Method in one picture

  1. Prompt the model to impersonate a respondent with demographics; show the concept; ask “How likely are you to buy?” but allow short free-text.
  2. Embed the text and compare to five anchors (Likert 1–5).
  3. Normalise cosine similarities to a pmf over 1..5, then compute the expected rating.

What the paper found (and what to expect)

  1. DLR (Direct Likert Rating): OK correlation (~0.8 ρ attainment) but poor distribution match (KS ≈ 0.26–0.45 depending on model/temperature).
  2. FLR (Follow-up Likert after text): higher KS than DLR; solid correlation; needs a good “Likert-expert” rater prompt.
  3. SSR (this article): best distributional realism (KS ≥ ~0.8) and mean-PI correlation on par with (or better than) FLR.

Illustrative histograms below show the direction of change (not the study data):

An orange histogram showing a relatively low response-distribution KS similarity to humans for Direct Likert Rating, peaking around 0.4.

An illustrative histogram plots KS similarity to humans for the FLR method, showing a peak count around 0.6 to 0.65.

Histogram showing an illustrative distribution of SSR response-distribution KS similarity to human ratings, peaking around 0.83.

Implementation choices that matter

  1. Anchors: Use 5 concise, domain-agnostic statements (from “definitely not” → “definitely yes”); average over multiple anchor sets to stabilise.
  2. Embeddings model: The paper used OpenAI embeddings; with Google’s Gemini API you can use gemini-embedding-001 and the new GenAI SDK’s models.embed_content.
  3. Persona: Adding demographic attributes to the generator improves signal; without it, correlation drops even if distributions look realistic.

Minimal SSR recipe

  1. Generate a short, natural PI statement with gemini-2.5-flash.
  2. Embed: compute vectors for the statement and five anchors with gemini-embedding-001 via client.models.embed_content(...).
  3. Score: cosine similarity → normalise → pmf → expected Likert (E[Likert]).
  4. Report: show the pmf, the expected score, and the text rationale.

An illustrative correlation-style scatter (not study data):

A scatter plot showing a strong positive correlation between Human mean purchase intent and Synthetic SSR mean purchase intent.

Practical notes

  1. Keep anchors short/generic; don’t bake in product features.
  2. Use 2–6 anchor sets and average pmfs.
  3. For reproducibility, set model temperature low for generation, e.g. 0.2–0.5 (the paper shows little sensitivity to temperature overall).
  4. Track both ranking signal (correlation of mean PI) and distribution similarity (e.g., KS).

A stylised demographic slice (not study data):

Line chart showing illustrative mean purchase intent scores across four age cohorts, comparing real data to two AI models.

Assets included (use in your post)

  1. fig1_ssr_mapping.png — Likert pmf produced by SSR (illustrative).
  2. fig2_dlr_ks_hist.png — DLR KS histogram (illustrative).
  3. fig2b_flr_ks_hist.png — FLR KS histogram (illustrative).
  4. fig2c_ssr_ks_hist.png — SSR KS histogram (illustrative).
  5. fig3_correlation_scatter.png — Human vs. SSR mean PI (illustrative).
  6. fig4_demographic_stratified.png — Mean PI by age cohort (illustrative).

Keep in mind

  1. Anchor selection can bias the mapping; average across sets and document anchor text.
  2. Distributions may look “too good” without personas while losing ranking signal; don’t drop demographics blindly.
  3. Respect privacy & brand IP when using real concepts; follow Google’s prohibited-use policies for the API.

References

  1. Maier, B. F. et al. LLMs Reproduce Human Purchase Intent via Semantic Similarity Elicitation of Likert Ratings, 2025.
  2. Gemini 2.5 Flash & model IDs: gemini-2.5-flash.
  3. Embeddings via GenAI SDK: models.embed_content with gemini-embedding-001.

Paper:

2510.08338v2 – LLMs Reproduce Human Purchase Intent via Semantic Similarity Elicitation of Likert RatingsDownload


Dan Petrovic · Oct 15, 11:52