# Semantic Similarity Rating (SSR) — Synthetic Consumer Research
## Overview
Semantic Similarity Rating (SSR) is a method developed by Colgate-Palmolive and PyMC Labs (2025) that uses LLMs to predict consumer purchase intent with ~90% correlation attainment. Instead of asking an LLM for a direct numerical rating (which produces mediocre results), SSR uses role-play and semantic analysis to extract realistic consumer preferences.
## The Problem with Direct LLM Ratings
Asking an LLM "Rate this product from 1 to 5" produces safe, middle-of-the-road responses that don't reflect real consumer behavior. LLMs tend to hedge and produce average scores when asked for direct numerical ratings.
## The SSR Method
1.**Demographic profiling** — The LLM is given a specific demographic profile (age, income bracket, lifestyle)
2.**Product concept exposure** — The LLM is shown a product concept or description
3.**Role-played free-text response** — The LLM is asked to role-play as that consumer and write down raw, unfiltered thoughts about the product
4.**Semantic translation** — A semantic model converts the free-text thoughts into a numerical score
## Key Results
- Tested against **57 real corporate surveys** and **9,300 actual human responses**
- Synthetic AI consumers matched real human buying behavior with **~90% correlation attainment**
- Accurately mirrored how different age brackets and income levels react to price changes
- Provided qualitative feedback that was deeper and more critical than human-written responses
## Important Caveats
- The metric is **correlation attainment** (not raw accuracy): the AI panel hits ~90% of human test-retest reliability
- Measures **survey purchase intent**, not actual purchase behavior
- Works best in **categories the model already knows well** during training
- Independent replication attempts have struggled to match the paper's results (per @miromusing)
## Implications for Market Research
- **Speed:** Can simulate 1,000 hyper-targeted customer interviews overnight
- **Cost:** Eliminates the need for expensive survey panels
- **Granularity:** Can A/B test pricing across every demographic instantly
- **Depth:** Provides richer qualitative feedback than typical survey responses
## Risks & Limitations
- **Data feedback loop:** LLMs are trained on data from the very market research they would replace — if that data source dries up, future models may lose accuracy
- **Replicability:** Early independent attempts suggest the method is not yet consistent enough to fully replace traditional research
- **Category dependency:** Performance degrades for niche or novel product categories
- **Survey vs. reality:** Measures stated purchase intent, not actual purchase behavior (a well-known gap in traditional research too)
