---
type: xpost
source_url: https://x.com/HowToAI_/status/2065118982659883350
retrieved: 2026-06-12
author: "@HowToAI_"
is_thread: true
quote_count: 872
retweet_count: 139
paper_url: https://arxiv.org/abs/2510.08338
tags: [llm, market-research, semantic-similarity-rating, colgate, synthetic-consumers]
---

# Colgate: LLMs Predict Purchase Intent at 90% Accuracy via Semantic Similarity Rating (SSR)

**Author:** How To AI (@HowToAI_)
**Posted:** 2026-06-11
**Source:** https://x.com/HowToAI_/status/2065118982659883350

## Summary

Colgate-Palmolive (in collaboration with PyMC Labs, 2025) published a paper demonstrating that LLMs can predict real purchase intent with ~90% correlation attainment using a method called **Semantic Similarity Rating (SSR)**.

### The Problem

Asking an LLM directly "Rate this product from 1 to 5" produces safe, middle-of-the-road garbage — mediocre numerical ratings that don't reflect real consumer behavior.

### The SSR Method

Instead of asking for a number, researchers:
1. Gave the LLM a **demographic profile** (age, income bracket, etc.)
2. Showed it a **product concept**
3. Asked it to **role-play as a consumer** and write down raw, unfiltered thoughts
4. Used a **semantic model** to translate those written thoughts into a numerical score

### Results

- Tested against **57 real corporate surveys** and **9,300 actual human responses**
- Synthetic AI consumers matched real human buying behavior with **~90% reliability**
- Perfectly mirrored how different age brackets and income levels react to price changes
- Provided detailed, qualitative feedback that was deeper and more critical than what actual humans wrote

### Key Clarifications (from thread)

- The metric is **correlation attainment**: the AI panel hits ~90% of human test-retest reliability (i.e., ~90% of the way to how consistent real people are with their own answers on a retest)
- It measures **survey purchase intent**, not actual purchase behavior
- Works best in **categories the model already knows well**

### Implications

- Destroys the economics of traditional market research
- Can simulate 1,000 hyper-targeted customer interviews overnight
- Can A/B test pricing across every demographic instantly
- No need to wait a month to see if a product will sell

### Critical Perspectives (from comments)

- @miromusing: Attempted replication struggled to get the same results — approach is useful but not yet consistent/reliable enough to replace real research
- @cenkercakin: The LLM is trained on data from the very resource it replaces — what happens when that data source dries up?
- @chris_byrne: Prof. Juster invented the Buyer Intention Scale (0-10) in 1966; Bain later rebadged it as NPS

### Paper Reference

- **Title:** (Colgate-Palmolive + PyMC Labs, 2025)
- **arXiv:** https://arxiv.org/abs/2510.08338