knowledge-base/raw/xpost/2026-06-11_colgate-llm-purchase-intent-ssr.md

type	source_url	retrieved	author	is_thread	quote_count	retweet_count	paper_url	tags
xpost	https://x.com/HowToAI_/status/2065118982659883350	2026-06-12	@HowToAI_	true	872	139	https://arxiv.org/abs/2510.08338	llm market-research semantic-similarity-rating colgate synthetic-consumers

Colgate: LLMs Predict Purchase Intent at 90% Accuracy via Semantic Similarity Rating (SSR)

Author: How To AI (@HowToAI_) Posted: 2026-06-11 Source: https://x.com/HowToAI_/status/2065118982659883350

Summary

Colgate-Palmolive (in collaboration with PyMC Labs, 2025) published a paper demonstrating that LLMs can predict real purchase intent with ~90% correlation attainment using a method called Semantic Similarity Rating (SSR).

The Problem

Asking an LLM directly "Rate this product from 1 to 5" produces safe, middle-of-the-road garbage — mediocre numerical ratings that don't reflect real consumer behavior.

The SSR Method

Instead of asking for a number, researchers:

1. Gave the LLM a demographic profile (age, income bracket, etc.)
2. Showed it a product concept
3. Asked it to role-play as a consumer and write down raw, unfiltered thoughts
4. Used a semantic model to translate those written thoughts into a numerical score

Results

• Tested against 57 real corporate surveys and 9,300 actual human responses
• Synthetic AI consumers matched real human buying behavior with ~90% reliability
• Perfectly mirrored how different age brackets and income levels react to price changes
• Provided detailed, qualitative feedback that was deeper and more critical than what actual humans wrote

Key Clarifications (from thread)

• The metric is correlation attainment: the AI panel hits ~90% of human test-retest reliability (i.e., ~90% of the way to how consistent real people are with their own answers on a retest)
• It measures survey purchase intent, not actual purchase behavior
• Works best in categories the model already knows well

Implications

• Destroys the economics of traditional market research
• Can simulate 1,000 hyper-targeted customer interviews overnight
• Can A/B test pricing across every demographic instantly
• No need to wait a month to see if a product will sell

Critical Perspectives (from comments)

• @miromusing: Attempted replication struggled to get the same results — approach is useful but not yet consistent/reliable enough to replace real research
• @cenkercakin: The LLM is trained on data from the very resource it replaces — what happens when that data source dries up?
• @chris_byrne: Prof. Juster invented the Buyer Intention Scale (0-10) in 1966; Bain later rebadged it as NPS

Paper Reference

• Title: (Colgate-Palmolive + PyMC Labs, 2025)
• arXiv: https://arxiv.org/abs/2510.08338