Study finds no-persona LLM prompting predicts real audience click-through better than persona-based simulation

A new arXiv paper (2609.25010v1) tested whether LLM "synthetic personas" predict real audience click-through better than a plain no-persona LLM prompt. Using the Upworthy Research Archive as ground truth for headline A/B tests, the no-persona baseline beat a ten-persona panel on the reliable subset (n=399): Kendall tau 0.361 vs 0.084, top-1 accuracy 49.2% vs 34.6%, with non-overlapping confidence intervals. The result held across three Upworthy splits, directionally on a separate news dataset, and across three Gemini tiers plus GPT-4.1.

Persona-based LLM panels are widely sold and used in marketing and research to simulate audience reactions before running real tests. This paper suggests that dressing up an LLM as a persona doesn't improve prediction of actual click behaviour and may make it worse, so teams relying on persona simulation for headline or copy testing may be trusting a noisier signal than a simple no-persona prompt would give them. The effect is concrete and replicated within the study, so it's a legitimate reason to question a common workflow, not just run with it.

The "personas are worse than not using them" framing overreaches. This is one narrow task (headline CTR ranking) on one dataset family (Upworthy, plus a partial check on a second news dataset), with a small reliable subset (399 items) and no independent replication yet. It doesn't establish that personas fail at other simulation tasks (e.g. product feedback, ad copy, survey response), only that they underperformed here.

1. 01Independent replication of the tau 0.361 vs 0.084 gap by another lab or on a different dataset outside the Upworthy family
2. 02Whether the result extends beyond headline CTR to other persona-simulation use cases (e.g. product reviews, ad creative, survey panels)
3. 03Any response or counter-study from vendors selling synthetic-persona research tools, and whether they publish their own validity data against real behavioural ground truth