Can synthetic respondents replace real B2B survey participants? What the research shows
What the published tests show about how close AI personas get to real survey respondents, why B2B may be harder, and where synthetic data can still help.
By Jennifer Ray, founder of Valid N ResearchPublished
Real people vs AI personas
1,519 US adults answered a coffee survey. AI personas, each built to match one of them on age, gender, and other demographics, answered the same questions.
Bought from Dunkin' in the past year45 pt gap
Real47%
AI92%
Drink coffee at least once a day35 pt gap
Real56%
AI91%
Never drink coffee17 pt gap
Real17%
AI0%
The AI answered like keen coffee drinkers. Not one said they never drink coffee.
Verasight, 2025. GPT-5 personas. A consumer test, not B2B.
Key points
Synthetic respondents are simulated participants: an AI model told to answer as a particular kind of person. Their answers are synthetic responses, not observations from a real person. Used openly, they're a research tool. Passed off as real respondents, they mislead the buyer.
In published tests, they sometimes match the broad pattern of real answers (more than 75% agreement with people's ratings of brands, in one study), but they often miss the actual numbers. In a test of coffee questions, they were off by about 20 percentage points on average.
The misses follow a pattern: less variation than real people, answers that lean positive or toward what you'd expect, and subgroups that look more like stereotypes than people.
Almost all of the published tests used consumers or the general public. The two B2B tests we found, both run by research companies, raised different concerns: one found less varied answers from synthetic IT professionals on job attitudes, and the other found weak accuracy for some subgroups and for year to year change. Neither is definitive; both were commercial evaluations that weren't peer reviewed, and neither found synthetic data a substitute for real respondents.
They're most useful before fieldwork: pretesting a questionnaire, checking a screener, and generating ideas to test with real people.
For market sizing, pricing, segment comparisons, trackers, and anything you publish as survey findings, use real respondents. If a vendor offers synthetic data, ask how it was validated against real responses from your audience.
Vendors promise synthetic survey answers in minutes instead of weeks, at a fraction of the usual cost. You give an AI model a persona, such as an IT director at a 2,000 person software company, and it answers your questionnaire the way that person might. Large survey platforms now sell them. Qualtrics announced synthetic panels in 2025, saying they could cut survey costs by up to 70%. So the question comes up in more and more briefs: can these replace real respondents?
For measurement, the published research says not yet. Synthetic respondents can get the broad direction right on familiar topics, and the methods are improving. But their answers tend to be too uniform, often too positive, and too stereotyped, and tests often show weaknesses on small subgroups and unfamiliar questions, which are challenges B2B researchers commonly face. This guide covers what synthetic respondents are, what the tests show, why B2B is a harder case, and where they can still be useful.
A note on where we stand. We sell real B2B sample, so we have an interest in the answer. That's why every study below is linked so that you can check the results for yourself.
What are synthetic respondents?
A synthetic respondent is a simulated survey participant. Instead of recruiting a person, you give a model (usually a large language model like GPT, Gemini, or Claude) a description of who it should be, and ask it your questions. Researchers sometimes call a set of them a silicon sample, a term from a 2023 study by Argyle and colleagues.
The terms aren't standardized, and vendors use them for quite different products. Ask which one you're being offered.
Four kinds of synthetic respondent: what each means and what it starts from
Term
What it usually means
What it starts from
Synthetic persona
A model answers as a described type of person
A written profile, such as job title, industry, and company size
Digital twin
A model answers as a specific real person
That person's earlier survey answers, or an interview with them
Synthetic boost
Generated records added to a real sample, often to fill out a small subgroup
A real survey of the same audience
Synthetic panel
A ready made pool of personas you can survey on demand
The vendor's model and whatever data it was given
Two related things use the same technology but aren't synthetic respondents:
An AI-assisted response comes from a real person who uses a chat tool to help write their answers.
A bot or AI agent posing as a person is fraud. In a 2025 PNAS study, an AI agent built by a Dartmouth researcher to test survey defenses passed standard attention checks in 99.8% of 6,000 trials. Our guide to preventing survey fraud covers how to keep them out.
The key distinction is how the output is used: a disclosed synthetic response is modeled data, while an agent submitted as though it came from a verified human is respondent fraud. The 2025 ICC/ESOMAR International Code covers this directly. A client must be told when AI or synthetic data is used to compile a dataset or to analyze, report, or interpret the findings, and how much human oversight was involved. Published findings must disclose AI or synthetic data when it played a significant role. The Code binds ESOMAR members and the research associations that adopt it.
How close do synthetic respondents get to real survey data?
It depends on the topic, the method, and what you measure. Some studies found close matches on broad patterns. Others found large errors in the specific numbers a study would report.
Agents built from two hour interviews with 1,052 people, answering General Social Survey questions
What it found
As a share of the consistency people show when they retake the same questions two weeks later, interview agents reached 83%, survey agents 82% and agents using both 86%, against 74% for demographics alone. An earlier version reported 85%
ChatGPT personas based on US election study respondents, rating political and social groups
What it found
Averages close, but far less variation than real people. Nearly half the regression results differed significantly, and the same prompt gave different results three months later
52 single answer questions on six topics, 2,000 US adults
What it found
Off by 14.5 points on average. Questions where people could choose more than one answer were left out of that average because the model did so badly on them
Claude Opus 4.6 answering as members of Pew's American Trends Panel, given each person's profile and earlier survey answers, on nearly 300 questions across three 2026 waves
What it found
Off by 12.4 percentage points on average, measured as the gap between the synthetic and real share choosing each answer option. The average gap topped 15 points on about 28% of questions
Asking for one answer per demographic persona roughly tripled the error in the spread of answers, compared with simply asking the model for the overall distribution. No method matched a second real sample
These studies are not equal in weight. Argyle, Li, and Bisbee appeared in peer-reviewed journals, and Kaiser appeared in a journal article. Park, Maier, and Gordon are preprints or working papers. Verasight's are company reports, and Pew's is an independent methods experiment published by a research organization. None of them tested B2B audiences.
Two patterns stand out.
The closer matches are on broad patterns. Which brands people see as similar, how groups rank, which way a relationship runs. The misses come on the numbers a report usually leads with: the share who bought something, how satisfied a segment is, how far apart two subgroups are.
Better results required real data about the people being simulated. Park's agents were built from two hour interviews with the actual people, and did better than agents built from demographics alone. In Verasight's coffee test, adding each person's real coffee habits to the prompt cut the error on a pricing question from 20 points to 9. A synthetic respondent built from a job title and a company size has much less to go on.
What the misses look like
The Verasight coffee test is a clear example. The synthetic respondents were given the demographics of 1,519 real US adults who had answered the same questions.
Real and synthetic respondents on five coffee questions in Verasight's 2025 test
Question
Real respondents
Synthetic respondents
Aware of Starbucks
91%
100%
Aware of Dunkin'
87%
100%
Bought from Dunkin' in the past year
47%
92%
Drink coffee at least once a day
56%
91%
Never drink coffee
17%
0%
From Verasight's 2025 test. The synthetic respondents were given the demographics of the real ones.
The synthetic sample behaved like a population of keen coffee drinkers who knew every big brand, and none of them said they never drink coffee. If you were sizing a market from it, every one of those numbers would have misled you.
Where do synthetic respondents go wrong?
The errors in these studies aren't random. They follow a handful of patterns, and several could be especially consequential in B2B research.
Too little variation
Synthetic answers cluster together. In Bisbee and colleagues' study, the synthetic answers on one measure varied about half as much as the real ones. That makes small samples look more precise than they are. In a worked example in the paper, at the usual 80% power, the synthetic data implied 16 respondents would be enough to detect a change that the real data said needed 129. Don't size a study from synthetic variation, and don't treat synthetic records as independent respondents in significance tests.
Answers that lean positive, or toward the expected
In the Kaiser study, synthetic ratings were more positive than real ones, especially for well known brands. In the coffee test, awareness of the big brands was 100%. The direction of the distortion depends on the model: in Pew's test, one model portrayed a more polarized public than the real one and another a more moderate one.
Stereotypes instead of people
A 2025 study in Nature Machine Intelligence compared four language models with 3,200 people across 16 demographic identities, and found the models misportrayed groups and flattened the differences within them. An earlier test found the opinions of language models differed from those of US demographic groups by about as much as Democrats and Republicans differ on climate change, even when the model was told which group to represent.
Missing the unusual answers
In Verasight's 2026 test, on questions where people could choose more than one answer, the model never chose 24 of the 77 options. One option picked by 40% of real people was never picked at all. Pew's 2026 test found the same on public opinion questions: on nearly half of them, at least one answer option was never chosen by any synthetic respondent, and real people said "not sure" about four times as often. Small groups, minority views, and "don't know" answers are the easiest to lose.
Weak on anything new
In a working paper by Brand, Israeli, and Ngwe, willingness to pay estimates from language models were sometimes close to human ones, but often inaccurate and sometimes pointed the wrong way. Training the model on earlier surveys helped with new features in the same product category, but not with new categories or with differences between customer segments.
Knowing too much
In a test of the wisdom of crowds, Aher and colleagues found that some newer models gave answers more accurate than real people would, which they called hyper-accuracy distortion. A real buyer doesn't know every vendor's pricing or every product's specification. Pew saw it too: on a question about NATO, 99% of synthetic respondents answered correctly against 56% of the public. If you're measuring awareness or understanding, a model that knows the answer isn't telling you what your market knows.
Results that move
Bisbee and colleagues ran the same prompt in April, June, and July 2023. The July results, collected after the model behind ChatGPT was updated in late June, differed markedly from April's. If the model behind a synthetic panel changes, a tracker built on it can shift even if the market doesn't. Our guide to brand tracker sample drift covers the same problem with real sample.
Validation that rewards the wrong thing
In the Gordon study, the method that was worst at reproducing the spread of answers was among the best at predicting which answer would come out on top. A vendor that validates only on the top answer can look accurate while getting the distribution wrong.
Is B2B a harder case for synthetic respondents?
We'd expect so, though it hasn't been measured properly. Most published tests used consumers or the general public, usually in the US, on topics with a lot of public text behind them: politics, big brands, everyday products. B2B studies differ in ways that line up with the weaknesses above. What follows is our reasoning from that evidence, not a tested result.
The audiences are small and specific. A study of finance leaders at companies with 1,000 or more employees, or of security teams on a particular platform, depends on groups that are thinly represented in what models learn from. Errors were already larger for subgroups in the consumer tests: in Verasight's first test, the best model was off by about 4 points overall and about 8 points for subgroups.
A persona leaves out what drives the answer. A real IT director's answers depend on their current systems, contract dates, budget cycle, and the last vendor that let them down. Job title, industry, and company size leave almost all of that out, and the coffee test showed how much better results were with real behavior data about each person.
Comparing segments is usually the point. B2B reporting compares company size, industry, and role. Flattening the differences between subgroups hits the analysis buyers care about most.
The questions are often new. B2B research is often about a new product, a new price, or a category that's still forming, which is where the evidence is weakest.
There's no one to verify. With a real B2B sample, the question is whether the respondent really holds the role. With a synthetic sample, nobody holds it. You're relying entirely on how well the model was built and checked.
We found two published B2B tests. Both were run by research companies that sell work with real respondents, so they have an interest in the result, as we do.
Emporia Research, a B2B sample company, compared LinkedIn verified IT decision makers with two kinds of synthetic respondents: personas of IT decision makers written by an AI model, and synthetic answers built from the real respondents' LinkedIn profiles. The survey used closed questions about their jobs, and on the satisfaction questions, Emporia reported that both synthetic groups leaned more positive than the real respondents. For the personas:
Job satisfaction. 69% of the personas were strongly satisfied with their current role. The most common real answer was somewhat satisfied (47%).
Work life balance. Every persona answered somewhat satisfied, compared with 45% of real respondents.
On a 1 to 10 rating, real answers ran from 3 to 10, while synthetic answers ran only from 6 to 10. Emporia didn't publish its sample sizes or which model it used, and it's a company case study. These questions were about job attitudes, not buying technology, so they say nothing directly about purchase decisions.
B2B International, a B2B research agency, tested a synthetic data supplier on its annual global survey of more than 3,000 B2B buyers. In the first test, the supplier was given two thirds of the real 2024 sample and asked to generate the last third. The filled data set was no more accurate overall than simply using the smaller real sample, and it was less accurate for some subgroups, such as seniority and business type. A driver analysis run on it looked little like the real one. In the second test, the supplier was given the 2023 data and asked to simulate 2024. By the agency's own measure, the figures averaged about 86% accuracy, but they moved in the right direction from one year to the next only 42% of the time.
Neither test was peer reviewed, and each tested one company's product on one data set. Treat them as illustrations rather than proof, but they point the same way as the consumer research.
Where can synthetic respondents help?
A 2024 review by Sarstedt and colleagues concluded that synthetic samples hold the most promise early in the research process, in pretests and pilot studies, rather than as the main data. That matches the evidence above. Good uses include:
Pretesting a questionnaire. Run the questions past a few dozen personas to catch wording that reads two ways and answer options that are obviously missing, before any real respondent sees them. It's a first pass only. It doesn't test whether the routing is programmed correctly, and it doesn't replace testing with real people or a soft launch. If the questionnaire is confidential, use an AI tool your company or client has approved for confidential material.
Checking a screener. Ask a model to try to qualify for your study while playing someone who shouldn't. If it gets through, your qualifying questions are too easy to guess. It's a one way test: if the model is turned away, that tells you nothing about fraudsters using real profiles, and it doesn't test your other fraud checks. Our guide to IT decision maker screener questions covers questions that are harder to bluff.
Generating ideas. Brainstorm features, objections, answer lists, and wording to test. Treat everything it produces as a hypothesis for real respondents to confirm or reject.
Screening early concepts, with care. In the Maier preprint, synthetic purchase intent for personal care products came close to how consistent real people are, but only when the model answered in words that were then mapped to the rating scale. The Kaiser study found synthetic brand ratings biased upward. Mapping words to the scale made its ratings more varied, but they were still too positive. We haven't seen either shown for B2B audiences. If you use synthetic ratings at all, use them to cut a long list, and test the shortlist with real respondents, sized with our guide to B2B concept test sample size.
When do you need real B2B respondents?
For any number that will drive a decision or be published as a survey finding. Modeled estimates can be legitimate when the method and its uncertainty are disclosed, but on current evidence, synthetic data isn't a stand-alone substitute.
When to use synthetic respondents and when to use real ones, by task
Task
Synthetic respondents
Real respondents
Drafting and pretesting a questionnaire
Useful
Still need a soft launch
Brainstorming features, objections, and answer options
Useful as a starting point
Confirm which ones matter
Market sizing and incidence
Not reliable on current evidence
Needed
Pricing and willingness to pay
Not reliable, sometimes the wrong direction
Needed
Comparing company sizes, industries, or roles
Not reliable, differences get flattened
Needed
Tracking change over time
Not reliable, the model can change
Needed
New products or categories
Least evidence
Needed
Statistics you'll publish or put in a press release
Only as clearly labeled modeled results with the method disclosed
Needed for survey findings
Real respondents are not an automatic guarantee of quality. B2B panels can suffer from coverage and selection bias, inaccurate job claims, and inattentive answers, so good screening, source documentation, and quality checks are still essential. This comparison assumes appropriately qualified human participants, not any unverified online sample.
How to evaluate a synthetic data vendor
If you're offered synthetic or boosted data, these questions separate a tested product from a demo:
Which model, and which version? Will it stay fixed for your project, and will you be told if it changes?
What is each respondent built from? A written persona, real survey data, or interviews with real people? For B2B, how was the business audience represented?
How was it validated, and against whom? Ask for a comparison with real responses from your audience and category, not a general population poll. Ask to see the comparison itself, not just a summary score.
Which measures? Ask for the error on the share choosing each answer, the spread of answers, and the differences between subgroups, not just whether the top answer matched.
How big are the errors for your segments? Ask for results by the company sizes, industries, or roles you plan to report.
Can the results be reproduced? Ask what happens when the same questions are run again a month later.
What real data sits underneath, and how old is it? A synthetic boost is only as good as the real sample it builds on.
What error would be acceptable for your decision? Agree it before the test, and ask how the method compares with a smaller real sample or a simple baseline.
How is leakage prevented? Ask how validation questions and answers are kept out of the model's training, prompts, and tuning, and whether results hold when the same frozen prompts are rerun.
How will it be disclosed? A supplier that follows the ICC/ESOMAR Code has to tell you when AI or synthetic data was used and how much human oversight was involved. If it played a significant role, findings you publish must say so too.
The American Association for Public Opinion Research's 2026 task force report gives a general way to judge any AI tool by the task it's used for: whether it targets the intended task, how well it performs it, whether results hold when prompts or models change, and whether the whole process reproduces consistent conclusions. It treats synthetic responses as the riskiest use it considers, and says data generated by AI should be clearly distinguished from data from human respondents. It's general survey guidance, not a test on B2B audiences.
Using synthetic and real data together
If you use both, keep them clearly apart:
Label synthetic results wherever they appear, and report them separately from real ones.
Check against a real sample from the same audience before relying on anything, for example, by running the synthetic sample alongside a soft launch and comparing the two.
Check the inputs. Before putting a client questionnaire, respondent histories, or interview transcripts into any AI tool, confirm the client has approved it, that the data may be used this way, that consent is in place where required, how the vendor retains and trains on inputs, and what security controls apply. Remove identifying details where you can.
Report real base sizes. Don't add synthetic records to a subgroup to make the base size look bigger unless the method is disclosed and has been validated for that audience.
Use real data for statistics. Synthetic answers vary too little, so sample sizes and ordinary significance tests built on them will overstate your confidence, and generated records aren't independent human observations. Size the real study with the sample size calculator.
Sources
The studies and reports this guide draws on. Each title links to the original.
Park, J. S., Zou, C. Q., Kamphorst, J., Egan, N., Shaw, A., Hill, B. M., Cai, C., Morris, M. R., Liang, P., Willer, R., and Bernstein, M. S. (2024). LLM agents grounded in self reports enable general-purpose simulation of individuals. arXiv preprint 2411.10109, first posted November 2024 as Generative agent simulations of 1,000 people, revised June 2026.
Santurkar, S., Durmus, E., Ladhak, F., Lee, C., Liang, P., and Hashimoto, T. (2023). Whose opinions do language models reflect?. Proceedings of the 40th International Conference on Machine Learning, PMLR 202, 29971-30004.
Synthetic respondents are simulated survey participants. An AI model, usually a large language model, is told to answer as a particular kind of person and given your questions. They're also called synthetic personas or silicon samples. A digital twin is a narrower case, modeled on one specific real person.
Can synthetic respondents replace real survey respondents?
Not for measurement, based on the published research. They can match broad patterns on familiar topics, but they tend to give answers that are too uniform, often too positive, and too stereotyped, and they miss specific numbers by wide margins. They're most useful for pretesting and generating ideas before a real study.
How accurate are synthetic respondents?
It varies widely by topic and method. One study found more than 75% agreement with people's brand ratings. A test of coffee buying questions found synthetic answers off by about 20 percentage points on average. A 2026 test across six topics found an average error of 14.5 points on single answer questions, and the questions that allowed several answers went so badly they were left out of that average. The closest matches used real data about each person, such as interviews or past behavior.
Do synthetic respondents work for B2B research?
There's very little evidence either way, because almost all published tests used consumers or the general public. The two B2B tests we found were run by research companies, weren't peer reviewed, and measured different things. In Emporia's, synthetic personas gave more positive and less varied answers than real IT decision makers on questions about their jobs. In B2B International's, filling part of a real sample with synthetic records was no more accurate than using the smaller real sample, was worse for some subgroups, and tracked year to year change poorly. Neither shows that B2B audiences do worse than consumer ones. We expect they may, because they're small and specific, but that hasn't been measured.
What is the difference between synthetic respondents and survey bots?
The key distinction is how the output is used. A disclosed synthetic response is modeled data, while an agent submitted as though it came from a verified human is respondent fraud. Both can be built on the same language models, so the difference lies in disclosure and purpose, not the technology.
What is a digital twin in market research?
A digital twin is a synthetic respondent modeled on a specific real person, built from that person's earlier answers or an interview with them. Twins built from rich data have performed better than personas built from demographics alone. In one study, agents built from two hour interviews reached 83% of the consistency people show when they retake the same questions two weeks later, and 86% when interviews and surveys were combined, compared with 74% for agents built from demographics alone.
When should you use synthetic respondents?
Before fieldwork, for pretesting questionnaires, checking screeners, and generating ideas. Use real respondents for market sizing, pricing, segment comparisons, and tracking, and for survey findings you plan to publish. Modeled or synthetic results can be appropriate when they've been validated for your audience and are clearly labeled and disclosed, but not as a stand-alone substitute.
Do you have to disclose synthetic data?
Under the 2025 ICC/ESOMAR International Code, yes. The Code binds ESOMAR members and the research associations that adopt it. Clients must be told when AI or synthetic data is used to compile a dataset or analyze the findings, along with the extent of human oversight, and published findings must disclose it when it played a significant role. Presenting synthetic answers as real respondents breaks the Code and misleads the buyer.
Will synthetic respondents get better?
Probably. The models are improving, and studies with richer data about real people produced closer results. But the core problems, too little variation and stereotyped subgroups, showed up across models and years. Ask for current validation on your own audience rather than relying on general claims.
Send the spec. Get real numbers back.
Audience, market, target n, expected interview length. Feasibility the same business day for standard audiences, with reachable counts, an estimated qualification rate we believe and a quoted cost per complete that stays fixed for the agreed brief.
We use cookies that are needed to run the site and keep the forms working. We would also like to set analytics cookies to understand which pages are useful. Analytics stay off unless you accept.