Scalewell / Blog

Synthetic audiences: can AI buyers test your B2B messaging before you spend?

AI buyer panels can screen out weak B2B messages before paid spend, but the evidence says they cannot pick close winners or tell a CFO from a practitioner.

A panel of AI figures reviewing message cards while one real person looks on with a magnifying glass

Synthetic audiences are panels of AI personas that answer research questions the way a target buyer might. For B2B messaging they can screen out weak or confusing messages before paid spend, but they cannot yet pick a winner between close options or tell you how each role on a buying committee will react.

Picture the quarterly planning meeting. Three message angles are on the slide, the LinkedIn budget is fixed, and someone from the product team asks why you can't just ask an AI which one buyers will like. It's a fair question. A real test on a narrow ICP takes weeks, and a vendor demo just showed a panel of 500 "CFOs" answering in minutes.

This post is about the method itself: what a synthetic panel is, what the published evidence says for and against it, and a set of rules for when a head of demand gen can trust one. Deciding which messages to fund before launch is covered in our guide to message testing before paid spend, and reading tests once ads are live is covered in AI creative testing.

Synthetic audiences are AI personas answering on behalf of real buyers

A synthetic audience is a set of large language model personas, each given a profile (role, industry, company size, sometimes past answers or interview transcripts), then asked the same questions you would put to a survey panel or a focus group. Vendors sell them as synthetic research, synthetic personas or AI focus groups. The names differ, and the mechanics are mostly the same.

The idea comes from academic work. In 2022, Lisa Argyle and colleagues published Out of One, Many, which built "silicon samples" by conditioning GPT-3 on sociodemographic backstories taken from real participants in large US surveys. They argued that proper conditioning lets a model emulate response distributions from many human subgroups. Most commercial tools are a descendant of that idea, with a chat interface and a dashboard on top.

What separates one tool from another is the grounding. A panel built from a job title and a one-line description is the model's general impression of that kind of person. A panel built from your own customers' interviews, survey answers or sales calls is closer to a compressed copy of people you actually know. That difference turns out to matter more than the model you pick, and it decides most of what follows.

Catching weak messages before they cost a round of LinkedIn testing is the kind of recurring job Scalewell sets up for a team to hand to AI, grounded in its own sales conversations and closed deals.

The evidence for synthetic research is real but narrow

The strongest result so far comes from Joon Sung Park and colleagues, who interviewed 1,052 US adults for two hours each, then built an AI agent for each person. Asked questions from the General Social Survey, agents built from the interview and survey data together matched people's real answers 72.8% of the time. The people themselves matched their own answers 79.5% of the time when asked again two weeks later, and agents built only from demographics reached 60.4%.

Read those numbers as a ladder. Demographics alone leave agents about 19 points short of how consistent real people are with themselves, and rich, personal grounding cuts that gap to about 7 points. A B2B vendor selling personas built from a job title is selling the bottom rung.

bar chart of agreement with real answers on General Social Survey questions in Park et al. (arXiv 2411.10109, current version): demographics-only agents 60.4%, survey-based agents
Grounding moves synthetic answers closer to how consistent real people are with themselves.

Bain & Company reported a commercial version of the same pattern in May 2026. For a consumer technology client, it built digital twins from historical respondent-level data and ran them through the tasks of a prior large conjoint study, which was kept out of the training data. Bain says "the digital twins replicated about 90% of key outcomes from the original research", including which features mattered most and the right launch or retain calls. Bain also makes the B2B point directly: some markets have "too few key customers, such as CFOs in a single industry, to reliably sample", which is exactly why synthetic panels appeal to B2B teams.

So the case for is this: when the personas are built from real data about the people they represent, they can reproduce aggregate answers well enough to be useful. Both results rest on rich, person-level data, and Bain's is the firm's own client work rather than an independent study.

The evidence against shows up in the places B2B cares about most

The same research that supports synthetic panels also shows where they break, and the breaks line up with how B2B buying works.

Answers are too tidy. James Bisbee and colleagues compared ChatGPT personas with the American National Election Studies in Political Analysis in 2024. They found "less variation in responses than in the real surveys": the synthetic standard deviation on one key measure was 16.1 against 31.4 in the real data. In their analysis, 48% of regression coefficients from the synthetic data differed significantly from the real ones, and the sign flipped in 32% of those. A panel that understates disagreement will make two messages look further apart, or closer together, than buyers would.

Answers drift. The same paper found that "the same prompt yields significantly different results over a 3-month period", which the authors link to changes in the underlying model that they could neither see nor control. If you test in March and again in June, part of any difference may come from the model rather than the market.

Panels are polite. Nielsen Norman Group's review of synthetic users found they "often provide shallow or overly favorable feedback" and that "synthetic users seem to care about everything." In one study, synthetic learners claimed to have finished online courses that real learners mostly abandoned. A message test where every option scores well is a test that can't fail, and a test that can't fail tells you nothing.

Subgroups get flattened. This is the one that hurts B2B the most. B2B International gave a synthetic provider two-thirds of its Superpowers Index data (3,000-plus B2B decision-makers) and asked it to simulate the rest, according to NewtonX's account of the test. Overall accuracy came back at 86%, but cut by seniority, company size and business type, the synthetic data "flattened everything, masking the subgroup differences that B2B research exists to find." When simulating the next wave of the study, synthetic outputs matched actual trends 42% of the time. NewtonX sells human B2B research, so weigh its framing, but the numbers come from the test itself. A September 2026 paper by Florian Kutzner and colleagues makes the same general point in its abstract: aggregate accuracy hides the misrepresentation of subgroups.

Small buying committees are where synthetic panels get unreliable

A B2B message rarely wins by appealing to an average buyer. It wins because the CFO stops objecting, IT security sees its question answered, and the operations lead sees their week getting easier. Those are three small groups with different worries, and they're the subgroups that synthetic data flattens.

6sense's 2025 Buyer Experience Report, a survey of buyers across North America, EMEA and APAC, found that 94% of buying groups ranked preferred vendors before first contact. Your message has to work on each member while they research alone, long before sales gets a call. An 86% headline match that blurs the CFO into the practitioner is wrong in the exact spot where B2B messaging is decided.

There's a second problem with small committees: there is very little real data to ground each role in. Each Park-style agent was built from a two-hour interview with one person. Your team might have a few dozen recorded calls with CFOs in your ICP, and most of those calls were with prospects who were already talking to sales. The panel inherits whatever bias those calls carry.

And there's the recency problem. If your category is shifting, perhaps because a new regulation landed or a competitor reset pricing, a panel built on last year's conversations answers with last year's opinions. Brand-new categories are worse still, because nobody in the training data has bought the thing yet.

An honest summary: synthetic panels are most reliable for questions about a broad, well-documented audience, and least reliable for a narrow, shifting committee where each role holds a veto. Most B2B paid media lives closer to the second description.

When to trust a synthetic panel on B2B messaging

A synthetic panel earns its place as a filter before paid spend, not as a judge of which message wins. We'd use the scorecard below on any question you plan to put to one.

Question you want answeredTrust a synthetic panel?Why
Is this message clear, or full of jargon?YesComprehension does not depend on subgroup nuance
Which objections will this message raise?Yes, as a list to checkPanels surface plausible objections; sales confirms which are real
Which of these five angles are clearly weak?Yes, with groundingLarge gaps survive the tidy-answer bias
Which of two close messages will win?NoUnderstated variance makes close calls unreliable
How will the CFO react compared with IT security?No, unless each role is grounded in its own dataSubgroup differences get flattened
Will this message produce pipeline?NoSynthetic research produces no behavioral data
How will buyers react to a new category or a market shift?NoPanels answer from past data

The pattern is simple once you see it. Use the panel to cut a long list down to a short one, and to make each surviving message clearer. Then let real buyers decide between the survivors, either with a quick human check or with an in-market test on a channel like LinkedIn.

Funnel narrowing from many draft messages through a filter to a small group of ad tiles
Draft messages go through a synthetic panel that removes unclear and clearly weak angles, then a small human check, then a paid test on LinkedIn or Google that measures pipeline.

That split matters for budget. An in-market LinkedIn test on a narrow ICP is slow because the audience is small, and LinkedIn won't run a matched audience under 300 members (see our guide to AI audience segmentation for the other platform floors). Every weak message the panel removes is one fewer cell splitting that small audience, which means the remaining test has a better chance of finishing.

Backtest a synthetic panel before you let it vote

The only way to know whether your panel works for your buyers is to test it against answers you already have. Pick past message tests where you know the outcome on something that matters, ideally opportunities created rather than clicks, and ask the panel to predict them without telling it the result.

A hypothetical example shows how. Imagine a team selling workflow software to mid-market finance departments. It has six past LinkedIn tests where one message clearly beat the other on opportunities. It builds a panel grounded in 40 recorded sales calls and its win and loss notes, then asks the panel to choose between each pair. If the panel picks the right message in five of the six, that's a reason to use it for screening. If it picks right in three of six, it's doing no better than a coin, and the team should not let it remove anything. Even five of six would happen by luck about one time in nine (7 in 64), so a pass earns the panel a screening job, not a vote.

We'd set the bar before running the backtest, not after. In this hypothetical, five of six is the bar, and anything under it means the panel stays a brainstorming tool. Rerun the backtest every quarter or after any change to the underlying model, since the Bisbee results show outputs move on their own.

Three things improve a panel that falls short, in rough order of payoff. Ground each role in its own material instead of one blended persona. Add recent material, such as the last quarter's calls, so the panel isn't answering from an old market. And make the panel disagree on purpose by asking each persona for its strongest objection before asking for a score, which works against the politeness NN/g describes.

Synthetic audiences mislead teams that treat scores as results

The most common failure is treating the panel's ranking as a result. A dashboard that says message B scored 7.8 and message A scored 7.4 looks like evidence, and it gets pasted into the planning deck. Given what Bisbee found about variance, a gap that size from a synthetic panel says nothing about which message buyers would choose.

The second failure is quieter. Mark Himmelsbach, co-founder of the creative AI tool RYA, warned in a Marketing Dive guest column in November 2026 that "constant scoring risks making a team more conservative and risk-averse rather than more creative and productive." A panel trained on what buyers said before rewards messages that sound like what buyers heard before. If every brief now has to pass the panel, the bold angle never gets tested at all, and somebody will eventually ask why the ads all sound alike.

The third is buying the bottom rung of the ladder. A tool that builds personas from a job title and a firmographic is selling the model's stereotype of a CFO. Ask any vendor what the personas are grounded in, how often that data is refreshed, and whether they will run your backtest before you sign. Our AI vendor due diligence questionnaire has the data and model questions to add.

Bain's own advice fits here: it says "it's most effective to start by augmenting, not replacing, existing research methods." For a B2B team that means the panel sits in front of buyer conversations and paid tests, never in place of them.

How Scalewell gets a grounded buyer panel working for your team

The gain is a shorter list of better messages reaching paid tests, which leaves your LinkedIn budget fewer cells to split. After a free 30-minute call, we run an audit of your team's week that scores every recurring job for AI, and those scores become a ranked roadmap. When message screening ranks high, the build sprint grounds the panel in your own calls and win and loss notes, runs the backtest against past tests, and places it inside the tools your team already uses, typically over 8 to 12 weeks. B2B paid media practitioners do the work. Your team then learns to refresh the panel and rerun the backtest without Scalewell.

Where to start with synthetic audiences

Start with the backtest before any tool decision. Pull your last five or six message tests with a clear winner on opportunities, write down the result, and see whether any panel you're considering can call them blind. That one exercise tells you more than any vendor demo.

If the panel passes, put it in front of your next round of LinkedIn testing as a filter and nothing more. If you'd like help setting that up on your own data, Scalewell offers a free 30-minute call to talk it through.

Frequently asked questions

How much do synthetic audience tools cost?

Pricing varies widely by vendor and is usually set per study or per seat, so ask for a quote on the exact study you plan to run. Compare the quote with what a small human panel of your actual buyers would cost, and with the LinkedIn budget a wasted test cell burns. A cheap tool that misleads a test costs more than it saves.

Can I build a synthetic panel with ChatGPT or Claude myself?

Yes, a general assistant can play a set of personas if you give it role profiles and real material such as call transcripts. The hard parts are the same as with a vendor: grounding each role, keeping the persona prompts fixed between runs, and backtesting against known results. Without those, a do-it-yourself panel is a brainstorm.

Are synthetic personas the same as buyer personas?

No, a buyer persona is a written profile a team uses to brief creative and sales. A synthetic persona is an AI model playing that profile and answering questions. A good buyer persona is useful raw material for a synthetic one, but a persona document on its own is too thin to ground a reliable panel.

Is it legal to build synthetic personas from customer call recordings?

That depends on your consent language, your contracts and where your customers are based. Recorded sales calls often carry consent for internal quality use, which may or may not cover feeding them to a third-party AI tool. Ask legal before uploading transcripts, and prefer tools that keep your data out of model training.

Do synthetic audiences work for ad creative as well as copy?

Some tools accept images and video, but the evidence base is thinner than for text. Panels can flag an unclear visual or a missing product shot. They can't tell you whether the creative stops a scrolling buyer, because that is behavior, and synthetic research produces no behavioral data.