In July I wrote about custom audiences in ChatGPT Ads and a problem I figured was going to get expensive.

Someone's in ChatGPT comparing products, and the answer may already recommend your company. Your ad shows up under it, they click, they buy, and the dashboard hands the ad the sale. How many of those people were buying anyway?

At Webflow we shut off paid branded search completely and self-serve acquisition moved less than 3%. We'd been paying to catch people who were mostly coming without the ads, and the dashboard had been calling it a top performer the whole time.

ChatGPT is harder to read than that. With branded search, at least the person typed your name. In a conversation the question can be generic and the answer can put you in front of the buyer first. Then they click your ad and it gets credit for demand the answer already captured.

So I recommended suppressing customers and active pipeline, then running an incrementality test. I also called out that there was no native holdout.

On October 5, OpenAI said it's working with Haus, Measured and WorkMagic to explore geo-based experiments for ChatGPT Ads. It calls its incrementality work "still in its early stages."

It's cool to see them addressing this. I'd read the announcement closely, though, as what it describes is geo experiments run through measurement partners, and nothing in it puts a holdout inside Ads Manager. If you want to know what your ads added, setting up the test is on you.

OpenAI put a few early results in the announcement. It chose which ones to show, which is what I'd do in their seat, so I read them the way I read any vendor case study.

WeightWatchers' attributed CPA on ChatGPT Ads came in 15.3% below its blended paid-search benchmark, measured by DV Rockerbox. Triple Whale found that 93% of Portland Leather's visitors from ChatGPT Ads were new.

I'd be happy with either number and I still wouldn't know what the ads added. Branded search was a top performer on our dashboard too, right up until we turned it off. And a new visitor may have been headed your way already.

The result from Dose, a wellness brand, gets closer. WorkMagic reported statistically significant lift, with 67% of incremental purchases coming from net-new customers. OpenAI's measurement post adds that WorkMagic's geo-lift study estimated 2.3x more incremental orders than last-click attribution captured. Which cuts against my own headline. If that estimate holds, last-click was catching less than half of the orders the ads drove.

I buy that it can happen. Somebody gets served an ad, keeps going in the conversation, shows up on your site later, and your analytics files the sale under Direct. I wrote last month about a dashboard undercounting like that: a weekly mix model that BlueAlpha (a company I advise) built for MUBI put Meta's incremental signups for one US week at roughly thirteen times what the dashboard reported. So a dashboard can miss in either direction, and if you budget off clicks alone you can end up cutting spend that was working.

I wouldn't carry the Dose number into my budget yet. The 2.3x is against last-click. It says nothing about how incremental orders compare with what Ads Manager reports once click and view-through windows are counted, and the platform's number is the one I was worried about. It's also a single advertiser, and your test can come out different, which is the reason to run one. And it was published with no test dates, no markets, no order counts and no confidence interval.

View-through is where I'd slow down. The same measurement post says you can add a one-day view-through window to your reporting, which counts conversions that follow an impression with no click. It also says advanced conversion optimization learns from eligible click-through and view-through conversions, and that the window settings "control attribution in reports; they do not change how campaigns optimize."

So I'd keep click-through and view-through in separate columns. I just wouldn't kid myself that it changes what delivery is learning from.

Here's what I'd ask OpenAI. When people are partway through a purchase, is the optimizer getting better at finding new buyers, or better at finding people who were about to buy?

OpenAI's own early analysis of global campaigns optimized for deeper-funnel outcomes found that 52.7% of eligible one-day view-through conversions happened within an hour of the matched ad impression. Maybe the ad worked that fast, or maybe that person had already decided and the ad happened to be there.

A purchase after an impression only proves the order things happened in. To know whether the ad changed anything, you need a group that never got it.

Boring but important: write down your attribution windows and do not touch them during a test. If you have to change one, mark the date, because otherwise reported conversions can jump from a settings change while the business does exactly what it did the month before.

For a new-customer campaign I'd suppress existing customers and active pipeline. OpenAI's help docs say audiences under 25,000 matched users can be used for exclusions, so even a small pipeline list works. (They also say to avoid custom audiences on campaigns targeting the EEA or Switzerland while ads personalization is unavailable there.) Expansion and reactivation can run as their own campaigns with their own math. Suppression keeps you from paying for people you already have. It tells you nothing about everybody else, so it doesn't replace the test.

Then I'd check the plumbing: pixel, Conversions API, deduplication between them, and the purchase or qualified-lead event itself. OpenAI has an Event Quality Score for this. I'd look at it, then reconcile what the platform says it received against my orders or CRM.

Before any data comes in, I'd agree with finance on what result keeps the budget and what result grows it. In ecommerce that's incremental margin after discounts, returns, fulfillment and media cost. In B2B it's qualified pipeline, with the close-rate and margin assumptions behind it written down.

At Affirm we ran holdouts: geo tests, ghost ads, before-and-afters where we shut a channel off in half the country and watched what happened to signups and volume. A geo test runs the ads in some markets and holds them off in comparable ones. History tells you roughly what the test markets would have done without the campaign, and the gap is your lift. Here I'd start by asking Haus, Measured or WorkMagic, the partners OpenAI named for geo experiments. You can also set one up yourself, because OpenAI's docs say you can target by region or market, though new product-feed campaigns only target at country level.

Measure the outcome in your own business data, across every path. A purchase that lands in Direct counts, and so does one that another platform is claiming. You're looking for a change in the total.

I'd ask the measurement partner five questions before launch:

A confidence interval won't save a bad comparison. And a low-volume B2B company may need more time or more deals than it has to get a useful geo test. I'd want to hear that before I've paid for the study.

If the math says you can't detect anything, I'd do what we did with branded search. Turn the ads off for a clean stretch, hold everything else as steady as you can, and watch the total. It's crude and seasonality can fool you. I'd still trust it more than the dashboard.

One page: spend, attributed acquisition, estimated incremental acquisition, how wide the uncertainty is, the decision.

If attributed CPA looks great and measured lift is weak, I'd go look at suppression, audience overlap, and whether the answer already recommends you. You may be paying a toll on buyers who were coming anyway.

When it goes the other way, ordinary attribution and a test that shows profitable lift, I'd find the paths my reporting missed, raise spend in steps, and keep checking that the added budget pays.

And if the result is inconclusive, write "inconclusive" in the report and figure out whether more volume, a longer test or a different design would settle it.

Retest after any big change in spend, audience or placement, because what a campaign returns at a small budget can change as you scale it.

What I want out of all this is a way to catch spend that's taking too much credit and spend my analytics never sees. If you're running ChatGPT Ads, put the test design next to the campaign plan before you approve the next increase. It's a lot easier to agree on the standard of proof before there's a great-looking dashboard in the room and people who want to defend it.