Skip to content
— Attribution & measurement12 min read

Fourteen questions your agency should be able to answer.

Not questions to ask before you hire someone. Questions to ask of the people already spending your money, about your business rather than their dashboards.

SN
Founder, ADSRUNNER

What does your agency actually know about your business?

Not about your campaigns. Your agency almost certainly knows your campaigns well: the cost per click, the ROAS in each ad set, the search terms that ran up spend last Tuesday. I mean the business. Whether last month made money. Whether the customers you paid for were new or were people who would have come back anyway. Whether the drop in March was demand or a broken tag.

Most agencies cannot answer those questions, and it is rarely because the people are not smart. It is because of where they are standing. They can see the ad platforms. The platforms are built to answer one question very well: did the ads get credit? Every other question the business has sits in systems the agency never connected, the store, the analytics account, the customer list, and so it goes unasked. The monthly report arrives full of numbers and quiet on the only thing you wanted to know.

Below are fourteen questions a serious owner already has. They are grouped into money, truth, and growth. For each one I describe what a real answer requires and what an evasive one tends to sound like. You can score your current setup against them in about ten minutes with the agency accountability scorecard.

This is a different list from the questions to ask before hiring anyone. Those test whether an agency will be good. These test whether the setup you already have can see your business. If you are still choosing, start with questions to ask before hiring an ads agency.

Why the questions go unanswered

Think of the numbers around your marketing as four ledgers. The first is what each ad platform claims it caused. The second is what your analytics tool recorded. The third is what tracking you own, on your own domain, saw. The fourth is what your store or billing system actually booked, after refunds.

They never agree. They count different things, over different windows, with different incentives, and each platform grades its own work. A setup that reads only the first ledger can be very busy and very confident and still not know whether the business is growing. That is the visibility gap. Every question below closes part of it, and every one needs at least two of the four ledgers read side by side.

QuestionWhat a real answer needs
Was last month profitable after product cost and returns?Store orders, product cost, refunds, and spend
Which products carry the account, and which consume it?Product-level results across the ad accounts
What did a new customer really cost?Your customer list, matched against purchases
How much revenue came from new customers?The same match, split by channel and source
Four platforms claim a sale. Which earned it?Your own tracking and several attribution models
Why do platform, analytics, and store disagree?All four ledgers on shared definitions
Did sales drop, or did tracking break?Site behavior next to store orders
Is the number on the screen from today?A freshness check on every source
Where is money leaking today?Daily checks ranked by dollars
Which campaign could take more budget?Performance against the account’s own baseline
What changed overnight, and is it worth money?Same-weekday history and a dollar bar
Is growth new demand or existing customers?New versus returning, by channel
Is the budget on the right platform?Platforms compared on one set of definitions
Which creative is wearing out?Creative results read before revenue falls
The fourteen questions, and the minimum each one needs to be answered honestly.

Money and profit

1. Was last month actually profitable, once product cost, shipping, and returns are in?

Ad platforms report revenue they can see, with none of the costs underneath it. A 4x ROAS on a product with a 20% margin after shipping and returns is a loss. The platform will still report it as a strong month, and the call about it will still sound like a win.

A real answer starts from the store: orders, net revenue, product cost, refunds, and the spend that bought them, set side by side. It sounds like a number with a margin attached. An evasive answer sounds like a ROAS figure followed by the word healthy. If nobody on the account knows your contribution margin, nobody knows what ROAS you can afford.

2. Which products are carrying the account, and which are quietly consuming it?

Most reporting stops at the campaign. The money is usually won or lost at the product. A Shopping or Performance Max campaign can look fine on average while three products earn everything and forty others spend steadily without a sale. The average is the disguise.

A real answer names products: which ones pay for themselves after cost, which ones spend without return, and what the plan is for each. An evasive answer defends the campaign. The follow-up question that separates them is simple. Show me the ten products with the most spend and the least margin.

3. What did it really cost to win a new customer?

Blended ROAS counts a loyal customer who was coming back anyway as if the ads had won them. For an established brand, a large share of attributed revenue can be exactly that. The account looks efficient because it is being credited for loyalty the business already paid for.

A real answer uses your customer list. Identifiers are hashed, matched against purchases, and every sale is classified as new, returning, or not yet identifiable. New-customer CAC then becomes a number instead of a guess. It is often the least comfortable figure a brand sees all year, and the most useful. An evasive answer quotes the platform new-customer column without saying the platform can only call someone new if it has not seen them before.

4. How much of this month’s revenue came from new customers?

Without the list, every sale looks like a win for the ads. With it, revenue splits into new and returning, by channel and by source. A month that looked like growth can turn out to be retention with a media budget attached. That is not a failure. It is a different decision, and you cannot make it if the split does not exist.

A real answer is a split with a date on it: how far back the customer history goes, and how many purchases could not be matched. An evasive answer is a single revenue figure and a growth percentage.

Truth and measurement

5. Four platforms claim the same sale. Which one earned it?

Each platform records the journey from where it stands. Meta sees its own impressions and clicks. Google sees its own. Each applies its own window and its own model, and each will happily claim a purchase another platform also claims. Add the claims up and they routinely exceed what the store booked.

A real answer is not a single model presented as truth. It is the journey recorded on tracking you own, then several ways of giving credit shown side by side: last click, first click, linear, time decay, position based, and data driven. Where the models agree on a channel, the confidence is earned. Where the credit swings depending on the model, the allocation is a matter of opinion, and that is where an incrementality test earns its cost. An evasive answer picks the model that makes the recommended channel look best.

6. Why do the ad platform, the analytics tool, and the store disagree?

Because they are counting different things. Platforms count conversions on the day of the click, inside their windows, often gross of tax and refunds. Analytics counts sessions and applies its own credit rules, and it takes a day or two to settle. The store counts money that actually cleared. None of them is broken. They answer different questions.

A real answer shows the four ledgers in one view and explains the gap between them in plain language. It sounds like: the platforms claim about this much more than the store booked, which is normal double counting, and here is why this month’s gap is or is not normal. An evasive answer blends them into one figure nobody can audit. I wrote the long version of this in why Meta, Google, GA4, and Shopify report different revenue.

7. Did sales drop, or did the tracking break?

Demand rarely falls off a cliff overnight. Tracking does, often. A site release removes a tag, a consent banner changes its defaults, someone edits a conversion action. If purchases in the ad platform collapse while page views carry on as normal and the store keeps taking orders, the business did not change. The measurement did.

A real answer checks the store before touching the account. An evasive answer starts changing bids. The difference matters because every change made against broken measurement teaches the bidding algorithm something false, and you pay for that lesson twice. The full sequence is in performance dropped: a systematic diagnostic method.

8. Is the number on the screen from today?

A stale data feed looks exactly like a calm account. If yesterday’s spend never arrived, the dashboard shows a quiet day, and a quiet day invites no questions. Decisions made on a missing day are worse than no decision, because they feel informed.

A real answer knows when each source last refreshed and says so. An evasive answer has never thought about it. Ask when the numbers in the last report were pulled, and whether any source was behind.

Waste and growth

9. Where is money going out with nothing coming back, today?

Search terms with spend and no conversions. Placements that never return. Landing pages that receive traffic and convert nobody. Ads that are disapproved while the budget still tries to spend. Budget meant to work that is sitting still. None of this is dramatic, which is exactly why it survives between monthly reviews.

A real answer is a ranked list, ordered by dollars rather than by how alarming the percentage looks. A 300% swing on a $40 campaign is not an emergency. A 12% leak on a $40,000 campaign is. An evasive answer is a list of optimizations made, with no sense of which of them mattered.

10. Which campaign could take more budget and still pay back?

Accounts that have plateaued are often not out of market. They are out of attention. Review time goes to whatever broke, and the campaign doing the job with room to grow goes unmentioned because nothing is wrong with it.

A real answer compares performance with the account’s own recent baseline and points at campaigns that are holding their return while limited by budget. It comes with a test: how much more, for how long, and what result would stop it. An evasive answer asks for a bigger budget across the board.

11. What changed overnight, and is it worth real money?

Everything changes overnight a little. The useful question is which changes clear a bar worth a person’s attention. Comparing today with the same weekday in previous weeks removes most of the noise, because Mondays and Saturdays behave differently in almost every account.

A real answer filters by dollars and by the account’s own history, not by a generic benchmark from another industry. You hear about what matters and not about every flicker. An evasive answer is either silence until the monthly call or a flood of alerts nobody reads.

12. Is growth coming from new demand, or from people who already buy?

Scaling spend into your own customers feels like growth and prices like a tax. Retargeting and brand search look brilliant in the platform because they are placed in front of people already on their way to buy. Once new and returning are split by channel, you can see whether extra budget is buying reach or renting loyalty.

A real answer shows new-customer ROAS beside blended ROAS for each channel. The gap between the two is the part of the performance that belongs to your brand rather than to the ads. An evasive answer defends retargeting on its ROAS alone.

13. Is the budget on the right platform?

Compared fairly, with the same definitions, the channel that looks worst in its own dashboard is sometimes the one that should grow. Platforms with generous view-through windows look strong in their own reports. Platforms that mostly close the sale look strong in last click. Neither view is a fair comparison.

A real answer compares Google, Meta, Microsoft, TikTok, and Pinterest on one set of definitions, next to what the store booked, and names a dollar amount to move with a reason. The point is not to crown a winner. The point is a dollar moved on evidence. An evasive answer says every channel plays a role.

14. Which creative is wearing out before the revenue shows it?

Creative tires on a curve. Frequency climbs, click-through softens, cost per result drifts up, and all of it is visible before the month is lost. By the time revenue shows the decline, the account has already paid for it.

A real answer reads creative performance across the ads that are running and changes the testing plan while the winners are still winning. An evasive answer refreshes creative on a calendar, or when someone complains.

How to use the list without starting a fight

Pick three questions, not fourteen. Choose the ones closest to the decision you are about to make: a budget increase, a channel shift, a quarter that felt worse than the report said. Ask them in writing, and ask for the answer from your own data rather than from a benchmark.

Then listen to the shape of the answer. A good answer is specific, carries a date and a source, and admits what it cannot see. Uncertainty stated precisely is a sign of competence. A vague answer delivered with confidence is the opposite.

If the honest answer to most of them is we cannot see that, the problem is usually not effort. It is that the setup was never built to connect the ledgers. That is fixable, and the fix is mostly measurement, not media.

What the list is not

These questions do not require any particular software, including ours. A capable analyst with access to the store, analytics, the ad accounts, and a clean customer export can answer most of them by hand. It is slow, and it tends to happen once a quarter instead of every day, but it is possible.

We built the ADSRUNNER platform because doing it by hand for every client every day was not. It reads the ad accounts, analytics, the store, our own first-party tracking, and the customer list side by side, checks them daily, and ranks what it finds by the money at stake. The answers still go to a person. The platform does not spend the money, and every change that affects a client goes through an approval they control.

Nor is the list a test of intelligence. Some of the sharpest media buyers I know could not answer half of it, because nobody ever gave them the store. Ask the questions to find out what your setup can see. Then decide whether that is enough for the money you are spending.

— Common questions
What questions should I ask my current ad agency?

Ask questions about your business rather than your campaigns. Was last month profitable after product cost and returns? What did a new customer cost, measured against your own customer list? Which platform actually earned a sale that several platforms claim? Was the last drop a demand change or a tracking break? Where is money leaking today, ranked by dollars? A setup that can answer these from your own data can see your business.

Why does my agency report a different revenue number from Shopify?

Ad platforms count conversions they believe they caused, on their own attribution windows, usually on the day of the click and often before refunds. Several platforms can claim the same order. Shopify counts money that actually settled. The difference is normal. What matters is that your agency can explain the gap and anchors decisions to what the store booked.

How do I know if my agency is measuring new customers correctly?

Ask how a customer is classified as new. If the answer relies only on the ad platform or a pixel, it will overcount new customers, because anyone the pixel has not seen before reads as new. A more reliable method matches purchases against your own customer list with identifiers hashed, and states how far back that history goes.

Is it reasonable to expect an agency to answer all fourteen questions?

For a brand spending seriously on paid media, yes, at least most of them. Some need data the agency must be given, such as product cost and the customer list. If an agency has never asked you for those, that is itself an answer.

Written by , founder, adsrunner. If this resonated and you want to apply it to your own account, you can book a strategy call or run a free audit.

How we research, source figures, and handle corrections: editorial policy.

— What we learn

Got value from this one?

The next one lands in your inbox. Account-level analysis written the way we brief our own operators. Unsubscribe in one click.

— More to read

Continue with these.

— Attribution

Why Meta, Google, GA4, and Shopify report different revenue.

Meta says one number, Google another, GA4 a third, and Shopify books something smaller than all of them combined. The reasons are mechanical, and once you know them, the gap between the four stops being a problem and starts being a diagnostic.

Read the post
— Measurement

What a new customer really costs.

Blended ROAS makes every established brand look efficient, because it credits the ads for customers the business already won. Separating new from returning, against your own customer list, is how acquisition cost becomes a number instead of a guess.

Read the post
— Agency craft

Questions to ask before hiring an ads agency.

A good question has a property almost nobody checks for: a weak agency cannot answer it well by being articulate. That single test disqualifies most of the questions in circulation — "what is your process," "can you show me case studies" — and it produces a much shorter list where each question comes with the answer that should reassure you and the answer that should end the meeting.

Read the post
— Measurement

MER vs ROAS: the argument is a decoy.

Every version of this article tells you to stop trusting ROAS and start trusting MER. That advice is not wrong so much as beside the point: MER and blended ROAS are the same economics inverted, and neither is safer than the revenue figure you feed it. Here is the ladder that ranks your revenue sources by trustworthiness, the arithmetic that turns one week of data into three contradictory decisions, and how to set the floor from your own P&L.

Read the post
— Agency craft

Why we left automatic execution switched off.

The ability to carry out changes on ad accounts automatically is built and tested in our platform. It is switched off. Switching it on is a decision about client money, and a system being capable of something is not a reason to let it.

Read the post

Want this kind of thinking on your account?

Book a strategy call. We'll review your account and show you specifically what we'd do differently.