How to check an agency case study for fiction

Agency case studies are a literary form. They have a hero, a darkness, a turning point and a number that glows. Some of them are true enough to hire on. Many are true in the way a trailer is true: the shots happened, the plot did not. A few are borrowed — another team’s work, another office’s win, a client’s own media team wearing the shop’s logo in the PDF.

Australian marketers do not need to become forensic accountants. They need a habit of asking for the bits a case study always leaves out: the baseline, the offer, the other agencies in the room, the window, the people who would still take a call. This desk will not invent a ROAS to make a reel look honest. You should not accept one that cannot be explained without the reel.

The habit belongs in working together because you will use it in a pitch and again at month six. It is also how you read this site. The editorial standards page is why we do not publish league tables of unnamed clients and miraculous returns.

The anatomy of a useful case study

A useful one names the constraint. A category that cannot make a health claim. A retailer with franchisees. A brand that had no creative. A tracking setup that was a mess. Without a constraint, you are reading an advertisement for ease.

It names the work, in verbs. What they made, what they changed, what they stopped. “Full-funnel excellence” is not a verb.

It names a number that could have gone the other way, and the way it was counted. Spend. A volume. A cost. A time period. A comparison to a previous period or to a holdout. It does not have to be a return figure. In many Australian categories a clean return figure is a fantasy anyway.

It names the client in a way you can verify, or it is honest about why it cannot — a government program, a conflict, a brand still in market. Anonymity is sometimes real. It is also how a shop recycles one win across six pitches.

If those parts are present, you can have a conversation. If they are not, you are being recruited.

Missing baselines are the most common fiction

“We lifted conversions 140 per cent” is not a result. It is a reveal. From what. Over how long. Against what seasonality. With what spend. With what change to the offer, the site, the pixel, the attribution window.

Ask for the same number, same definition, for the period before they arrived. Ask who pulled it — the shop, the client, a platform UI. Ask whether the conversion event was redefined in the first month. A new event is a new story. It may be a better story. It is not a lift.

Ask what happened to the number when they left, if they have left. A case study that ends on the month of the award is a campaign. A practice should survive a quarter without a trophy.

This is not cynicism about good work. Good work can bear a baseline. Shops that cannot produce one are usually hiding a start point that was broken, tiny, or measured differently. Sometimes the honest sentence is “the tracking was unusable, so we fixed the tracking”. That sentence is hireable. The glowing percentage is not.

Borrowed credit, and the other people in the room

Australian accounts are crowded. A creative shop, a media shop, an in-house team, a media owner, a founder who changed the price. Case studies hate crowds. They prefer a single protagonist.

Ask who else was paid. Ask who owned the offer. Ask who wrote the landing page. Ask whether the “performance” win sat on the back of a brand campaign someone else made, or a retail event that would have moved volume with a mediocre ad. A shop that can say “we did this part” is more trustworthy than a shop that did everything in the PDF and nothing in the meeting.

Ask who sat on the account. The case study’s strategist is often a pitch resource. The people you will get are the people available in 2026, not the people who did a piece of work in 2021. How to choose a digital marketing agency in Australia is mostly this question in longer form.

Ask whether the client will take a call. Not a written reference the shop typed. A call. You will not always get one. The way the shop talks about the possibility is data. “They’re busy” forever is a tell. “Here’s the marketing lead, they know we’re pitching” is a relationship.

Holding-company case studies need a second look. Work from an office in another country, or from a sister agency, will appear when the local team is thin. Ask where the humans sat. A reel from London is not a Sydney pod.

Numbers that cannot be reproduced

Platform screenshots are not third-party verification. They are the platform talking about itself, in a window the operator chose, on a day they chose to capture.

Ask for the date range and the attribution setting as if you were going to open the account. Then, if you are serious, ask to open a redacted view. Shops that know the work can do this. Shops that inherited a PDF cannot.

Beware of precision. A return to two decimal places on a small spend is comedy. A national brand figure with no spend attached is a press release. A “cost per acquisition” that does not say what was acquired — a sale, a lead, a post-click, a view-through — is a noun looking for a job.

Beware of averages that hide the offer. A catalogue with one hero SKU on a deep discount will produce a beautiful average. The shop may have been right to lean on it. You need to know, because you may not have that SKU.

Do not ask them to invent a number they do not have. Ask them to drop the number they cannot defend. Silence is more useful than a second decimal.

Awards, reels, and other adjacent genres

Awards are useful as a record of craft and as a record of what the industry wanted to clap for that year. They are not a measurement system. Entries are written to win a room. The same incentives that produce a case-study trailer produce an awards film.

Reels are recruiting. Watch them for taste if taste is the job. Do not watch them for evidence of incrementality.

Testimonials on a website are a third genre. The Australian Consumer Law still cares whether a testimonial misleads. You should care whether the person still works at the client, and whether they are describing the same product you are about to buy. A testimonial about a brand film is not a reference for a search retainer.

Guest posts and sponsored mentions are a fourth. If a shop is buying praise, it should be labelled. Our own write for us page is the version of that rule we will live with. A case study that only exists as native content on a friendly site is an ad.

A twenty-minute interrogation you can run in the pitch

Minute one to five: get them off the PDF. “Tell it without the slides.” Listen for constraints and verbs.

Minute five to ten: baseline and window. Same number, earlier period, same definition. Who pulled it.

Minute ten to fifteen: the crowd. Other agencies, in-house, the offer, the site. What they will admit they did not do.

Minute fifteen to twenty: the people. Who is still employed. Who will be in your weekly. Whether the client will take a call.

If you have time left, ask what failed. A shop with no failures has not been in enough rooms, or has edited them out.

Price this habit against what Australian agencies actually charge. A cheap retainer that can only show tiles and a borrowed win is not cheaper. It is an unexamined risk with a monthly.

The desk’s view

Treat a case study as a claim. Ask for a baseline, a crowd and a human who was there. Hire shops that can say what they did not do. Leave shops whose best number cannot survive a previous year. The work you want is usually quieter than the PDF. That is not a reason to be bored. It is a reason to be specific.

If a shop believes we have been unfair to a public, checkable case, write to the desk. Bring the baseline. We will not print a glowing percentage to keep a pitch warm.