Ask a business whether AI is paying off and you'll usually hear one of two numbers: how many people have a licence, or how many hours a survey says everyone saved. Neither answers the question. A licence tells you what you bought, not what changed. A survey tells you how people feel about a tool, which is worth knowing, but it isn't a measurement. If you want to know how to measure AI ROI in a way you'd trust with a budget decision, you have to stop measuring the tool and start measuring the work.
Why the usual numbers mislead
Seat counts go up whether or not anything improves. Plenty of businesses pay for accounts that get opened twice a month. Usage dashboards are a step better, but they count prompts, not outcomes: a hundred prompts that produce a draft someone rewrites from scratch is a cost, not a saving.
Self-reported "hours saved" has a subtler problem. People are generous estimators when a tool feels fast, and nobody subtracts the twenty minutes spent checking the output or fixing the one confident mistake that went out. The feeling of speed is real. The saving might be smaller, or it might be larger than anyone thinks. Without a before, you can't tell.
Measure jobs, not tools
Start with the recurring jobs AI actually touches. Not "marketing" or "admin" — specific, repeatable pieces of work with a clear start and finish: answering a quote request, writing up a site visit, reconciling a supplier statement, turning a meeting into follow-up emails. Pick three. If you can't name three jobs AI is doing in your business, that's the finding, and it's more useful than any dashboard. We wrote about choosing those first jobs in how to use AI in your business like a power user.
For each job, capture four things:
Time per job. From the moment it lands to the moment it's done, including review. Not keystrokes — elapsed effort.
Errors and rework. How often does the output need fixing, and how often does a mistake reach a customer? This is the number most AI measurement skips, and it's the one that decides whether the saving is real.
Turnaround. How long the customer or colleague waits. Sometimes AI doesn't save much effort but halves the wait, and in a sales process that can matter more than the hours.
Volume. How many of these jobs happen in a week. A big saving on a job that happens twice a month is a small saving.
Get a before, even a rough one
The honest version of AI ROI needs a baseline, and most businesses skipped it because the tools arrived faster than anyone planned. You can still get one. Pick a job where some people use AI and some don't, or where the work can be done both ways for two weeks, and time both. Keep it crude: a shared sheet with a start time, an end time and a tick box for "needed fixing" is enough. Two weeks of rough numbers beats a year of impressions.
A saving that nobody spends isn't a return. It's slack. The question isn't only how much time AI freed up — it's what the business did with it.
Count what it costs, not just what it saves
The other side of the ledger is easy to leave blank. Subscriptions are the visible part. The less visible parts are the time someone spends reviewing output, the setup and upkeep of anything automated, and the occasional expensive mistake. If an automated job runs on a schedule, add the cost of noticing when it quietly stops — we covered why that matters in what an "AI employee" actually is. None of these should scare anyone off. They just belong in the sum.
Then ask where the time went
This is the step almost everyone misses. Suppose the numbers are good: quote requests that took forty minutes now take fifteen. Where did the other twenty-five minutes go? If the answer is "more quotes, answered faster", that's a return you can see in the pipeline. If it's "the same number of quotes and a slightly quieter afternoon", the tool may be worth keeping, but it isn't paying for itself in any way a business plan would recognise.
So the last measurement is qualitative, and it's a conversation rather than a spreadsheet: what are people doing now that they couldn't before? The best answers are specific — following up every lead the same day, finally getting to the backlog, taking on work that used to be turned down.
What we don't know yet
It's early, and the measurement methods are still settling everywhere, not just in small businesses. Some gains are hard to pin down: better first drafts, fewer blank-page mornings, a question answered in a minute instead of an afternoon of searching. We'd be lying if we said those fit neatly into a formula. And the tools move so fast that a measurement from spring can be out of date by autumn, which is one reason we think keeping up with AI news should be a short weekly habit rather than a flood. Measure the few jobs that matter, re-check them every quarter, and treat the numbers as directional rather than precise.
Where to start
This week, write down the three recurring jobs AI touches in your business, and for each one note roughly how long it takes and how often it needs fixing. Next month, do it again. That's the whole method, and it's more than most businesses have. If you'd rather have someone map how the work actually moves through your business, put one number on what it costs to run it that way, and tell you the three things worth fixing first, that's what our THINQ Diagnostic does.
