When not to use AI

We build AI systems for a living, so it may seem odd to write about when not to use them. But the most useful thing we can do for a client is often to say "you don't need a model for this". The projects that go wrong with AI are rarely the ones where the model wasn't clever enough. They are the ones where a model was used for a job that a few well-written rules would have done better.

Here is the test we use, and the reasoning behind it.

01#The two-question test

Before reaching for a model, ask two questions about the task:

• →Can the logic be written down as rules? Not "is it simple", but could a careful person write a page that covers how the decision is made, with conditions and outcomes?
• →Does someone need to explain or defend the result? To a customer, an auditor, a regulator or a manager, now or later.

If the answer to both is yes, use rules. If the logic can't be written down because the input is too varied, AI may be the right tool. If the result must be explained but the logic can't be written down, you need a careful design with a person in the loop, and possibly a different process altogether.

02#Why rules win when they can

They are explainable by construction

When a rule rejects an expense claim, it can say exactly why: the amount exceeded the limit for that category. Anyone can read the rule and agree or disagree with it. A model's decision can be accompanied by an explanation, but that explanation is itself generated, and it is not guaranteed to be the real reason. For decisions that affect people's money, access or rights, that difference matters.

They behave the same way every time

The same input gives the same output, today and next year. You can write tests that prove a rule does what it should and keep proving it after every change. Models are improving in consistency, but they can still respond differently to inputs that look identical to a person, and a model update from your provider can shift behaviour without any change on your side.

They are cheap and fast

A rule runs in microseconds for effectively nothing. A model call takes noticeably longer and costs money each time. For a task that runs a million times a month, that difference is significant, and it never goes away.

They fail loudly

When a rule meets an input it wasn't written for, it can stop and say so. When a model meets an unfamiliar input it usually produces a confident answer anyway. Silent, plausible mistakes are the hardest kind to catch.

03#Where AI genuinely helps

None of this is an argument against AI. It is an argument for using it where rules break down, which is mostly at the edges of a process, where unstructured material comes in.

• →Reading free text. Customer emails, support tickets, survey responses. Rules for these turn into an endless list of keywords and exceptions. A model can classify, summarise and extract the useful parts far more reliably.
• →Reading documents. Invoices in a hundred layouts, contracts, forms that arrive as scans. Pulling structured fields out of these is a strong use of AI.
• →Drafting for a person to finish. A first reply to a customer, a summary of a long thread, a draft job description. The person stays responsible for what is sent, and the model saves them the blank page.
• →Searching by meaning. Finding the relevant policy, past ticket or document when the words don't match exactly.

What these share is that the input is messy and the cost of a slightly imperfect output is low, or a person checks it before it matters.

04#The combination that usually works best

Many good designs use both. A model does the reading, and rules do the deciding.

Take an incoming invoice. A model reads the document and extracts the supplier, the amount, the order number and the line items, with a confidence level for each. Then ordinary rules take over: does the order exist, does the amount match within the agreed tolerance, is the supplier approved? If everything checks out, the invoice is approved automatically. If any field has low confidence, or any rule fails, it goes to a person with a clear explanation of what didn't match.

This gives you the flexibility of AI where the input is unpredictable and the reliability of rules where the decision has to be defended. Each part is doing the job it is best at. When an auditor asks why an invoice was paid, the answer is a rule, not a probability.

05#Warning signs you are reaching for AI too early

• →You can't say what a correct answer looks like. If you can't write test cases, you can't tell whether the model is working. Define success first.
• →The process isn't agreed yet. If two experienced people would make different calls, a model will learn the inconsistency, not resolve it.
• →Volumes are tiny. Ten items a week may not justify any automation, intelligent or otherwise.
• →Nobody will review its output. For anything consequential, plan who checks what the model produces, and how often, before you build it.
• →The main reason is that it sounds modern. A rules engine is less exciting in a board update, and often the better answer.

The goal is working software that people trust. Sometimes that needs a model. Often it needs a clear rule, a good exception process, and a model only at the one step where the input is genuinely messy.