Why Fraud Engines Are Product Problems, Not Just ML Problems

The hardest decision in our fraud engine wasn't which model to use. It was what to do with a service centre we were fairly sure was misbehaving, in a city where it was the only one.

The data was clear enough. The score was high. But a score doesn't tell a coordinator what to do at 9 a.m. when a customer's air conditioner has stopped working in peak summer.

Block the centre, and an honest customer waits a week for a repair. Keep using it as normal, and money keeps leaking. Neither option is acceptable, and no model will invent a third one for you.

We designed one. A status that means “risky, but keep allocating”: the centre goes to the back of the queue whenever any alternative exists. If it is the only option, it still gets the job, but every one of those jobs goes straight into audit.

That single status taught me the core lesson of this project. Detection is maybe a fifth of a fraud engine. The rest is product. What happens after a flag, who owns it, how an entity gets out, and how the system learns it was wrong.

Here are six things I learned designing it.

1. A score is not a decision

Most fraud projects start with “let’s build a model that scores risk”. That’s the easy part.

The hard part is the sentence after the score. For every signal, we wrote down:

• what triggers it, and which entity it applies to;
• the severity;
• the immediate action, and whether it is automatic or manual;
• who owns it, and where it escalates;
• what outcome gets recorded at the end.

If you can't fill that row, you don't have a rule. You have an alert that someone will learn to ignore.

Design the response before you design the detection.

2. Fraudsters read your thresholds

When we analysed auto-approved repair claims, one pattern jumped out. Over a third of them landed on the same amount: a few rupees below the auto-approval limit.

That's not a model failure. It's a product leak. The moment you publish a limit, it becomes a target. Honest partners bill what the repair costs. Others bill what the system will wave through.

This showed up again later. While reviewing a new approval spec, I found proposed payout caps set at exactly the amount where fraud was already clustering. Nobody intended it. The cap just looked like a sensible round number.

The product fix is simple. For every limit you set, monitor how much volume sits just under it, partner by partner. A healthy partner's bills spread out. A gaming partner's bills pile up at the line.

3. Friction has to earn its place

Nearly 19 in 20 claims sent for manual review were approved anyway.

That number should worry any PM. It means the review queue was mostly a tax on honest partners, and it was eating the attention that the genuinely risky cases needed.

When we dug into why claims missed auto-approval, the top reason wasn't a risk check at all. It was a procedural gate around re-estimation. It slowed everyone down and caught almost nothing.

Friction is a budget. Spend it where signals converge, and remove it where it only creates queues.

4. Showing a number is not enforcing it

We launched an alert that showed reviewers a reference price for a spare part, based on past approvals. The idea was that reviewers would cut inflated asks.

Mostly, they didn't. When a partner asked above the reference, the large majority of those lines were still approved above it.

Two product gaps explained it. First, the reference was only information. Nothing asked the reviewer to act or explain. Second, many references were built from just one or two past cases. Reviewers sensed the number was shaky, and they were often right.

So the next version asks two questions. How confident is this reference? And at what confidence do we stop informing and start requiring a reason?

5. One signal is noise. Convergence is a case.

We ran several independent detectors: reused repair photos, amount clustering, unusual concentration in one sales channel, shared bank accounts.

Each on its own produced long lists. Many flags had innocent explanations.

The interesting partners were the ones that showed up on several lists at once. A small group was flagged by two or more unrelated mechanisms. That's where audits start.

Ranking by convergence is not a modelling trick. It's a prioritisation product for an audit team with limited hours.

6. The outcome loop is the product

A flag is a signal. It is not a verdict.

That sounds obvious, but most systems don't record what happened next. Without that, you can't tell a good rule from a noisy one. You can't defend a decision to a partner. And you can't reinstate anyone, because you never wrote down why they were restricted.

We built it around three pieces:

• A permanent flag log. Flags are never edited or deleted, even when status changes.
• An investigation outcome for every flag, including an explicit “false positive” marker.
• Named approvers for reversal. Restricting a partner is easy. Reinstating one needs Finance and Ops leadership to sign off.

Even overrides became data. Coordinators who change the system's assigned service centre now pick a reason. Most reasons are legitimate. The pattern of overrides is what we watch.

Where ML actually fits

I'm not arguing against models. They are excellent at finding patterns across hundreds of thousands of claims, and at ranking what to look at first.

But every choice that decides whether the engine works is a product choice:

Even the data is a product decision. Our partners entered the same amount sometimes with tax and sometimes without. Compare those to a price cap without normalising, and your “fraud” is just GST.

Before you build the model, answer these

1. 1.What happens in the first hour after a flag?
2. 2.Who can override it, and do they have to say why?
3. 3.How does a flagged entity get back to normal?
4. 4.What does a false positive cost the honest partner?
5. 5.Which limits in your process could someone learn to sit just under?
6. 6.How will you know a rule is wrong?

The best fraud engine isn't the one with the cleverest model. It's the one where every flag leads somewhere.