Risk Should Decay: Designing Fraud Scores That Forgive

In most fraud systems, a flag is a life sentence.

A partner flagged once in March is still “risky” in December, after nine clean months and thousands of good jobs. Nobody decided that. The score just never had a way to go down.

That isn't caution. It's a design bug. And fixing it turned out to be one of the most interesting problems in the fraud engine I designed.

Why scores that only go up stop working

When we looked at our own registry of flagged entities, roughly twice as many had ever been flagged as were currently active. That gap is the whole argument. A list that only grows soon describes history, not risk.

Permanent scores fail in four ways:

• The watchlist stops meaning anything. Given enough time, every long-standing partner collects a flag or two. When everyone is risky, nobody is.
• Good behaviour earns nothing. If nothing a partner does can lower their score, why would they change?
• Attention goes to old news. Reviewers spend time on last year's flag instead of this week's pattern.
• There's no way back. In a service network, you often can't just drop a partner. Sometimes they're the only one in town. You need a path from “restricted” to “trusted” that isn't a manual favour.

But naive decay is a gift to fraudsters

The obvious fix is to let scores fade over time. That's also the dangerous one.

If risk simply fades with the calendar, the playbook for a bad actor is easy: pause, wait it out, start again. Decay has to be designed so that waiting earns nothing and only clean behaviour earns trust back.

Here is how we built it.

1. Separate memory from judgement

The first rule: flags are never deleted. Every signal lives forever in an append-only log, with its evidence and outcome.

What decays is only the weight that old flag carries in today's score. The system remembers everything. It just stops treating old, unrepeated events as current.

This also keeps the system auditable. You can always answer the question, “Why did this partner's score change on this date?”

2. Different signals decay at different speeds

One clock for everything is wrong. A missing form field and a confirmed fake invoice are not the same kind of event.

The last row matters more than it looks. A cleared flag shouldn't just stop hurting the partner. It should hurt the rule's credibility, so noisy rules get tuned.

3. Earn it, don't wait for it

This is the part I care about most. Decay should run on clean activity, not calendar days.

A partner who did zero jobs for six months hasn't proved anything. A partner who did 400 clean, audited repairs has.

So the half-life of a flag is measured mainly in clean jobs since the flag, with time as a slower secondary clock. A simple version of the score looks like this:

In plain words: each flag starts at its weight (w). It halves every h clean jobs (n counts the clean jobs since that flag). r goes up if the pattern has come back before. The total never drops below a floor (F) set by any confirmed outcome.

4. Remember the relapse

If a pattern returns after it decayed, it should come back stronger than it started.

That single rule kills the “pause and wait” strategy. Gaming the decay now costs more than never offending at all.

5. Floors, ceilings and convergence

• Floors. Confirmed fraud sets a minimum score that time alone never removes. Getting below it requires a deliberate decision by named owners, with a reason on record.
• Ceilings. Many small, low-severity flags shouldn't stack up to the same score as one serious finding. Low-grade signals are capped unless they converge with something independent.
• Convergence. Two unrelated detectors pointing at the same partner count for more than the sum of their parts. That's where audits should start.

6. Show the trajectory, not just the number

A score of 42 tells a coordinator very little. “42, down from 70, falling for three months” tells them a lot.

So the engine keeps a score history, and the screen shows direction: improving, stable or worsening. It also shows which flags make up today's number. That makes the score explainable to the reviewer, to leadership, and, when the time comes, to the partner.

A worked example

Three partners, three very different stories. A permanent score would have treated A and C the same as B forever.

What I'm still working through

Decay is the most distinctive idea in this design, and also the one I'm most careful with. Open questions:

• Back-testing. Would this model have let any known bad actor back in too early? We keep re-running it against past cases as the half-lives are tuned.
• Who sets half-lives? These are business choices, not data-science ones. They need owners and a review cadence.
• Never-decay signals. Some signals, like customer payouts landing in a partner's own bank account, may deserve no decay at all.
• Transparency. Telling partners how trust is earned back motivates good behaviour. Telling them the exact maths invites gaming. The line sits somewhere in between.

The principle

Good fraud systems remember everything and forgive on evidence.

Memory protects the business. Forgiveness keeps the network healthy, keeps the watchlist sharp, and gives every partner a reason to be better tomorrow than they were yesterday.