Anubhav Lahoti
AI in practice
AIRisk

Risk & QA Analysis Copilot

An LLM-assisted analysis system for the Risk & QA team. It runs ten recurring fraud and cost-control analyses, plus one-off studies, in minutes instead of days.

What it does

A fraud and cost-control analysis system covering ten recurring analyses: service-centre overcharging, spare-part pricing, auto-approval rule performance, duplicate-image reuse, bank payout fraud, KYC changes, OTP bypass, service-centre allocation, and the cost impact of rule changes.

The analyst brings the business question, the data exports and the domain rules (what a zero amount means, which date to split on, which channels to exclude). The system turns those into analysis logic, runs it, tests the conclusion against alternative cuts, and returns findings, recommendations and documents. People decide what gets acted on.

What the LLM does, and what it doesn’t

The LLM does the thinking around the numbers, not the arithmetic.

  • Analysis design: turning a loosely worded question into a method, such as a like-for-like comparison or splitting a cost change into routing versus pricing.
  • Data work: reading unfamiliar files and spotting traps, such as data that only covers some cities, amounts entered both with and without GST, or default placeholder prices.
  • Code: writing and modifying the Python scripts that do the calculations.
  • Interpretation: checking whether a result is real. One apparent weekly improvement turned out to be a category-mix effect.
  • Communication: CEO summaries, PRD reviews and KPI strategy notes.
  • Calculation stays in Python. Every number comes from deterministic code, so results are reproducible and auditable.

Why it matters

  • Speed: follow-up questions like “remove reopened requests” or “exclude this channel” are a rerun, not a new project.
  • Depth: every headline is tested under several cuts before it reaches a decision-maker. That is how we found one region’s saving came entirely from job routing, while another’s cost rise was pricing, not routing.
  • Joined-up findings: results are cross-checked across modules. Most of the service centres flagged for amount clustering also showed up in duplicate-image fraud.
  • Memory: every run is compared with the last, tracking new and resolved offenders and KPI progress.

How it’s built

  • Input: drop a data export in a folder and run one command. The analysis type is detected from the columns.
  • Analysis: one Python script per analysis type, each with fixed, documented rules.
  • Output: a dated Word report with a fixed structure (summary, breakdowns, findings, actions). Reports are never overwritten.
  • Memory: each run saves a JSON snapshot. The next report includes “what changed since last run”, and a registry of known fraud patterns lets the system surface genuinely new ones.
  • KPIs: a scorecard tracks auto-allocation, auto-approval and new fraud patterns found per cycle. Python computes them; the LLM writes the strategy note from a compact summary rather than raw data, to keep cost low.

Why this approach

Custom ML fraud model
No labelled fraud data to train on, and the output is a score that is hard to explain or act on. Explainable signals such as amount clustering and image reuse are auditable.
Off-the-shelf fraud tool
Generic rules that don’t know business-specific signals: pricing ceilings, which field holds the real outcome, own versus partner service-centre routing.
LLM reading raw data
Expensive, unreliable on large files, and the numbers can’t be reproduced. Here code computes and the LLM reasons, so results stay exact.