Adjust claims desk · Quillmark Mutual
World clock Mon 17:00:00 CET paused · 60×

How it works

The design in ten minutes. The full design is one click further. docs/architecture.md

Adjust: the design in ten minutes

Adjust is a claims triage operator for a motor and property insurer. It reads the claims book as it changes (new claims, documents, payments, the contact log), finds what is wrong on the files, does what a written policy allows, puts the rest in front of a person with the evidence, and scores itself against an answer key. The full design has the data model, every check the controller makes and the replay mechanics. The running copy works on a synthetic world: Quillmark Mutual, an invented insurer in the invented country Estmark.

1. The problem, in one Monday

Storm Ivarra crosses four of Estmark's seven regions on a Saturday and a Sunday. On Monday at 09:00, 1,434 open claims have no adjuster (seed 23); 1,367 of them were reported after the storm began, 933 caused by it. Mixed into that pile:

  • the same flooded kitchen reported twice, once by phone and once online, one of them under a neighbouring peril;
  • a garage invoice dated a week before the crash, and a photo of the damage taken eight days before the loss;
  • three households that do not know each other, paid into one bank account;
  • a car whose repair estimate is ninety per cent of its value, with nobody calling it a total loss;
  • a payment above the requesting adjuster's authority, waiting for a signature since Friday;
  • files whose adjuster left last month and that have been quiet for six weeks, and recoveries nobody opened.

A desk of people finds each of these, eventually, by reading files one at a time. Adjust reads all of them as they land, and spends a person's time only where the policy says a person decides.

2. What a person sees

The main screen is the Desk, a claim-file workbench in three parts:

┌ In-tray ───────────┬ Claim file CL-26-133530 ───────────────┬ Why it is on the desk ──────────┐
│ needs a person     │ [Facts] [Documents] [Money] [People]   │ 09:00  RING INDICATOR   T2      │
│ fraud and money    │ the claim, the policy, the file history│ what it saw, the evidence rows,  │
│ quality of the file│ documents dated against the loss       │ the definition and version,      │
│ routine assignment │ estimate against the benchmark         │ the call, who approved, undo     │
└────────────────────┴────────────────────────────────────────┴──────────────────────────────────┘
  • The in-tray is every claim the operator touched, in the operator's order: a decision waiting for a person first, then fraud and money signals (a ring, a photo before the loss, dates that do not fit, a duplicate, two invoices, a payment above authority, a total loss), then the quality of the file (an estimate above the benchmark, a missing police report, a missed recovery, a stalled file), then routine assignment. Each card carries its stamps and the operator's last receipt.
  • The claim file is the claim as it stood at the clock: facts and policy, documents with their dates said against the loss ("8 days before the loss"), money with the estimate drawn against the benchmark's p50 and p90, and the people (claimant, adjuster, repairer, contact log). Fields a detector pointed at are underlined in red; what the operator wrote is in blue.
  • The evidence panel has one receipt per decision on that file: what it saw, the evidence rows, the definition it ran on and its version, rules or model, the call it made, the tier, who approved, and an undo button where the action can be undone.

Play runs the desk day on a fast clock (10x to 120x; at 120x the eight hours take four minutes). The in-tray fills, the counters climb, the receipts land on the tape, and with "follow the operator" on, the file in the middle switches to whatever the operator has just worked on. Two jumps start the clock at the moments worth watching: 10:00, when the weekend's claims pass the one-hour line and the rulebook assigns them, and 13:00, when invoices and payments arrive.

The other screens: the Ledger (every row, filterable, exportable), Approvals (the tier 2 queue), Accuracy (precision and recall per incident kind against the answer key), Definitions (the claims terms with owners, versions, SQL and tests), Ask (questions answered only through those definitions), Sources (which system each fact comes from) and the Known gaps.

3. What runs

flowchart LR
    BOOK["Claims book: claims, documents, payments, contacts"] --> DEF["Definitions: versioned YAML with SQL and tests"]
    DEF --> DET["Detectors: realtime, hourly, daily at 09:00"]
    DET --> PROP["Proposal: rules, or a model where there is a real choice"]
    PROP --> CTRL["Controller: policy.yaml, budgets, cooldowns, never-list"]
    CTRL -->|tier 0, 1| TOOLS["Tools on the claims system's calls"]
    CTRL -->|tier 2| Q["Approvals: a person decides"]
    CTRL -->|tier 3, conditions unmet| NO["Declined, with the reason"]
    TOOLS --> LED["Append-only ledger"]
    Q --> LED
    NO --> LED
    LED --> REC["Accuracy against the answer key"]
  • Definitions say what is true. A stalled file, a duplicate, a clean claim, a total loss, an open claim: each is a YAML file with an owner, a version and its history, SQL and tests. Where two departments mean different things (an open claim to Claims Operations is a file not closed; to Claims Finance it is a file with money still to pay), both meanings are kept and Ask shows both side by side.
  • Detectors evaluate those definitions as of the clock and open one incident per problem. Seven run on every tick (unassigned claims, duplicates, missing police reports, dates that do not fit, two invoices, estimates above the benchmark, payments above authority), three hourly (rings, total losses, photos before the loss), two once a day at 09:00 (stalled files, missed recoveries).
  • The proposal picks an action from the runbook for that kind. Rules decide where the runbook leaves no choice; a model is asked only where it does, and it can only propose.
  • The controller is separate from the model and re-checks every proposal against adjust/agent/policy.yaml. Anything not in that file is tier 3.
  • Tools make the change through the call a claims system exposes (PUT /claims/{id}/assignment, POST /siu/referrals), each with a before and after snapshot and an undo.
  • The ledger keeps every step. Nothing in it is edited; a decision, an approval or an undo is a new row that points at the one it changes.

4. Authority

tiermeaningactions
0just do itassign by the rulebook, request a document, attach the benchmark, open a recovery, add a diary task
1do it and tellfast-track a clean claim at or under EUR 1,500, hold payments, draft a note to a team lead
2a person decidesrefer to Special Investigations, approve a payment above authority, settle as a total loss
3neverdeny a claim, change a reserve, contact a claimant about fraud, cancel a policy, delete data

The limits are written next to the tiers: at most three automatic actions on one claim in any hour (the fourth goes to a person with the reason), thirty a minute across the book (the claims system's rate; the rest waits its turn), sixty a minute for the daily back-office checks on their own budget, and the same problem on the same claim handled once per thirty minutes. Each action has a cooldown, a condition it must meet, and the undo a person would use.

5. The twelve incident kinds

kindwhat it iswhat the operator does
unassigned claima new claim past an hour on the desk with no adjusterassigns by the rulebook, or fast-tracks a clean small claim
duplicate claimthe same loss reported twice, sometimes under a neighbouring perilholds the payments, drafts a note
missing police reporttheft, a hit-and-run or an injury with no report on filerequests it from the claimant
dates that do not fita repair before the loss, a loss before the policy beganasks for a Special Investigations referral
ring indicatorone phone or bank account across unrelated householdsasks for a referral of every claim in the ring
stalled claiman open file with no action for the rule's daysreassigns if the adjuster left, else a diary task
two invoicesthe same repair invoiced twice, sometimes again with taxholds the second payment, tells Claims Finance
recovery misseda paid claim with a third party at fault and no recoveryopens the recovery
total loss not flaggeda repair estimate past the vehicle's total-loss lineasks a person to settle it as a total loss
photo before the lossphoto metadata dated before the lossasks the claimant to confirm, or asks for a referral
estimate above benchmarkan estimate past the benchmark's p90 for its bandputs the benchmark on the file with a review task
payment above authoritya payment above the requesting adjuster's limitasks the team lead to approve

Some are cheap to catch (a date before another date); some are hard on purpose. A duplicate filed under a neighbouring peril, a hit-and-run filed as a collision, an invoice re-issued with tax added and a third party whose fault appears only in the police report each slip past their detector some of the time.

6. How it is measured

The world generator writes an answer key (incidents_truth) with every planted fault: its kind, the claim, ring or payment, the time it became true and the right handling. scripts/replay_day.py runs the operator over a window on a copy of the world, with scripted claims-desk people answering the queue (and undoing some actions they disagree with), and scores precision and recall per kind, the share handled without a person, reverts, time to resolve, and how many decisions rules and the model made. The detectors were tuned on seeds 7 and 11; seed 23 is the hold-out.

On seed 23, seven replayed days (six days of history and the storm Monday), all decided by rules: precision 99%, recall 97%, 93% of incidents handled without a person, 48 actions undone by a person, and the weakest detector, photo before the loss, at 59% precision. The command and the per-kind table are in the README; the Accuracy screen draws the same numbers from the ledger.

7. Plugging in

The world is a DuckDB file; a real deployment replaces it with read access to the claims system, the document store and the payments ledger, mapped to the same tables (the Sources screen lists which fact comes from which system). Each tool has one place where it calls the system; the endpoint on every ledger row names that call. Shadow mode (ADJUST_SHADOW=1) writes every tier 0 and tier 1 action as a "would do" row and changes nothing, which is how a pilot starts. The model is any OpenAI-compatible endpoint, set by environment variables; without one, rules decide.

The known gaps say what the numbers mean and what a pilot needs.