Adjust claims desk · Quillmark Mutual
World clock Mon 17:00:00 CET paused · 60×

How it works

The full design. Back to the ten-minute version. docs/architecture-full.md

Adjust: the full design

The ten-minute version says what the operator does and why. This page says how each part works, where it lives in the repository, and which check stops what. It is written for the engineer who will connect Adjust to a real claims system, and for the reviewer who has to sign off on what it may do.

The repository at a glance

pathwhat lives there
adjust/world/the synthetic claims book: generator, schema, names, planted faults, the answer key
adjust/dictionary/the claims terms: one YAML file per term, the evaluator, version history, tests
adjust/agent/detectors, cadence, proposals (rules and model), the controller, tools, the scripted desk
adjust/agent/policy.yamlthe operator's written authority
adjust/ledger/the append-only ledger, approvals and repeats
adjust/recal/matching detections to the answer key, the metrics, the drift signal
adjust/ask/questions answered through the dictionary only
app/the web screens (FastAPI, Jinja templates, htmx) over the same database
scripts/replay_day.pya replay on a copy of the world, with its report
scripts/screenshot_all.pyevery screen at 1440x900 through Playwright

Everything runs in one process against one DuckDB file. Nothing in the repository calls a network service except the optional model endpoint.

Data model

The world is one DuckDB file. Its tables (adjust/world/schema.sql):

tableholds
regions, townsthe seven regions of Estmark and their towns, with coordinates
teams, adjusterseight claims teams by line and kind, the adjusters with grade, region, authority limit, caseload, leave and leaving dates
vendorsgarages, glass specialists and building contractors, with network membership, labour rate, phone and bank account
policyholders, policiesthe policy book: holder (with phone and bank account), line, product, sum insured, deductible, cover start and renewal
benchmarksrepair-cost p50 and p90 per line, peril and size band, versioned
assignment_rulesthe rulebook: which team takes which line and value, in priority order
stormsnamed weather events with start, end and regions
claimsthe claim: policy, holder, line, peril, cause, loss and report times, estimate, third party, status, and a timestamp for every change (assigned_at, fast_track_at, siu_referred_at, total_loss_at, closed_at)
claim_eventsthe file history: reported, assigned, notes, estimate authorised, payment approved, closed
documentspolice reports, estimates, invoices, photos (metadata only: device, taken-at), each with its received time
paymentsrequested, approved, held and paid payments with payee, amount, invoice and the requesting adjuster
contactsthe phone and email log per claim
recoveriesrecoveries opened against a third party's insurer
claims_dailyreported and closed per day, line and region, for the charts
incidents_truththe answer key: kind, claim (or ring, or payment), start, end, the right handling
systems, fact_sourceswhich source system each fact would come from in a real deployment

Every column that changes over a claim's life carries the time it changed, and every event stream has its own time. A check therefore asks "as of now", a replay can run any window of the 90 days, and the Desk can show a file exactly as it stood at the clock: nothing dated after the clock is read.

The operator owns its own tables (adjust/agent/schema.sql): messages (the outbox: drafts nobody has sent), flags (referrals to Special Investigations), tasks (diary tasks), document_requests, process_gaps (kinds with no runbook), detector_runs (when each check last ran on the logical clock) and replay_events (markers for the accuracy timeline).

Logical time

Nothing asks the wall clock. adjust/clock.py holds the logical time; detectors, tools, the scripted desk, the ledger and the screens all read it. A replay moves it in steps; Play moves it at 10x to 120x. Every ledger row keeps its logical time, so a stored run can be streamed back later by that time.

Definitions

adjust/dictionary/definitions/*.yaml: one file per term. A term has params, synonyms and one or more variants; each variant has an owner, a department, a version with its history, SQL with {now}, {period_start} and param placeholders, and tests. A param change is appended to _history.jsonl and bumps the version of every variant that uses it; nothing is edited in place. The detectors call dictionary.evaluate and nothing else, so the operator acts on exactly the definition on file, at the version the ledger records.

Terms with two meanings keep both:

termmeaning onemeaning two
open claimClaims: reported and not closedFinance: money still owed on the file, closed or not
clean claimClaims: small, no injury, no theft or hit-and-run, policy long enough in force, not referredFraud: also no recent claim on the policy, the claimant's own phone and bank account, no photo before the loss
total lossClaims: latest estimate past the engineering share of the vehicle's valueFinance: a lower share, because hire car and storage count too
stalled claimClaims: nothing done on the file (no event, no outbound contact) for the rule's daysService: nobody has spoken to the claimant for the rule's days

When a term has no canonical variant, Ask shows every meaning side by side and the detector names the one it uses. The Definitions screen can preview a param change (what_changes): which claims a move of stalled_days from 10 to 7 would add or drop, computed through the same governed SQL before anyone commits the change.

Detection and cadence

adjust/agent/detectors.py maps each incident kind to its term and variant. The policy says which run when (adjust/agent/cadence.py, with last runs kept in detector_runs so a restart does not repeat them):

cadencekinds
every tickunassigned claim, duplicate claim, missing police report, dates that do not fit, two invoices, estimate above benchmark, payment above authority
hourlyring indicator, total loss not flagged, photo before the loss
daily at 09:00stalled claim, recovery missed

A detection becomes an incident with a stable id (kind and subject: a claim, a ring key, a payment), its evidence rows, and the definition and version that produced it. A ring incident lists every claim it covers.

The loop

adjust/agent/runner.py, once per tick:

  1. Detect. Run the checks that are due.
  2. Filter. Drop what needs no new decision: handled inside the dedupe window (30 minutes), waiting in the approval queue, overridden by a person, done once for good (once_per_incident), or inside an action's cooldown.
  3. Propose. adjust/agent/workers.py: the rule table (rules.py, one small function per kind) proposes, unless the runbook leaves a real choice and the kind is not in rules_decide; then the model is asked. Proposals run in parallel (parallel.py), and an identical proposal for the same kind, subject and evidence is reused inside a run.
  4. Decide. The controller checks the proposal (below).
  5. Act. Execute (tier 0 and 1), queue (tier 2) or decline (tier 3, or a condition not met), and write the ledger. A runbook step can chain: after a full team, assign_adjuster becomes a note to the team lead.

Items held back by the per-minute rate wait in line and go first on the next tick. The controller's decisions and every ledger write stay on the runner's thread in detection order, because budgets and cooldowns depend on what was just done.

The controller

adjust/agent/controller.py is separate from the model and re-reads policy.yaml on every decision. For each proposal it checks, in order:

  1. the action exists in the policy and is not on the never list;
  2. the action is in the runbook for this kind of incident;
  3. its arguments are exactly the ones the tool takes, with allowed values where the policy lists them;
  4. every claim, payment or ring it names is in the incident's evidence (the model cannot point the operator at a file it was not shown);
  5. the action's written condition holds;
  6. the action is not inside its cooldown on this subject;
  7. the claim has budget left (three automatic actions in any rolling hour);
  8. the book has rate left (thirty a minute for file work, sixty a minute for the daily back-office checks on their own budget).

The outcome is one of: execute, queue for a person, wait (rate), or decline with the reason. A used-up claim budget sends the item to a person with that reason; a rate limit makes it wait; anything unknown is tier 3. Changing what the operator may do means changing policy.yaml, whose version and change note are on every screen that uses it.

Tools and undo

adjust/agent/tools/claims.py, outbox.py and rulebook.py. Each tool re-reads the world, refuses what no longer applies (a claim already assigned, a payment already paid), and returns before and after snapshots.

actiontierthe call it stands forundo
assign_adjuster0PUT /claims/{id}/assignmentback to the previous adjuster, or back on the desk
request_document0POST /claims/{id}/document-requestswithdraw the request
attach_benchmark0POST /claims/{id}/noteswithdraw the note, cancel the review task
open_subrogation_note0POST /recoverieswithdraw the recovery
create_task0POST /claims/{id}/taskscancel the task
fast_track1PUT /claims/{id}/fast-trackback to the desk
hold_payments1PUT /payments/{id}/holdrelease the hold
draft_note1outbox draftcancel the draft (it was never sent)
refer_to_siu2POST /siu/referralswithdraw the referral
approve_payment2PUT /payments/{id}/approvalwithdraw the approval before the money leaves
settle_total_loss2PUT /claims/{id}/total-lossremove the flag, release the holds, withdraw the settlement

Undo restores updated rows only if nobody has changed them since, and neutralises inserted rows (a request withdrawn, a recovery withdrawn, a new payment held, a claim event compensated). Nothing is deleted. The rulebook (assignment_rules, version 4) picks the team for the line and value, then the working adjuster with the most free places; adjusters on leave or past their leaving date are skipped.

Ledger

adjust/ledger/: append-only; the API has no update and no delete.

columnmeaning
atlogical time of the row
incident_id, kind, subject_type, subject_idwhat was found, on what
evidencethe rows the detector saw
definition_term, definition_versionthe definition it ran on
proposal, decisionwhat was proposed (rules or model, with the rationale) and what the controller decided
action, args, endpoint, tierthe call and its tier
resultbefore and after snapshots
statusauto_done, escalated, approved, rejected, declined, rolled_back, failed, shadow
human, note, parent_idwho decided, why, and the row this one acts on
model, tokens, costwhen a model was asked

A status change is a new row pointing at its parent; approvals have their own table with who and when. A repeat of the same decline inside its cooldown is folded into the first row (ledger_repeats), so one incident stays one line.

Approvals and the scripted desk

Tier 2 items wait in the approval queue. During desk hours each one also puts a one-line approval request in the outbox for the claims duty manager. On the screens a person approves or rejects with a note; an approval runs the tool at that moment, under the same checks.

In a replay, scripted people answer (adjust/agent/humans.yaml): a duty manager, a Special Investigations lead, a motor claims lead, a team lead and a Fast Track lead, each with desk hours and a decision delay on the logical clock. Their rules say which items they approve, which they reject, and which automatic actions they undo (a fast-tracked storm claim with an estimate above a line, a duplicate hold on losses ten hours apart). Every one of their rows is labelled "scripted claims-desk responses (synthetic)".

The model path

adjust/agent/llm.py speaks to any OpenAI-compatible endpoint set by ADJUST_LLM_BASE_URL, ADJUST_LLM_API_KEY, ADJUST_LLM_MODEL and ADJUST_LLM_FALLBACKS. It is asked only where the runbook leaves a choice (for example, whether a photo before the loss asks the claimant to confirm or goes to Special Investigations), answers JSON, and is validated against the policy before use. A missing, failing or invalid answer falls back to the rule table, and the ledger says which path decided. The drift signal on the Accuracy screen ("the cliff") watches whether the model is getting less sure, more often invalid or more often reverted.

Process gaps

A kind of incident with no runbook gets no action. adjust/agent/gaps.py files one backlog row per kind and, where people have handled similar items through the queue, drafts a runbook from what they approved, rejected and undid. A person turns the draft into policy.

Shadow mode

ADJUST_SHADOW=1: every tier 0 and tier 1 action is written as a shadow row naming the call it would have made, and the world is left exactly as it was. Tier 2 items still go to the queue as questions. This is how a pilot starts on a live book.

Replay and accuracy

scripts/replay_day.py copies the world (it refuses data/ and any database whose ledger already has rows), runs the history days on five-minute steps and the storm Monday on thirty-second steps, with the scripted desk answering the queue, and writes out/replay-<label>.json.

adjust/recal/matching.py matches each detection to the answer key by kind, by key (claim, ring or payment; a ring detection covers every claim it lists) and by time overlap. adjust/recal/metrics.py reports, per kind and overall:

  • precision: detections that matched an incident in the key, over all detections;
  • recall: incidents in the key that were detected, over all incidents overlapping the window;
  • handled without a person: incidents resolved by tier 0 and 1 actions that nobody undid;
  • reverts: actions a person undid;
  • time to resolve: from the moment the problem became true to the action that resolved it;
  • decided by rules, by a model: counts, with tokens and cost when a model was used.

The weakest detector (lowest precision) is named on the Accuracy screen. Tuning notes for seeds 7 and 11 are in adjust/dictionary/TUNING.md; seed 23 is the hold-out.

The screens

app/ is FastAPI with Jinja templates and htmx; ADJUST_UI_SERVICES=real reads the database named by ADJUST_DB (default data/adjust.duckdb).

  • Desk (/): the claim-file workbench. app/desk_queue.py groups ledger rows per claim (a ring row lands on every claim it covers; a person's decision follows its parent row) and orders the in-tray: a decision waiting for a person, then fraud and money signals, then the quality of the file, then routine assignment, newest first inside each. app/desk_file.py reads the open claim as it stood at the clock, marks the fields a detector pointed at and the rows the operator wrote, and splits it into Facts, Documents, Money and People. The evidence panel renders one receipt per ledger row on that claim.
  • Play: if the ledger already holds rows for the desk day, Play streams them back as the clock reaches them (a recorded run; nothing is written). On a world with an empty desk day, Play starts the operator itself on the logical clock and its actions land as they happen. The screen polls every two seconds while the clock runs; with "follow the operator" on, the file in the middle switches to the claim just worked on.
  • Ledger, Approvals, Accuracy, Definitions, Ask, Sources (/map), How it works and Known gaps read the same database and the same documents as this page.

Every number on a screen is a query over the world or the ledger; the footer names the world seed and whether the ledger is a recorded run or the live operator.

Tests

pytest -q covers: the world's determinism and scale and every planted kind; each detector catching its planted cases and every definition passing its own tests; the controller refusing tier 3, unknown actions, arguments not in the evidence and over-budget actions; every tool's undo; the scripted desk; a replay on a tiny window writing its report and refusing a used ledger; every screen answering 200 and saying "synthetic data" once; the Desk's ordering, grouping, clock and tabs; and the public copy's voice.