Accuracy
The operator measured against the answer key: 7 replayed days, Tue to Mon. Measured against 2,189 injected incidents (incidents_truth) between the operator's first and last tick, Tue 09:00 to Mon 17:01 CET, 7 replayed days. Incidents from before it was switched on are not counted. A detection matches a planted incident of a kind it reports, on the same claim, ring or payment, within 20 minutes.Precision0.9902,139 detections
Recall0.968of 2,189 injected
Resolved automatically92.6%auto rate, 7-day mean
Reverted by people48actions a person undid
Model callsnonerules only, nothing to pay per resolved incident
Median time to resolve34 min 30 smedian, all resolved · 11 h 43 min when a person decides
Precision per detector, per day
the 6 busiest detectors; the rest are in the table belowReverted actions
who undid the operator, and why| When (CET) | Claim | Kind and call | By | Why |
|---|---|---|---|---|
| Mon 14:12 | CL-26-133781 | unassigned_claim UNDO PUT /claims/29781/fast-track | Saar B. (Fast Track lead, scripted) | storm roof damage over 1,300 needs a surveyor, not fast track [scripted claims-desk responses (synthetic)] |
| Mon 14:06 | CL-26-133777 | unassigned_claim UNDO PUT /claims/29777/fast-track | Saar B. (Fast Track lead, scripted) | storm roof damage over 1,300 needs a surveyor, not fast track [scripted claims-desk responses (synthetic)] |
| Mon 13:50 | CL-26-133763 | unassigned_claim UNDO PUT /claims/29763/fast-track | Saar B. (Fast Track lead, scripted) | storm roof damage over 1,300 needs a surveyor, not fast track [scripted claims-desk responses (synthetic)] |
| Mon 13:05 | CL-26-133718 | unassigned_claim UNDO PUT /claims/29718/fast-track | Saar B. (Fast Track lead, scripted) | storm roof damage over 1,300 needs a surveyor, not fast track [scripted claims-desk responses (synthetic)] |
| Mon 12:00 | CL-26-133664 | unassigned_claim UNDO PUT /claims/29664/fast-track | Saar B. (Fast Track lead, scripted) | storm roof damage over 1,300 needs a surveyor, not fast track [scripted claims-desk responses (synthetic)] |
| Mon 11:49 | CL-26-133654 | unassigned_claim UNDO PUT /claims/29654/fast-track | Saar B. (Fast Track lead, scripted) | storm roof damage over 1,300 needs a surveyor, not fast track [scripted claims-desk responses (synthetic)] |
| Mon 11:47 | CL-26-133652 | unassigned_claim UNDO PUT /claims/29652/fast-track | Saar B. (Fast Track lead, scripted) | storm roof damage over 1,300 needs a surveyor, not fast track [scripted claims-desk responses (synthetic)] |
| Mon 11:31 | CL-26-133640 | unassigned_claim UNDO PUT /claims/29640/fast-track | Saar B. (Fast Track lead, scripted) | storm roof damage over 1,300 needs a surveyor, not fast track [scripted claims-desk responses (synthetic)] |
| Mon 11:09 | CL-26-133601 | unassigned_claim UNDO PUT /claims/29601/fast-track | Saar B. (Fast Track lead, scripted) | storm roof damage over 1,300 needs a surveyor, not fast track [scripted claims-desk responses (synthetic)] |
| Mon 11:09 | CL-26-133585 | unassigned_claim UNDO PUT /claims/29585/fast-track | Saar B. (Fast Track lead, scripted) | storm roof damage over 1,300 needs a surveyor, not fast track [scripted claims-desk responses (synthetic)] |
| Mon 11:06 | CL-26-133525 | unassigned_claim UNDO PUT /claims/29525/fast-track | Saar B. (Fast Track lead, scripted) | storm roof damage over 1,300 needs a surveyor, not fast track [scripted claims-desk responses (synthetic)] |
| Mon 11:06 | CL-26-133519 | unassigned_claim UNDO PUT /claims/29519/fast-track | Saar B. (Fast Track lead, scripted) | storm roof damage over 1,300 needs a surveyor, not fast track [scripted claims-desk responses (synthetic)] |
| Mon 11:04 | CL-26-133501 | unassigned_claim UNDO PUT /claims/29501/fast-track | Saar B. (Fast Track lead, scripted) | storm roof damage over 1,300 needs a surveyor, not fast track [scripted claims-desk responses (synthetic)] |
| Mon 11:04 | CL-26-133475 | unassigned_claim UNDO PUT /claims/29475/fast-track | Saar B. (Fast Track lead, scripted) | storm roof damage over 1,300 needs a surveyor, not fast track [scripted claims-desk responses (synthetic)] |
| Mon 11:03 | CL-26-133448 | unassigned_claim UNDO PUT /claims/29448/fast-track | Saar B. (Fast Track lead, scripted) | storm roof damage over 1,300 needs a surveyor, not fast track [scripted claims-desk responses (synthetic)] |
| Mon 11:00 | CL-26-133383 | unassigned_claim UNDO PUT /claims/29383/fast-track | Saar B. (Fast Track lead, scripted) | storm roof damage over 1,300 needs a surveyor, not fast track [scripted claims-desk responses (synthetic)] |
| Mon 10:58 | CL-26-133339 | unassigned_claim UNDO PUT /claims/29339/fast-track | Saar B. (Fast Track lead, scripted) | storm roof damage over 1,300 needs a surveyor, not fast track [scripted claims-desk responses (synthetic)] |
| Mon 10:57 | CL-26-133285 | unassigned_claim UNDO PUT /claims/29285/fast-track | Saar B. (Fast Track lead, scripted) | storm roof damage over 1,300 needs a surveyor, not fast track [scripted claims-desk responses (synthetic)] |
| Mon 10:55 | CL-26-133243 | unassigned_claim UNDO PUT /claims/29243/fast-track | Saar B. (Fast Track lead, scripted) | storm roof damage over 1,300 needs a surveyor, not fast track [scripted claims-desk responses (synthetic)] |
| Mon 10:51 | CL-26-133127 | unassigned_claim UNDO PUT /claims/29127/fast-track | Saar B. (Fast Track lead, scripted) | storm roof damage over 1,300 needs a surveyor, not fast track [scripted claims-desk responses (synthetic)] |
| Mon 10:49 | CL-26-133080 | unassigned_claim UNDO PUT /claims/29080/fast-track | Saar B. (Fast Track lead, scripted) | storm roof damage over 1,300 needs a surveyor, not fast track [scripted claims-desk responses (synthetic)] |
| Mon 10:48 | CL-26-133070 | unassigned_claim UNDO PUT /claims/29070/fast-track | Saar B. (Fast Track lead, scripted) | storm roof damage over 1,300 needs a surveyor, not fast track [scripted claims-desk responses (synthetic)] |
| Mon 10:48 | CL-26-133051 | unassigned_claim UNDO PUT /claims/29051/fast-track | Saar B. (Fast Track lead, scripted) | storm roof damage over 1,300 needs a surveyor, not fast track [scripted claims-desk responses (synthetic)] |
| Mon 10:46 | CL-26-133005 | unassigned_claim UNDO PUT /claims/29005/fast-track | Saar B. (Fast Track lead, scripted) | storm roof damage over 1,300 needs a surveyor, not fast track [scripted claims-desk responses (synthetic)] |
| Mon 10:46 | CL-26-132998 | unassigned_claim UNDO PUT /claims/28998/fast-track | Saar B. (Fast Track lead, scripted) | storm roof damage over 1,300 needs a surveyor, not fast track [scripted claims-desk responses (synthetic)] |
| Mon 10:43 | CL-26-132938 | unassigned_claim UNDO PUT /claims/28938/fast-track | Saar B. (Fast Track lead, scripted) | storm roof damage over 1,300 needs a surveyor, not fast track [scripted claims-desk responses (synthetic)] |
| Mon 10:43 | CL-26-132926 | unassigned_claim UNDO PUT /claims/28926/fast-track | Saar B. (Fast Track lead, scripted) | storm roof damage over 1,300 needs a surveyor, not fast track [scripted claims-desk responses (synthetic)] |
| Mon 10:43 | CL-26-132921 | unassigned_claim UNDO PUT /claims/28921/fast-track | Saar B. (Fast Track lead, scripted) | storm roof damage over 1,300 needs a surveyor, not fast track [scripted claims-desk responses (synthetic)] |
| Mon 10:43 | CL-26-132919 | unassigned_claim UNDO PUT /claims/28919/fast-track | Saar B. (Fast Track lead, scripted) | storm roof damage over 1,300 needs a surveyor, not fast track [scripted claims-desk responses (synthetic)] |
| Mon 10:42 | CL-26-132898 | unassigned_claim UNDO PUT /claims/28898/fast-track | Saar B. (Fast Track lead, scripted) | storm roof damage over 1,300 needs a surveyor, not fast track [scripted claims-desk responses (synthetic)] |
| Mon 10:39 | CL-26-132830 | unassigned_claim UNDO PUT /claims/28830/fast-track | Saar B. (Fast Track lead, scripted) | storm roof damage over 1,300 needs a surveyor, not fast track [scripted claims-desk responses (synthetic)] |
| Mon 10:37 | CL-26-132784 | unassigned_claim UNDO PUT /claims/28784/fast-track | Saar B. (Fast Track lead, scripted) | storm roof damage over 1,300 needs a surveyor, not fast track [scripted claims-desk responses (synthetic)] |
| Mon 10:37 | CL-26-132781 | unassigned_claim UNDO PUT /claims/28781/fast-track | Saar B. (Fast Track lead, scripted) | storm roof damage over 1,300 needs a surveyor, not fast track [scripted claims-desk responses (synthetic)] |
| Mon 10:36 | CL-26-132750 | unassigned_claim UNDO PUT /claims/28750/fast-track | Saar B. (Fast Track lead, scripted) | storm roof damage over 1,300 needs a surveyor, not fast track [scripted claims-desk responses (synthetic)] |
| Mon 10:36 | CL-26-132736 | unassigned_claim UNDO PUT /claims/28736/fast-track | Saar B. (Fast Track lead, scripted) | storm roof damage over 1,300 needs a surveyor, not fast track [scripted claims-desk responses (synthetic)] |
| Mon 10:36 | CL-26-132719 | unassigned_claim UNDO PUT /claims/28719/fast-track | Saar B. (Fast Track lead, scripted) | storm roof damage over 1,300 needs a surveyor, not fast track [scripted claims-desk responses (synthetic)] |
| Mon 10:34 | CL-26-132717 | unassigned_claim UNDO PUT /claims/28717/fast-track | Saar B. (Fast Track lead, scripted) | storm roof damage over 1,300 needs a surveyor, not fast track [scripted claims-desk responses (synthetic)] |
| Mon 10:34 | CL-26-132704 | unassigned_claim UNDO PUT /claims/28704/fast-track | Saar B. (Fast Track lead, scripted) | storm roof damage over 1,300 needs a surveyor, not fast track [scripted claims-desk responses (synthetic)] |
| Mon 10:33 | CL-26-132655 | unassigned_claim UNDO PUT /claims/28655/fast-track | Saar B. (Fast Track lead, scripted) | storm roof damage over 1,300 needs a surveyor, not fast track [scripted claims-desk responses (synthetic)] |
| Mon 10:22 | CL-26-132387 | unassigned_claim UNDO PUT /claims/28387/fast-track | Saar B. (Fast Track lead, scripted) | storm roof damage over 1,300 needs a surveyor, not fast track [scripted claims-desk responses (synthetic)] |
| Mon 10:21 | CL-26-132348 | unassigned_claim UNDO PUT /claims/28348/fast-track | Saar B. (Fast Track lead, scripted) | storm roof damage over 1,300 needs a surveyor, not fast track [scripted claims-desk responses (synthetic)] |
| Mon 10:19 | CL-26-132313 | unassigned_claim UNDO PUT /claims/28313/fast-track | Saar B. (Fast Track lead, scripted) | storm roof damage over 1,300 needs a surveyor, not fast track [scripted claims-desk responses (synthetic)] |
| Mon 10:18 | CL-26-132291 | unassigned_claim UNDO PUT /claims/28291/fast-track | Saar B. (Fast Track lead, scripted) | storm roof damage over 1,300 needs a surveyor, not fast track [scripted claims-desk responses (synthetic)] |
| Mon 10:18 | CL-26-132272 | unassigned_claim UNDO PUT /claims/28272/fast-track | Saar B. (Fast Track lead, scripted) | storm roof damage over 1,300 needs a surveyor, not fast track [scripted claims-desk responses (synthetic)] |
| Mon 10:16 | CL-26-132251 | unassigned_claim UNDO PUT /claims/28251/fast-track | Saar B. (Fast Track lead, scripted) | storm roof damage over 1,300 needs a surveyor, not fast track [scripted claims-desk responses (synthetic)] |
| Mon 10:15 | CL-26-132213 | unassigned_claim UNDO PUT /claims/28213/fast-track | Saar B. (Fast Track lead, scripted) | storm roof damage over 1,300 needs a surveyor, not fast track [scripted claims-desk responses (synthetic)] |
| Mon 10:15 | CL-26-132202 | unassigned_claim UNDO PUT /claims/28202/fast-track | Saar B. (Fast Track lead, scripted) | storm roof damage over 1,300 needs a surveyor, not fast track [scripted claims-desk responses (synthetic)] |
| Mon 10:13 | CL-26-132169 | unassigned_claim UNDO PUT /claims/28169/fast-track | Saar B. (Fast Track lead, scripted) | storm roof damage over 1,300 needs a surveyor, not fast track [scripted claims-desk responses (synthetic)] |
Drift alarms
precision down 15 pts week on week| Mon | green | all detectors | Week-on-week alarms need 14 replayed days; the ledger has 7. waiting for data |
| Mon | green | model | No drift: answers usable, confidence steady, few reverts. recal.drift |
Per detector, 7 days
precision: of what it flagged, how much was a planted incident| Detector | Detections | Precision | Reports |
|---|---|---|---|
| date_mismatch | 34 | 1.000 | date_mismatch |
| double_invoice | 80 | 1.000 | double_invoice |
| duplicate_claim | 45 | 0.956 | duplicate_claim |
| estimate_above_benchmark | 62 | 0.919 | estimate_above_benchmark |
| missing_police_report | 134 | 1.000 | missing_police_report |
| payment_over_authority | 42 | 1.000 | payment_over_authority |
| photo_before_loss | 35 | 0.600 | photo_before_loss |
| ring_indicator | 7 | 1.000 | ring_indicator |
| stalled_claim | 161 | 1.000 | stalled_claim |
| subrogation_missed | 156 | 1.000 | subrogation_missed |
| total_loss_unflagged | 12 | 1.000 | total_loss_unflagged |
| unassigned_claim | 1,371 | 1.000 | unassigned_claim |
Planted incidents, by kind
recall: of what was planted, how much it found; the counts add up to 2,189| Planted kind | Planted | Found | Recall | Reported by |
|---|---|---|---|---|
| unassigned_claim | 1,372 | 1,371 | 0.999 | unassigned_claim |
| subrogation_missed | 179 | 156 | 0.872 | subrogation_missed |
| stalled_claim | 166 | 161 | 0.970 | stalled_claim |
| missing_police_report | 142 | 134 | 0.944 | missing_police_report |
| double_invoice | 105 | 80 | 0.762 | double_invoice |
| estimate_above_benchmark | 57 | 57 | 1.000 | estimate_above_benchmark |
| duplicate_claim | 52 | 43 | 0.827 | duplicate_claim |
| payment_over_authority | 42 | 42 | 1.000 | payment_over_authority |
| date_mismatch | 34 | 34 | 1.000 | date_mismatch |
| photo_before_loss | 21 | 21 | 1.000 | photo_before_loss |
| total_loss_unflagged | 12 | 12 | 1.000 | total_loss_unflagged |
| ring_indicator | 7 | 7 | 1.000 | ring_indicator |
| total | 2,189 | 2,118 | 0.968 |