Accuracy
The operator scored against the answer key the world generator wrote: every planted fault, its kind, its time, the home or building, and the correct handling. Replayed from Tue 08:00 to Mon 20:00; the busy window starts Mon 08:00.
scored on the replayed ledger
Weakest detector: Duplicate ticket, precision 88% on 26 detections, recall 100%. It is listed in known gaps with the reason.
Day by day
Tue 27 Oct
- detections
- 207
- precision
- 100%
- new faults
- 11
- recall
- 100%
- no person needed
- 83%
- reverted
- 2
Wed 28 Oct
- detections
- 13
- precision
- 92%
- new faults
- 8
- recall
- 100%
- no person needed
- 100%
- reverted
- 0
Thu 29 Oct
- detections
- 11
- precision
- 100%
- new faults
- 8
- recall
- 88%
- no person needed
- 86%
- reverted
- 1
Fri 30 Oct
- detections
- 14
- precision
- 93%
- new faults
- 9
- recall
- 100%
- no person needed
- 100%
- reverted
- 0
Sat 31 Oct
- detections
- 54
- precision
- 100%
- new faults
- 10
- recall
- 100%
- no person needed
- 100%
- reverted
- 0
Sun 1 Nov
- detections
- 32
- precision
- 100%
- new faults
- 29
- recall
- 100%
- no person needed
- 97%
- reverted
- 1
Mon 2 Nov · rent day
- detections
- 53
- precision
- 96%
- new faults
- 48
- recall
- 100%
- no person needed
- 68%
- reverted
- 0
detectionsactionsfaults that began that dayRecall per day counts only faults that began that day, so an old fault caught on the first day is not missed again on every later one.
Per incident kind, the replayed week
| Incident kind | Detections | Precision | Faults in the key | Recall | Busy window detections | Busy window precision |
|---|---|---|---|---|---|---|
| Duplicate ticket weakest duplicate_ticket |
26 | 88% | 23 | 100% | 6 | 67% |
| Inspection overdue inspection_overdue |
14 | 100% | 14 | 100% | 0 | n/a |
| Invoice without ticket invoice_without_ticket |
22 | 100% | 22 | 100% | 15 | 100% |
| Lift out, no other lift lift_outage_single |
10 | 100% | 10 | 100% | 5 | 100% |
| Long vacancy long_vacancy |
14 | 100% | 15 | 93% | 3 | 100% |
| Renewal not sent renewal_not_sent |
38 | 100% | 38 | 100% | 0 | n/a |
| Offer outside band renewal_out_of_band |
24 | 100% | 24 | 100% | 16 | 100% |
| Rent arrears rent_arrears |
183 | 100% | 143 | 100% | 0 | n/a |
| Ticket past SLA ticket_sla_breach |
19 | 89% | 18 | 94% | 3 | 100% |
| Water alarm, no ticket water_alarm_no_ticket |
12 | 100% | 12 | 100% | 4 | 100% |
Precision is the share of detections that match a planted fault (low means false alarms). Recall is the share of planted faults active in the span that the operator detected, not the share fixed. In the busy window a fault counts as caught if the operator had found it at any point of the replay up to the window's end: busy window recall 100% of 253 faults, precision 96% on 52 detections.