Keysproperty operations Corbelstone Residential
Architecture in ten minutesIn fullKnown gapsdocs/known-gaps.md

Known gaps

This page lists what Keys does not handle, and what its numbers can and cannot show. Read it before you trust a figure in the README.

The data is synthetic

Every building, home, lease, payment, ticket and person is generated from a seed (python -m keys.world.generate --seed 23 --out data/keys.duckdb). Nothing comes from a real portfolio.

What that gives:

What it cannot show:

The weakest detector: duplicate ticket

On the seed 23 week, duplicate ticket has the lowest precision of the ten detectors: 88.5% on 26 detections, and 4 of 6 on the busy day. Two separate faults of the same kind in the same building, opened close together, look like one report made twice: two leaks on different floors, two hallway lights. The common-area window was narrowed from 48 to 8 hours on the tuning seeds (keys/dictionary/TUNING.md), and tickets raised by two different alarms are never paired, but the definition still has no floor or location inside the building to tell two hallway faults apart. The scripted maintenance lead reverted three of these merges in the replay ("two separate faults on different floors, not one"). A real help desk with a location field would close most of the gap; merging stays tier 0 because a revert is cheap.

The world answers only a little

The operator's tools change the world, and the world now replies to five of them: a vendor the operator dispatches attends (inside what is left of the SLA, or within two hours) and the job is done and the ticket closed a few hours later; the site team's later ticket for an alarm the operator already ticketed is merged; a tenant who got an arrears reminder pays in full 1 to 3 hours later (60% of them, fixed per lease); a home listed at the band price is let 3 to 9 hours later (30%), a home with a concession 1 to 5 hours later (70%), with a new lease written. Those delays are compressed to hours so a twelve-hour replay can show them; real tenants take days. A re-issued renewal offer is still not answered. The answer key does not move with the operator's actions, and the replay cannot show whether an action helped. It shows whether the operator saw the fault and did what the policy says.

Who decides: rules by default, the model only for a real choice

Most incidents are decided by rules: when the runbook leaves one action at tier 0 or 1, a fixed rule fills it in and no model is called. The model is asked where there is a real choice (dispatch, re-dispatch or replace a vendor; remind or go to legal; list or offer a concession). The README run used rules only; no model was configured, so the model's quality on these choices is not measured.

What the numbers mean

The main screen

Not built