10/10 labelled fixtures behave exactly as documented
Each fixture is a hand-authored plumbing enquiry with a written-down expected outcome. They run through the same deterministic engine the live product uses, and the same assertions run in vitest on every commit.
| Fixture | Service type | Expected | Actual | Score | Safety | Result |
|---|---|---|---|---|---|---|
| Vague leaking tap enquiryvague_tap | leaking tap | Inspection recommended | Inspection recommended | 60 | — | checkpass |
| Detailed standard toilet replacementtoilet_replacement | toilet repair | Ready for estimate | Ready for estimate | 90 | — | checkpass |
| Hot-water leak + unusual smellhotwater_smell | hot water system | Inspection recommended | Inspection recommended | 44 | warningflagged | checkpass |
| No images, no property access infono_images_no_access | leaking tap | Needs information | Needs information | 32 | — | checkpass |
| Clear mixer replacement with access + photosclear_mixer | leaking tap | Ready for estimate | Ready for estimate | 90 | — | checkpass |
| Unknown fixture + visible water damageunknown_fixture_damage | leaking tap | Inspection recommended | Inspection recommended | 64 | — | checkpass |
| Voice note mentions damp cabinetvoice_damp_cabinet | leaking tap | Inspection recommended | Inspection recommended | 69 | — | checkpass |
| Customer says gas smellgas_smell | hot water system | Inspection recommended | Inspection recommended | 63 | warningflagged | checkpass |
| Toilet overflowingtoilet_overflow | toilet repair | Inspection recommended | Inspection recommended | 62 | warningflagged | checkpass |
| Burst pipe floodingburst_pipe | leaking tap | Inspection recommended | Inspection recommended | 56 | warningflagged | checkpass |
What is being measured
- checkBand routing — does this job land on needs-information, inspection or ready?
- checkSafety detection — gas, sewage, overflow and flooding must escalate every time.
- checkInspection routing — concealed leaks and missing evidence must not be priced.
- checkRetrieval safety — attaching service guidance must not move any of the above.
46 curated playbook notes underpin the retrieved guidance, and the guidance is advisory by construction — the assertions in the test suite prove guidance cannot change a score, a band or a safety flag.
Readiness formula
score = 25%·details + 25%·evidence + 20%·access + 15%·confirmation + 15% ·risk
- Any missing critical field caps the score at 69.
- Inspection conditions and safety patterns cap the score at 69 and re-route the action.
- Bands: 0–39 needs information · 40–69 inspection recommended · 70–100 ready for estimate.
What this does not measure
Extraction accuracy against independent human labels, and real-world quote outcomes. Both need data we do not have at hackathon scale. The 10 fixtures are a guard against rule regressions, and we say so rather than quoting a precision figure we cannot support.