Skip to content

Learn AI by fixing what breaks.

Real incidents from AI systems that pass their evals and still get it wrong. Yours to diagnose and repair.

Browse incidents

FreeNo sign-upNo API key

run · INC-1042 · before your fix
C1-001C1-002C1-003C1-004C1-005+5

decline · governing version V2 · waiting 4y → 2026-06-15

“Under V2 a waiting period of four years applies from 2022-06-15… the claim date 2025-01-10 falls inside that waiting period.”

05 retrieval.rank · weights recency 2.0 · applicability 0.0

06 resolve.version · applicability not enforced — taking top-ranked wording (V2)

07 decide.waiting_period · claim 2025-01-10 → deny

The citation is real. The quotation is accurate. The contract is the wrong one.

Incidents

All incidents
core

The grounded answer that denied the wrong claim

Wrong policy version selected - faithful citation, wrong contract

InsuranceRAG & retrievalMetadata filtering

20–40 min· 30 cases

core

The answer that belonged to someone else

Customer access and cache isolation failure

InsuranceCachingAuthZ & privacy

20–35 min· 16 cases

advanced

The document that became an instruction

Evidence injection and unsafe tool authorization

InsuranceAgents & toolsPrompt injection

25–40 min· 14 cases

A training sandbox. The insurer, the policyholders, the documents and the incidents are invented for teaching — no real company, policy or law is described, and nothing you do here touches a real claim or payment.