Pick an incident
0/3
incidents resolved
Domain
Topic
Level
3 incidents · 5 planned
- InsuranceINC-1042core
The grounded answer that denied the wrong claim
Wrong policy version selected - faithful citation, wrong contract
20–40 min 1810 practice20 held out- ▸Separate 'the answer quotes a real document' from 'the answer applies the right document'.
- ▸Make contract applicability - not recency - decide which version governs.
Not started0% - InsuranceINC-1188core
The answer that belonged to someone else
Customer access and cache isolation failure
20–35 min 115 practice11 held out- ▸Place the authorization decision before retrieval and before cache access.
- ▸Scope cache entries by trusted identity and by the contract state they describe.
Not started0% - InsuranceINC-1263advanced
The document that became an instruction
Evidence injection and unsafe tool authorization
25–40 min 115 practice9 held out- ▸Separate evidence from instruction so document text cannot select a tool.
- ▸Enforce an allowlist and typed arguments in the application, not in the model's output.
Not started0%
Planned — not built yet
- Healthcarenot built yet
The retriever that forgot which embedding version it indexed
Index rebuilt with a new model while queries kept the old one
coreEmbeddings & vector searchRAG & retrievalTracing & observability - Fintech & paymentsnot built yet
The KYC agent that approved a payee it should have stopped
Screening result cached past its validity window
advancedAgents & toolsAuthZ & privacyCaching - Customer supportnot built yet
The handoff that lost the customer
State dropped between two agents mid-conversation
coreMulti-agent handoffsTracing & observability - Customer supportnot built yet
The judge that graded its own output
An LLM-as-judge eval that rewards the failure it should catch
advancedLLM-as-judgeEvals & grading - Legal & compliancenot built yet
The stream that truncated a refusal into an approval
Partial output committed before the model finished the sentence
introStreaming & latencyTracing & observability