Diagnoses
Every trace ends with one diagnosis: what went wrong, the stage it blames, and why.
A diagnosis is the single conclusion for a trace. It has a label, a reason, its source (who decided), and the stage it blames. Every trace has exactly one.
| Diagnosis | Stage it blames | Meaning |
|---|---|---|
| Healthyhealthy_answer | None | Retrieval, grounding and the answer all passed. |
| Correct refusalcorrect_refusal | None | The context didn't cover the question, and the model said so. |
| Retrieval missretrieval_miss | Retrieval | The retrieved context doesn't match the question. |
| Weak retrievallow_context_relevance | Retrieval | The context is only loosely related to the question. |
| Unsupported claimunsupported_claim | Grounding | The answer states something the context doesn't support. |
| Wrong answerwrong_answer | Answer | Grounded, but it doesn't answer the question, or it refuses one it could answer. |
| Needs reviewneeds_review | None | The checks couldn't decide; a person should look. |
How it's reached#
Three checks look at each trace:
- Context relevance: does the retrieved context match the question?
- Groundedness: is every claim in the answer supported by that context? The judge splits the answer into claims and checks each one.
- Answer relevance: does the answer address the question?
A refusal ("the context doesn't say") is checked first: it's a correct refusal if the context really lacked the answer, and a wrong answer if the context had it. Otherwise the first check that failed, in the order above, decides the diagnosis. All three passing makes a healthy answer; checks that can't decide make "needs review".
Healthy#
"Healthy answer" and "correct refusal" both count as healthy: the pipeline did the right thing. Everything else is a failure, except "needs review", which asks for a person to look.
Who decided#
A diagnosis can come from a cheap estimate or from the LLM judge, and the dashboard always says which. See Estimates and judge verdicts.