TraceroAI

Traces

A trace is one run of your pipeline, recorded from the question to the answer.

A trace records one run of your pipeline answering one question: the question, what was retrieved, the prompt, the answer, how long each step took, and what it cost. The server adds its checks, one diagnosis, and any feedback.

What's in a trace#

PartWhat it holds
QuestionThe user's question, and the rewritten retrieval query if you log one
RetrievalThe chunks your retriever returned: text, source, rank, scores, metadata
PromptThe prompt sent to the model, its version and template name (optional)
GenerationThe answer, model, provider, parameters, and token counts
TimingTotal time, and time per step when you use the context manager
ProjectProject id, environment
MetadataAnything you pass as metadata=

The server adds the checks it ran (its findings), the diagnosis, the cost (from token counts and its price table) and feedback (thumbs up or down, with a comment). What a trace contains lists every field and where data can go.

Ids and time#

The SDK picks a trace's id before sending it, so you can log it alongside your own records straight away. Sending the same id twice stores one trace, which makes retries safe.

Steps#

Today a trace is one RAG interaction. Agents, which make several model and tool calls per question, need a trace to hold a tree of steps; that's in development, along with accepting traces over OpenTelemetry. The decision behind it is recorded in the repository.