Traces, agent runs, LLM/tool calls, and state transitions
Practice this question🔴 MUST KNOW · Observability explains what happened in real executions; evaluation decides whether that behavior is acceptable.
Overview
- A trace connects the operations serving one request or task.
- Each timed operation is a span, such as an LLM call, retrieval, or tool execution.
- An agent run may include several spans and persisted state transitions across multiple requests.
Why this matters
A final error message does not reveal whether delay or failure came from retrieval, a provider, a queue, or a business service.
An everyday example
A parcel tracking history shows which transfer introduced a delay.
A technical example
Trace request → intent model → claim lookup → answer model → output validation, carrying run and business IDs.
How it works
- Create trace/run correlation
- Start/end spans
- Record safe metadata and errors
- Propagate IDs across queues/services
- Link later approval/resume traces
- Query timelines.
Architecture and flow
API → queue/worker → model/tool spans → trace collector → investigation.
The insurance assistant example
Staff can identify that a delayed answer came from the claims API, not the model.
When to use it
Every production AI path, including ingestion and asynchronous jobs.
When to avoid it
Avoid logging full prompts, documents, or credentials merely because they help debugging.
Interview answer
“I trace model calls, tools, retrieval, and state transitions with shared run IDs, and correlate them with durable business outcomes.”
Benefits, trade-offs, and production considerations
- Separate technical tracing from immutable business audit events.
- Trace sampling may omit runs; a claim approval audit must not.
- Protect identifiers from unauthorized dashboard access.