Traces, agent runs, LLM/tool calls, and state transitions
Words made simple
A few useful meanings before you read the explanation.
- LLM
- A model trained to work with language. It can write or summarize text, but its answer still needs checking against reliable information.
- Agent
- An application that uses a model to choose its next action within defined limits. It can use tools and continue through several steps.
- State
- The data describing where a workflow is now, what it already knows, and what still needs to happen.
More words from this lesson
- Trace
- A linked record of the steps in one request, including model calls, retrieval and tools. It helps locate delays or failures.
- AI
- Software techniques for tasks such as recognizing patterns, understanding language or generating content.
- API
- A way for one piece of software to ask another for data or an action. A claims API might return the current status of a claim.
- Retrieval
- Finding and fetching information that is useful for the current question.
- Persistence
- Saving information so it remains available after a request ends or a process restarts.
🔴 MUST KNOW · Observability explains what happened in real executions; evaluation decides whether that behavior is acceptable.
Overview
- A trace connects the operations serving one request or task.
- Each timed operation is a span, such as an LLM call, retrieval, or tool execution.
- An agent run may include several spans and persisted state transitions across multiple requests.
Why this matters
A final error message does not reveal whether delay or failure came from retrieval, a provider, a queue, or a business service.
An everyday example
A parcel tracking history shows which transfer introduced a delay.
A technical example
Trace request → intent model → claim lookup → answer model → output validation, carrying run and business IDs.
How it works
- Create trace/run correlation
- Start/end spans
- Record safe metadata and errors
- Propagate IDs across queues/services
- Link later approval/resume traces
- Query timelines.
Architecture and flow
API → queue/worker → model/tool spans → trace collector → investigation.
The insurance assistant example
Staff can identify that a delayed answer came from the claims API, not the model.
When to use it
Every production AI path, including ingestion and asynchronous jobs.
When to avoid it
Avoid logging full prompts, documents, or credentials merely because they help debugging.
Interview answer
“I trace model calls, tools, retrieval, and state transitions with shared run IDs, and correlate them with durable business outcomes.”
Benefits, trade-offs, and production considerations
- Separate technical tracing from immutable business audit events.
- Trace sampling may omit runs; a claim approval audit must not.
- Protect identifiers from unauthorized dashboard access.
Can you explain it without the notes?
Traces, agent runs, LLM/tool calls, and state transitions
Use your own words. Explain an example and one limitation before checking the explanation.
Check your explanation
A trace connects the operations serving one request or task. Each timed operation is a span, such as an LLM call, retrieval, or tool execution. An agent run may include several spans and persisted state transitions across multiple requests.
Turn this concept into a clear answer.
Practice this conceptKeep connecting the ideas.
Next lesson: Latency, tokens, cost, errors, and service objectivesAdapted from 13_AI.MD · “Traces, agent runs, LLM/tool calls, and state transitions”. Examples are study fragments; framework behavior can depend on your version.