Observability

Beginner — What do you trace in an AI request?

• Authentication, retrieval, model calls, tool calls, state transitions, validation, and final outcome, linked by run/request IDs.
• I record timing, usage, and safe metadata without indiscriminately logging private content.

Intermediate — Time to first token versus total latency?

• Time to first token measures how soon generation begins producing text, while total latency includes all required work and completion.
• Streaming improves perceived progress but may not reduce total runtime or cost.

Senior — Trace versus audit log?

• A trace diagnoses execution and performance and may be sampled.
• A business audit records important actions, actors, decisions, and outcomes with appropriate integrity/retention.
• I link them but do not let sampled traces replace claim approval records.

Scenario — The API returned 200 but the user got no complete answer. Why?

• Streaming headers may have been sent before an error or disconnect.
• I inspect terminal events and run/message status rather than HTTP code alone, then check provider completion, validation, cancellation, and persistence spans.