AI Engineering
Learn LLMs, RAG, agents, secure tools, and production AI through practical insurance examples.
What do you trace in an AI request?
Authentication, retrieval, model calls, tool calls, state transitions, validation, and final outcome, linked by run/request IDs. I record timing, usage, and safe metadata without indiscriminately logging private content.
Time to first token versus total latency?
Time to first token measures how soon generation begins producing text, while total latency includes all required work and completion. Streaming improves perceived progress but may not reduce total runtime or cost.
Trace versus audit log?
A trace diagnoses execution and performance and may be sampled. A business audit records important actions, actors, decisions, and outcomes with appropriate integrity/retention. I link them but do not let sampled traces replace claim approval records.
The API returned 200 but the user got no complete answer. Why?
Streaming headers may have been sent before an error or disconnect. I inspect terminal events and run/message status rather than HTTP code alone, then check provider completion, validation, cancellation, and persistence spans.
Difficulty is an editorial estimate. Completion means you chose to mark a question as understood.