One search. Connected ideas.

AI and LLM fundamentals

A trace connects the operations serving one request or task. Each timed operation is a span, such as an LLM call, retrieval, or tool execution. An agent run may include several spans and persisted state transitions across multiple requests.

Langfuse and LangSmith provide AI-oriented trace and experiment views. OpenTelemetry standardizes telemetry instrumentation and export across services. Sentry provides application error and performance investigation capabilities; exact AI-specific…

These platforms help collect traces, datasets, scores, and experiment results. LangSmith integrates closely with the LangChain ecosystem; Langfuse provides AI observability and evaluation workflows; Phoenix supports tracing and evaluation analysis. They are…

Latency metrics describe waiting time, including queue delay, time to first token, and total completion time. Token and cost metrics show resource usage. Error metrics classify failed, blocked, incomplete, and unknown-outcome runs.

Illustrative timings and identifiers:

Message roles identify the purpose and source of information. System or developer instructions describe application behavior, user messages contain requests, assistant messages contain model responses, and tool results supply external observations. Exact role…

An AI gateway sits between your application and model providers. It can normalize access, choose approved models, enforce quotas, collect usage, and handle eligible fallback. It is similar to an API gateway specialized for model traffic.

Functions package operations; classes group state and behavior; modules are importable Python files or packages. Exceptions represent errors that interrupt normal control flow until handled. These serve the same broad purposes as functions, classes, imports,…

Vertex AI is the familiar Google Cloud AI platform name used in many existing projects; its model documentation currently routes into evolving Google enterprise agent-platform documentation. Cloud Run executes containerized applications and workloads. Cloud…

Beginner — What do you trace in an AI request?

Provider failover chooses an allowed alternative after eligible failure. Multi-tenancy scopes records and operations; security enforces trust boundaries. Observability connects execution paths to their outcomes and resource costs.

Langfuse and LangSmith provide AI-oriented trace and experiment views. OpenTelemetry standardizes telemetry instrumentation and export across services. Sentry provides application error and performance investigation capabilities; exact AI-specific…

A trace connects the operations serving one request or task. Each timed operation is a span, such as an LLM call, retrieval, or tool execution. An agent run may include several spans and persisted state transitions across multiple requests.

Latency metrics describe waiting time, including queue delay, time to first token, and total completion time. Token and cost metrics show resource usage. Error metrics classify failed, blocked, incomplete, and unknown-outcome runs.

Illustrative timings and identifiers:

AI and LLM fundamentals

Authentication, retrieval, model calls, tool calls, state transitions, validation, and final outcome, linked by run/request IDs. I record timing, usage, and safe metadata without indiscriminately logging private content.

Time to first token measures how soon generation begins producing text, while total latency includes all required work and completion. Streaming improves perceived progress but may not reduce total runtime or cost.

A trace diagnoses execution and performance and may be sampled. A business audit records important actions, actors, decisions, and outcomes with appropriate integrity/retention. I link them but do not let sampled traces replace claim approval records.

Streaming headers may have been sent before an error or disconnect. I inspect terminal events and run/message status rather than HTTP code alone, then check provider completion, validation, cancellation, and persistence spans.

These platforms help collect traces, datasets, scores, and experiment results. LangSmith integrates closely with the LangChain ecosystem; Langfuse provides AI observability and evaluation workflows; Phoenix supports tracing and evaluation analysis. They are…

Message roles identify the purpose and source of information. System or developer instructions describe application behavior, user messages contain requests, assistant messages contain model responses, and tool results supply external observations. Exact role…

A sequence of transformations or calls, such as prompt → model → parser. A chain can be deterministic and is not automatically an agent; an agent chooses some actions based on observations.

AI Engineering learning module

AI Engineering learning module

AI Engineering learning module

AI Engineering learning module

AI Engineering learning module

AI Engineering learning module

AI Engineering learning module

AI Engineering learning module

AI Engineering learning module