Learn AI Engineering.

A practical textbook for building AI applications, agents, and reliable client solutions.

Priority key

AI and LLM fundamentals

Artificial intelligence, or AI, is the broad field of making software perform tasks such as recognizing speech, finding patterns, or understanding language. Machine learning, or ML, learns patterns from examples instead of requiring a rule for every case.…

A large language model, or LLM, is a model trained on large amounts of data to process and generate language. It learns patterns in how text is arranged and can use those patterns for summarization, extraction, explanation, and tool selection. It does not…

Tokens are the small units of input and output processed by a model. A token might be a word, part of a word, punctuation, or another encoded unit. Token counts depend on the tokenizer, language, and content; characters and words are only rough estimates.

A context window is the amount of model input and generated content a model can handle within its documented limits. Context is the particular information you provide for one call: instructions, messages, evidence, and tool results. A large window gives…

Temperature changes how sharply a model favors its higher-scoring next-token choices. Lower values generally reduce variation; higher values allow more variation. Top-p, also called nucleus sampling, restricts selection to a set of tokens covering a chosen…

Message roles identify the purpose and source of information. System or developer instructions describe application behavior, user messages contain requests, assistant messages contain model responses, and tool results supply external observations. Exact role…

Prompt engineering is writing clear task instructions, inputs, constraints, and output expectations. Zero-shot prompting gives instructions without examples; few-shot prompting adds a small number of input/output examples. A prompt is part of the application…

JSON output is machine-readable data, such as an object with named fields. Structured output adds a declared schema so the response follows expected fields and types when the provider supports schema-constrained generation. Valid JSON, valid schema, and valid…

Function calling lets a model propose a named function and structured arguments. Tool calling is the broader pattern that includes functions and capabilities such as search or code execution. A proposed tool call is an untrusted request that your application…

Streaming delivers response events as work happens instead of waiting for the entire response. Text events may contain small pieces called deltas. The browser can show progress and partial text while the server continues processing.

Multimodal models process more than one kind of input or output, such as text with images or audio. Vision means interpreting visual content; audio understanding includes spoken words and sometimes sounds or vocal context. Support differs by model and…

Model selection chooses a model for a task. Routing chooses among models or processing paths per request, while fallback handles a failed or unavailable preferred path. Latency is waiting time; cost includes tokens, tools, retrieval, infrastructure, and…

Providers supply hosted model inference and related capabilities. OpenAI, Anthropic, and Google offer their own model ecosystems; Azure OpenAI provides access to supported OpenAI models within Azure's service and identity environment. Common concepts…

Python is a dynamically typed language widely used by AI libraries. A list is an ordered collection, a dictionary maps keys to values, and a set stores unique hashable values. Python uses indentation to define code blocks rather than braces.

Functions package operations; classes group state and behavior; modules are importable Python files or packages. Exceptions represent errors that interrupt normal control flow until handled. These serve the same broad purposes as functions, classes, imports,…

A virtual environment isolates a project's Python packages. pip installs packages into the selected environment. Project structure separates API routes, business services, adapters, and tests so the AI code does not become one notebook-sized script.

Python async def creates a coroutine function; await lets other work proceed while an awaitable operation waits. Type hints describe expected inputs and outputs for developers and type checkers. HTTP clients such as HTTPX provide asynchronous requests,…

Pydantic validates data against Python model definitions. FastAPI uses declared routes and schemas to build HTTP services with validation and API documentation. Think of Pydantic as a role similar to Zod and FastAPI as a Python web framework with typed…

NumPy provides efficient numerical arrays and vector operations. Pandas provides table-like data structures for cleaning and analyzing datasets. Jupyter provides interactive notebooks that mix code, results, and explanations.

An embedding is a list of numbers representing useful features of content, often its meaning. That list is a vector; its number of entries is its dimension. Semantic similarity means closeness in meaning, such as “car accident” and “vehicle collision,” even…

Similarity measures how close two vectors are according to a selected metric. Cosine similarity compares their direction, effectively asking whether they point in a similar direction regardless of magnitude. Dot product and Euclidean distance are other common…

An embedding model creates vectors; an embedding pipeline manages generation, versions, and storage. An embedding is a derived index representation, not a replacement for the original text. Different models, dimensions, or preprocessing choices can produce…

A vector database stores vectors and supports similarity queries with metadata. Vector search is the numerical operation; semantic search is the application goal of finding meaningfully related content. Existing databases can also add vector indexes, so you…

Keyword search matches words or indexed terms rather than learned meaning. BM25 is a common relevance formula that rewards useful matching terms, accounts for document length, and avoids letting repeated words dominate indefinitely. It is usually implemented…

Hybrid search combines lexical and semantic candidate retrieval. Metadata filtering narrows records by explicit attributes such as tenant, policy version, date, or region. Filtering answers “which records are eligible,” while ranking answers “which eligible…

Reranking is a second ranking stage that examines a short candidate list more carefully. A reranker often reads the query and each candidate together, allowing richer comparison than separate embeddings. It changes candidate order; it cannot recover a…

Pinecone, Qdrant, Weaviate, pgvector, MongoDB Vector Search, Elasticsearch, and OpenSearch provide different ways to store and retrieve vector-backed knowledge. Some are dedicated vector services; others extend databases or search engines you may already…

RAG stands for retrieval-augmented generation. It retrieves relevant external information and provides that information to a model before asking for an answer. It connects language generation to a changing knowledge source without retraining the model.

Ingestion is the pipeline that turns approved source files into searchable records. Parsing extracts text and structure; PDFs may contain actual text, scanned images, tables, or all three. OCR, optical character recognition, converts image-based letters into…

Chunking divides documents into retrieval-sized pieces. Chunk size controls how much content each piece contains; overlap repeats a small amount at boundaries. Metadata identifies the piece's source, permissions, version, and other useful attributes.

Retrieval selects candidate evidence. Context construction chooses which evidence and conversation details the model sees. Prompt construction combines trusted instructions, the user's question, and clearly labeled untrusted source material.

A citation points to evidence supporting a statement. Source attribution identifies the original document, location, and version. The model can suggest references, but the application must ensure they map to actual retrieved sources.

Query rewriting restates a request for search, often resolving references from conversation. Multi-query retrieval searches alternative formulations. Query decomposition splits a compound question into smaller questions that may require separate evidence.

Hybrid RAG uses lexical and semantic retrieval together. Reranking carefully orders the retrieved candidates. Parent-child retrieval searches smaller child chunks but supplies a larger parent section when the child is selected.

Multi-hop RAG retrieves information in dependent steps: one finding determines the next search. Agentic RAG lets a bounded agent choose retrieval tools or decide whether another search is needed. A fixed two-step retrieval workflow can be multi-hop without…

Corrective RAG detects inadequate retrieval and applies a correction, such as reformulating search. Self-reflective RAG checks whether evidence or an answer is sufficient and supported. Context compression reduces retrieved text to the parts useful for the…

Graph RAG retrieves through entities and relationships, such as customer → policy → endorsement → covered asset. It can combine graph traversal, document retrieval, and generated summaries. A graph represents connected facts; it is separate from LangGraph's…

RAG evaluation measures retrieval quality and answer quality independently and together. Fine-tuning changes model behavior through training; long context supplies a large body of content directly; traditional search returns evidence without generating an…

An AI agent is software that uses a model to choose actions toward a goal, observes results, and decides what to do next within application limits. A chatbot describes a conversational interface; it may or may not contain an agent. A workflow defines the…

An agent loop alternates decision, action, and observation. ReAct is a pattern that combines deciding what information/action is needed with executing tools and observing results. “Reasoning” here means useful task planning and decision-making, not a…

Tools are approved external capabilities. Planning breaks a goal into steps; state records current execution facts; memory carries selected information across turns or tasks. Autonomous action means the system can perform some actions without a new user…

Human-in-the-loop means a workflow includes a person who supplies information, reviews a result, or authorizes an action. The system can pause durably while waiting. Human approval is a business event with identity and scope, not a sentence in the prompt.

A router chooses a specialized path based on the request. It can use deterministic rules, a classifier, or an LLM with a bounded enum output. Routing does not automatically require a full agent loop.

A planner proposes steps; an executor performs permitted steps and reports results. Splitting the roles separates deciding from acting. The executor still enforces constraints rather than blindly following the plan.

A supervisor coordinates specialized workers and often combines their outputs. A handoff transfers responsibility for a conversation or task to another specialist. Coordination and transfer are different: the supervisor may retain ownership, while a handoff…

Reflection revises an output after examining it. A critic is a separate checking role that evaluates an output against criteria. Either can be model-based, rule-based, or a combination.

Multi-agent systems involve several model-driven roles cooperating. Hierarchical systems organize those roles into levels of coordinators and workers. Multiple ordinary functions in a workflow are not automatically multiple agents.

LangChain is a framework ecosystem for assembling model-driven applications. Model wrappers, prompt templates, and composable operations make integration easier. A chain is a sequence of transformations or calls; it can be fixed and does not automatically…

A retriever returns candidate documents for a query. A tool exposes an action with a description and argument schema. Structured-output wrappers ask supported models for typed data and help parse the result; these are adapters around the concepts already…

LangChain agent construction combines models, tools, and runtime behavior into an agent interface. Simple code uses direct SDK calls and ordinary functions instead. Both can produce production applications if their reliability and security requirements are…

LangGraph represents execution as a graph of steps sharing explicit state. Nodes do work and edges select the next step; the graph can contain both ordinary code and model decisions. It supports applications that need branching, repeated tool use, and…

State is the structured data shared by graph steps, such as current intent, evidence IDs, and task phase. A reducer specifies how a new update combines with an existing state field. Some fields replace earlier values; others safely combine multiple updates.

A node is a function performing one graph step. An edge describes the next step, while a conditional edge selects from allowed destinations based on state. Routing can use model output, deterministic checks, or both.

This is a minimal orchestration demonstration, not an insurance backend. Dependencies are @langchain/langgraph and zod. It uses the documented StateSchema style; older tutorials may use Annotation.Root. Select compatible package versions before adopting…

A checkpoint records execution progress and state at a recovery boundary. Persistence saves it outside the process. Memory is selected information reused by later turns or tasks; a checkpoint is not automatically good long-term user memory.

An interrupt pauses graph execution and exposes a payload for external input. Resumption supplies the requested value. The node containing the interrupt can run again from its beginning on resume, so code before the pause may repeat.

A graph can separate model decisions, tool execution, and stop checks into different nodes. The model node proposes calls; the tool node uses approved business services. Conditional routing returns to the model only while permitted progress remains.

A subgraph is a smaller graph used as a component of a larger one. A multi-agent graph assigns model-driven responsibilities to separate nodes or subgraphs. Subgraphs support modularity even when no multi-agent behavior is involved.

Graph streaming exposes progress such as state updates, node events, and model token events. It lets a UI distinguish “retrieving,” “waiting for approval,” and “complete.” Stream modes and event APIs vary by version.

Error handling distinguishes recoverable infrastructure failures, invalid user input, unsupported model decisions, and business failures. Retry repeats an operation after an appropriate transient failure. Recovery uses persisted progress and reconciliation to…

A production graph sits within an authenticated application, durable execution service, approved tools, checkpoint store, and monitoring system. LangChain supplies components; LangGraph controls stateful execution. A graph can be deterministic, agentic, or…

The OpenAI Agents SDK provides agent-oriented building blocks for model calls, tools, handoffs, and run management. It offers a higher-level agent interface than manually processing each API response. SDK features and provider integration details depend on…

Microsoft Agent Framework, or MAF, provides agent abstractions and explicit workflows. An agent handles model-driven interaction and tool use; a workflow connects agents and ordinary functions through controlled paths. Microsoft describes it as the successor…

Semantic Kernel is a Microsoft ecosystem for integrating models, plugins/functions, and agent capabilities into applications. AutoGen is known for multi-agent conversation and orchestration abstractions. Their historical APIs remain relevant in existing…

CrewAI organizes agent work around roles, tasks, and orchestration constructs such as flows. Google ADK is an agent development toolkit with agent and workflow-oriented components. Both provide reusable abstractions; neither makes a multi-agent design…

A deterministic workflow follows explicitly coded transitions for given inputs and recorded external outcomes. It can call nondeterministic services, but its orchestration rules remain controlled. It is the natural extension of validation, transactions, and…

An agentic workflow allows a model to choose some steps based on observations. The surrounding application limits the actions, resources, and impact of those choices. It is useful for variable tasks but offers less predictable paths and call counts.

A hybrid workflow combines deterministic control with model-driven interpretation or planning. A state machine defines named states and permitted transitions. The state machine constrains what can happen even when a model proposes something else.

A state machine models valid states and transitions. A DAG, directed acyclic graph, models dependencies without loops. Event-driven workflows react to events such as document.uploaded or review.approved instead of doing all work inside one HTTP request.

Long-running workflows persist progress while waiting for jobs, timers, or people. Retries repeat transiently failing work; timeouts cap waits; schedules start work at defined times. Durable waiting should not consume an open request or a sleeping application…

Idempotency makes repeating the same logical request avoid repeated effects. Compensation is a business action that offsets an earlier action when later steps fail. Human approvals are authenticated events permitting specific transitions.

Workflow engines and job systems provide different guarantees for dispatch, persistence, recovery, waiting, and coordination. Temporal is a durable workflow platform; BullMQ is a Redis-backed job library; Inngest and Trigger.dev offer developer-oriented…

MCP is a protocol for connecting AI applications to external tools and context. The host is the application using the model; a client is its protocol connection to a server; a server exposes supported capabilities. The model normally proposes operations,…

A tool is a named executable capability with argument metadata and a schema. Discovery lets a client obtain the server's available tool descriptions. A tool description helps the model choose a capability; it is not proof that the server or tool is…

Resources expose readable context through identified locations, such as policy documents. Prompts expose reusable task templates and arguments. Tools perform operations; resources provide content; prompts help compose a task.

Authentication establishes who a client/user is; authorization determines what that identity may do. MCP connection authorization and downstream business authorization are separate concerns. A valid connection does not automatically permit every customer's…

REST is an HTTP API design style for application services. Function calling is a model interaction mechanism for proposing structured operations. MCP standardizes integration discovery and invocation/context exchange; a plugin is a packaging/integration…

This example defines server-side contracts and dispatch logic, not a particular MCP SDK's registration syntax. Attach this registry to the selected SDK's authenticated request handlers, implement capability discovery, and encode its documented result…

Context is what the model sees now. Short-term memory is task/session information retained across nearby steps. Conversation memory is selected dialogue history or a summary used to maintain continuity.

Long-term memory stores selected information beyond one task. Semantic memory contains facts or preferences; episodic memory records past events; user-profile memory contains explicit attributes such as preferred language. These categories overlap and should…

State persistence stores task progress; checkpoints record recovery positions. Neither automatically means user preference memory. A durable claim workflow may need execution records even when conversational history is deleted under a different retention…

Summarization compresses earlier interaction into a shorter record. Memory retrieval selects relevant stored entries, sometimes through vectors and metadata. Both reduce context volume but can lose or distort information.

Prompt injection is an attempt to make a model follow an attacker's instructions instead of the intended task. Indirect injection places those instructions in external content, such as a PDF, webpage, email, or tool result. Jailbreaking generally attempts to…

Tool poisoning uses malicious tool definitions or descriptions to influence model behavior. Excessive agency gives the AI more permissions or autonomy than its task needs. A harmless-looking tool name can conceal a sensitive operation or destination.

Data leakage is unauthorized disclosure. PII means personally identifiable information, such as identity-linked names, contact details, and account identifiers. Sensitive business or health information and credentials need protection even when they do not fit…

Authentication verifies identity. Authorization decides allowed objects and actions. Least privilege grants only required capabilities; tenant isolation keeps one organization's data and operations separated from others.

Input validation checks permitted shapes and values; output validation checks generated data before use. Rate limiting constrains requests or resources by user, tenant, and operation. Secure tool services mediate database/API access through known business…

Input guardrails check requests and incoming content. Output guardrails check generated answers or data. Tool guardrails check proposed operations before execution and validate their results afterward.

Schema validation verifies data shape, types, required fields, and declared constraints at runtime. Zod is commonly used in TypeScript; Pydantic serves a similar role in Python. These validators check structure, not whether facts are true or actions are…

Content moderation categorizes content that may violate a product policy. PII detection identifies potentially private identity-linked information for masking, routing, or restricted handling. Both can use rules and models, and neither is perfectly accurate.

Hallucination mitigation reduces unsupported model claims using evidence, constraints, validation, and abstention. Business-rule validation checks authoritative domain requirements. Human approval permits a specific action after a person reviews its concrete…

A golden dataset is a curated set of representative inputs with expected outcomes, reference evidence, and scoring criteria. Regression testing reruns these cases after changes. AI tests often allow multiple acceptable phrasings while requiring the same…

Precision measures how many retrieved items are relevant. Recall measures how much relevant evidence was retrieved. Relevance indicates usefulness for the query; ranking metrics also consider whether good evidence appears early enough to be used.

Faithfulness asks whether the answer is supported by supplied evidence. Correctness asks whether it is right for the task and real applicable facts. Citation quality checks whether references point to supporting sources, not merely existing documents.

Tool-call accuracy measures whether the right allowed tool and arguments were selected. Task completion measures whether the intended business outcome actually occurred. Final text claiming success does not establish completion.

LLM-as-a-judge uses a model to grade an answer against a rubric. Human evaluation uses reviewers with defined criteria. Both produce judgments that need calibration and can have bias or inconsistency.

These platforms help collect traces, datasets, scores, and experiment results. LangSmith integrates closely with the LangChain ecosystem; Langfuse provides AI observability and evaluation workflows; Phoenix supports tracing and evaluation analysis. They are…

A trace connects the operations serving one request or task. Each timed operation is a span, such as an LLM call, retrieval, or tool execution. An agent run may include several spans and persisted state transitions across multiple requests.

Latency metrics describe waiting time, including queue delay, time to first token, and total completion time. Token and cost metrics show resource usage. Error metrics classify failed, blocked, incomplete, and unknown-outcome runs.

Langfuse and LangSmith provide AI-oriented trace and experiment views. OpenTelemetry standardizes telemetry instrumentation and export across services. Sentry provides application error and performance investigation capabilities; exact AI-specific…

Illustrative timings and identifiers:

Token optimization removes unnecessary model input/output while preserving task information. Prompt compression shortens instructions or evidence; context trimming selects only relevant history and sources. Shorter is useful only if important conditions…

Smaller models often reduce cost/latency but may perform worse on complex tasks; stronger models may justify higher cost for difficult reasoning or synthesis. Routing sends different tasks to appropriate paths. The right split comes from evaluation, not model…

Exact caching reuses results for the same normalized request and dependencies. Provider prompt caching can reduce processing/billing for supported repeated prefixes. Semantic caching reuses results for similar meanings, which is more risky because similar…

Batch processing groups noninteractive tasks; bounded concurrency controls simultaneous requests; rate limits constrain traffic over time. Streaming exposes progress earlier. Fallback uses an approved alternative after eligible failures.

An AI gateway sits between your application and model providers. It can normalize access, choose approved models, enforce quotas, collect usage, and handle eligible fallback. It is similar to an API gateway specialized for model traffic.

LiteLLM provides model-access abstraction and proxy/gateway capabilities. Portkey provides an AI gateway and operational controls. Cloud gateways can apply access and model traffic policies within an existing cloud API-management environment; exact AI…

Vision models interpret image content, such as objects, layout, handwriting, and visual relationships. Image understanding produces descriptions or structured observations; it does not automatically verify authenticity or determine business liability. Quality…

Document AI combines text extraction, OCR, layout/table understanding, and field extraction. A PDF may already contain usable text or require image interpretation. OCR reads characters; document understanding connects them to fields and structure.

Speech-to-text converts spoken words into text. Text-to-speech produces spoken output from text. Audio understanding can also process nontext information in audio, depending on the model, but reliable interpretation of emotions or identity should not be…

A voice agent handles a live conversation, including listening, responding, interruption, and tools. Video understanding combines information from sampled frames, timestamps, and sometimes audio. Both involve timing and event coordination beyond an ordinary…

Local or self-hosted models run on infrastructure you control instead of a provider's shared inference API. An API model outsources much of inference operation. Self-hosting can mean a laptop, on-premises servers, or controlled cloud GPU infrastructure.

Ollama simplifies running supported models locally through a runtime and API. Hugging Face hosts models, datasets, and libraries; Transformers is its widely used library for loading and using supported model architectures. A model repository is a distribution…

vLLM is an inference-serving system designed for efficient model execution. Serving techniques such as batching and attention-memory management help multiple requests share expensive hardware. GPU optimization is matching workload, memory, scheduling, and…

Quantization represents model values with fewer bits to reduce memory and sometimes improve speed. GGUF is a model file format used by compatible local inference runtimes; it is not itself a model or a guarantee of quantization level. Local embeddings run an…

Fine-tuning trains an existing model further using task-specific examples. It changes learned behavior rather than simply adding a document to the current prompt. It can target formatting, domain task performance, style, or consistent instruction following.

PEFT means parameter-efficient fine-tuning: update a smaller subset of parameters instead of the whole model. LoRA trains small low-rank adapter matrices attached to selected model operations. QLoRA combines adapter training with a quantized frozen base model…

Dataset preparation converts representative examples into the selected training format with reviewed expected outputs. Evaluation compares trained behavior against the baseline on separate data. RAG supplies current external knowledge at runtime; fine-tuning…

Azure OpenAI provides supported OpenAI model access through Azure's service environment. The platform historically called Azure AI Foundry is documented under Microsoft Foundry in the checked official pages. It groups model and AI application capabilities; it…

Azure AI Search provides searchable indexes and retrieval capabilities. Functions runs event-driven code; Container Apps runs containerized services/jobs. Key Vault manages secrets/keys, while Entra ID provides identity capabilities and access integration.

Bedrock provides managed access to supported foundation models and related capabilities. Lambda runs event-driven functions; ECS orchestrates containers; S3 stores objects such as original PDFs. OpenSearch provides search infrastructure, while IAM controls…

Vertex AI is the familiar Google Cloud AI platform name used in many existing projects; its model documentation currently routes into evolving Google enterprise agent-platform documentation. Cloud Run executes containerized applications and workloads. Cloud…

flowchart TD

A chatbot exposes a conversational model through your app. It may perform no tools or retrieval. Ordinary conversation alone does not make it an agent.

A streaming chatbot shows safe response pieces and progress as they arrive. Final completion remains a separate event.

A RAG chatbot answers conversations using retrieved knowledge. The dialogue interface and retrieval pipeline are separate responsibilities.

AI search often uses semantic or hybrid retrieval to return relevant results. Generation is optional. A result list can be sufficient.

A summarizer condenses supplied content while preserving the requested facts. It transforms evidence rather than creating new authoritative facts.

An extractor converts unstructured text/media into defined fields. Unknown fields should remain unknown instead of being guessed.

A classifier selects a category for input. An LLM classifier can handle varied wording with a bounded output label.

A support agent combines dialogue with safe tools and knowledge search to resolve requests. It may use a bounded loop or explicit graph.

A CRM assistant helps staff query customer records, summarize interactions, and prepare updates. It needs staff-scoped data access and explicit write policy.

An email assistant classifies, summarizes, searches, or drafts email. Sending is a separate consequential operation. Incoming email content is untrusted.

AI workflow automation embeds model tasks inside an ordinary business process. Python/FastAPI adds a service boundary for Python-specific parsing, numerical work, or model execution when needed. Node remains capable of hosted model calls and orchestration.

Atlas is MongoDB's managed database environment; MongoDB Vector Search indexes supported vector fields for similarity retrieval. Source metadata and application records can live alongside chunk documents. A vector/search index is distinct from ordinary…

MongoDB collections can store conversation messages, task/run state, approved preferences, and memory records. A LangGraph checkpointer requires its expected integration contract rather than an arbitrary collection called memory. Business claims remain…

Multi-tenancy supports several organizations while preventing cross-organization access. MongoDB often fits flexible document records; PostgreSQL provides relational constraints, joins, transactions, and optional row-level security with pgvector. Both require…

Scalability handles growing load. Availability means the system is accessible when needed; reliability means tasks produce correct, durable outcomes despite failures. Latency measures waiting, including queue time and user-visible completion.

Queues decouple arrival from processing. Caches reduce repeated eligible work. Business databases store authoritative records; vector indexes store derived retrieval representations. Each has a distinct consistency and recovery contract.

Provider failover chooses an allowed alternative after eligible failure. Multi-tenancy scopes records and operations; security enforces trust boundaries. Observability connects execution paths to their outcomes and resource costs.

A conversational interface over an eligible knowledge corpus. It combines a document pipeline with a request-time answer pipeline.

A bounded assistant for knowledge questions, account lookups, clarification, and escalation. It may perform approved low-risk actions.

A claim-preparation assistant inside an authoritative claim workflow. It gathers and drafts information without independently adjudicating claims.

A staff assistant that summarizes interactions and proposes CRM updates or follow-ups. Its tools obey staff role permissions.

An asynchronous pipeline turns uploads into validated extracted data and search indexes. It is primarily a workflow with AI stages.

A mailbox-aware assistant for triage, search, summaries, and reviewed outgoing messages. Sending is a controlled tool.

A bounded system gathers external evidence and produces a cited research draft. Its output is decision support, not verified universal truth.

Several specialized model-driven components cooperate under a controlled coordinator. Each has a measurable task and narrow context.

A permission-aware assistant over many organizational sources and users. It requires identity-linked document access, freshness, and trustworthy source references.

An insurance assistant helps authenticated customers understand documents, check claim status, prepare submissions, and reach staff. It combines RAG, exact service lookups, and bounded agent behavior. It does not become the authoritative policy or…

Agentic portion: interpreting wording, gathering missing information, and choosing among bounded read/search actions. Workflow portion: identity, required fields, approvals, transaction order, retries, and durable outcomes. LangGraph can coordinate the…

Never accept a model's customerId as proof of the current customer. Never let the model supply an unrestricted email URL, SQL query, database filter, or role field.

Enforce unique operation keys and optimistic draft versions. Keep large original files in protected object storage. Keep model-visible projections small. A document index and conversation cache are rebuildable derived data; claim and approval records need…

Apply CSRF protection where cookie-authenticated mutation routes require it, enforce origin/authentication policy, and use appropriate error semantics. A pending task may return 202; a chat sentence claiming success cannot substitute for a claim receipt.

A. “Check claim C42.”

A production reference design specifies failed and uncertain outcomes as carefully as successful ones. Evaluation checks behavior before release; observability detects failures in actual runs. Human escalation is an explicit supported outcome.

Build in this order. Each project extends a capability, not necessarily a separate repository. These are future learning projects; the notes remain in this one file. Move forward when you can explain the trade-offs and demonstrate the acceptance criteria,…

Requirements: Show progress/text incrementally, allow cancellation, and distinguish completion from stream failure. Preserve final message state.

Requirements: Upload approved PDFs, track ingestion, ask questions, return source references, and abstain when evidence is missing.

Requirements: Multi-tenant retrieval, incremental updates/deletions, policy versions, hybrid search, evaluated reranking, and source-aware caching.

Requirements: Read claim status, lookup policy, retrieve instructions, ask for missing IDs, and escalate. Start read-only.

Requirements: Convert the assistant into an explicit state graph, add durable progress, clarification/approval pauses, and safe resume.

Requirements: Add specialists only for measurable policy/claim/document tasks, with clear coordinator ownership and escalation.

Requirements: Expose reviewed customer lookup, order/policy lookup, ticket creation, and knowledge search to an approved host.

Requirements: Interpret input, prepare a draft, validate required fields, wait for review, execute once, and notify reliably.

Requirements: Combine policy RAG, claim status, guided intake, document processing, reviewed communication, staff escalation, and multi-tenant operations. Define supported tasks and explicit exclusions.

Practice saying the answer clearly, then explain the insurance example and the trade-off. A strong answer identifies the authoritative data, safe failure, and measurement—not only the framework name.

Beginner — What makes a good prompt?

Beginner — What is RAG?

Beginner — Explain an embedding.

Beginner — Why use a vector index?

Beginner — What does LangChain provide?

Beginner — What are nodes, edges, and state?

Beginner — Agent versus chatbot?

Beginner — What problem does MCP solve?

Beginner — What is MAF?

Beginner — Deterministic versus agentic workflow?

Beginner — Context versus memory?

Beginner — What is prompt injection?

Beginner — What is a guardrail?

Beginner — Why isn't normal unit testing enough?

Beginner — What do you trace in an AI request?

Beginner — What determines AI cost?

Beginner — Why learn Python as a MERN engineer?

Beginner — What is FastAPI's role?

Beginner — Where does the LLM call belong?

Beginner — What questions come before architecture?

A decision framework is a repeatable sequence of questions that turns a requirement into justified architecture. Each added capability should solve a concrete gap. It prevents treating every client request as an opportunity to deploy every AI framework.

Ask what the user actually needs and what the existing system already knows. Exact lookups, arithmetic, validation, and known transitions usually belong in normal code. AI is useful when language/media variation or learned pattern recognition provides value…

Use an LLM for language transformation, interpretation, synthesis, and bounded proposals. A dedicated classifier, OCR engine, search index, or template may solve the narrower task better. Define acceptable uncertainty and what happens when the model cannot…

Ask whether the answer depends on external knowledge and whether the corpus is too large or variable to supply directly. Use RAG for private/changing knowledge with source references; use exact tools for live records. If a handful of permitted documents are…

Use narrow tools for fresh data and actions. MCP is useful when approved hosts need a shared discoverable integration, not merely because the model has one function call. Keep the existing business API behind either a direct wrapper or the MCP facade.

If next steps are fixed, use a sequence of functions or a workflow. If next information actions genuinely vary, a bounded agent may help. Use LangGraph when explicit state, branching, review pauses, or recovery justify it; a short read-only loop can remain…

Yes when steps depend on durable business state, external events, review, timers, compensation, or repeated retries. Distinguish conversational agent state from the canonical business process. Choose a queue for independent jobs and a durable workflow…

Determine impact, ambiguity, consent, reversibility, and organizational authority. Approval must show the concrete action and bind to its version/hash, actor, scope, and expiry. Recheck current rules at execution; a long wait can invalidate earlier…

Map every trust boundary: UI, uploads, parser, retrieval, memory, provider, tools, business APIs, caches, and traces. Minimize information and authority, enforce tenant/object/action checks, constrain egress, validate schemas and rules, and test adversarial…

Define success, severe failures, and risk-based datasets before optimizing prompts. Separate retrieval from answer checks and agent decisions from actual business outcomes. Include missing information, stale evidence, injections, denied access, retries, and…

Trace calls and state transitions, correlate across services/queues, record usage and evidence IDs, and link to business receipts. Audit important actions separately with authenticated actors and durable outcomes. Redact sensitive content and set appropriate…

Count every model/tool call, embedding/reranking, infrastructure, retries, and review workload. Optimize cost per successful task subject to quality and privacy constraints. Use context selection, templates, smaller models, caching, batching, and strict agent…

Estimate request rate and model/tool amplification, provider quotas, concurrency, history size, corpus updates, and review arrival rate. Scale stateless APIs, separate ingestion from chat, bound queues, isolate noisy tenants, and serialize conflicting updates…

Worked decision: “Automatically approve claims from uploaded photos.” First separate observation from adjudication. Vision can identify proposed observations, OCR can extract document fields, and RAG can retrieve applicable terms. Approval requires…

Stable concepts, framework concepts, and changing APIs

For a client, write: Outcome and scope; users and permissions; authoritative data; ordinary-code baseline; AI responsibilities; controlled workflow and approvals; tools/data contracts; failure and reconciliation policy; evaluations; telemetry/audit;…

Use this after studying the full explanations. Spend 10 minutes on vocabulary, 10 on RAG and agents, 10 on security/workflows/evaluation, and 10–30 on the insurance design and scenario answers.

Ingestion: approved source → secure parse/OCR → coherent chunks + metadata → embeddings → complete published index.

LangGraph essentials: nodes return state updates; reducers define merge behavior; edges control next steps; model routes need validation; production needs durable checkpoint storage and authorized threads. An interrupted node can restart, so pre-pause side…

- Deterministic: known rules and transitions. Best for permission, arithmetic, approvals, and business commits.

Secure action flow: LLM proposal → allowlisted tool → identity/authorization → schema + rules → limits → business service → minimal result.

Evaluate no evidence, stale policies, malformed documents, ambiguous dates, denied access, injected content, restarts, duplicate submissions, and uncertain writes. Track source/model/prompt/graph versions, p95 latency, tokens, cost per successful task,…

Cost formula: input tokens × input rate + output tokens × output rate + tools/retrieval/embedding/infrastructure/review, with applicable caching and provider accounting. Prices must be checked for the real provider/model/account.

“Next.js provides the interface and Node authenticates, validates, and applies budgets. A bounded router selects policy RAG, exact claim lookup, guided intake, or staff escalation. RAG returns applicable policy evidence with citations; tools enforce…

What successful outcome does the client need, and can ordinary code provide it?