AI Engineering
Learn LLMs, RAG, agents, secure tools, and production AI through practical insurance examples.
What questions come before architecture?
What task should succeed, which data is authoritative, who may access it, what errors are costly, how fast it must respond, expected load, and available budget. Then decide whether deterministic code or AI is needed.
How do you scale agent traffic?
Estimate provider/tool call amplification, use admission controls and tenant quotas, isolate background work, and scale workers to real dependency capacity. More API replicas do not remove provider token limits or serialize conflicting thread updates…
How do you design degraded behavior?
Define acceptable alternatives per task: verified templates, search results, paused durable work, or staff escalation. Preserve data/permission/quality contracts and distinguish missing evidence, service failure, and unknown commit outcomes explicitly.
Design an insurance assistant in two minutes.
Use Next.js and an authenticated Node API, a bounded intent router, policy RAG, and authorized claim tools. Put draft validation, exact approval, and idempotent submission in deterministic services/workflows. Add scoped memory, durable state where needed,…
The decision mindset
A decision framework is a repeatable sequence of questions that turns a requirement into justified architecture. Each added capability should solve a concrete gap. It prevents treating every client request as an opportunity to deploy every AI framework.
Do I need AI? Can deterministic code solve it?
Ask what the user actually needs and what the existing system already knows. Exact lookups, arithmetic, validation, and known transitions usually belong in normal code. AI is useful when language/media variation or learned pattern recognition provides value…
Do I need an LLM?
Use an LLM for language transformation, interpretation, synthesis, and bounded proposals. A dedicated classifier, OCR engine, search index, or template may solve the narrower task better. Define acceptable uncertainty and what happens when the model cannot…
Do I need RAG?
Ask whether the answer depends on external knowledge and whether the corpus is too large or variable to supply directly. Use RAG for private/changing knowledge with source references; use exact tools for live records. If a handful of permitted documents are…
Do I need tools? Do I need MCP?
Use narrow tools for fresh data and actions. MCP is useful when approved hosts need a shared discoverable integration, not merely because the model has one function call. Keep the existing business API behind either a direct wrapper or the MCP facade.
Do I need an agent? Do I need LangGraph?
If next steps are fixed, use a sequence of functions or a workflow. If next information actions genuinely vary, a bounded agent may help. Use LangGraph when explicit state, branching, review pauses, or recovery justify it; a short read-only loop can remain…
Should this be a workflow?
Yes when steps depend on durable business state, external events, review, timers, compensation, or repeated retries. Distinguish conversational agent state from the canonical business process. Choose a queue for independent jobs and a durable workflow…
Where is human approval required?
Determine impact, ambiguity, consent, reversibility, and organizational authority. Approval must show the concrete action and bind to its version/hash, actor, scope, and expiry. Recheck current rules at execution; a long wait can invalidate earlier…
How do I secure it?
Map every trust boundary: UI, uploads, parser, retrieval, memory, provider, tools, business APIs, caches, and traces. Minimize information and authority, enforce tenant/object/action checks, constrain egress, validate schemas and rules, and test adversarial…
How do I evaluate it?
Define success, severe failures, and risk-based datasets before optimizing prompts. Separate retrieval from answer checks and agent decisions from actual business outcomes. Include missing information, stale evidence, injections, denied access, retries, and…
How do I observe it?
Trace calls and state transitions, correlate across services/queues, record usage and evidence IDs, and link to business receipts. Audit important actions separately with authenticated actors and durable outcomes. Redact sensitive content and set appropriate…
How much does it cost?
Count every model/tool call, embedding/reranking, infrastructure, retries, and review workload. Optimize cost per successful task subject to quality and privacy constraints. Use context selection, templates, smaller models, caching, batching, and strict agent…
How does it scale?
Estimate request rate and model/tool amplification, provider quotas, concurrency, history size, corpus updates, and review arrival rate. Scale stateless APIs, separate ingestion from chat, bound queues, isolate noisy tenants, and serialize conflicting updates…
Real client decisions
Worked decision: “Automatically approve claims from uploaded photos.” First separate observation from adjudication. Vision can identify proposed observations, OCR can extract document fields, and RAG can retrieve applicable terms. Approval requires…
Difficulty is an editorial estimate. Completion means you chose to mark a question as understood.