AI Engineering
Learn LLMs, RAG, agents, secure tools, and production AI through practical insurance examples.
What determines AI cost?
Input/output usage, model pricing, tool/search calls, embeddings, retries, and infrastructure. Human review and failed tasks also matter. I calculate total cost per completed useful task rather than only one API call.
When is semantic caching unsafe?
When similar requests differ in tenant, user permissions, policy version, date, or personalized facts. I restrict it to suitable stable tasks and include freshness and scope dependencies; exact caches also require proper keys and invalidation.
How do you optimize without hurting quality?
Establish a quality/security baseline, measure the expensive stages, and experiment with routing, context reduction, batching, or caching. Release only when held-out outcomes, tail latency, and privacy requirements remain acceptable.
Five agent steps became fifty after a change. What happens?
A strict step/time/token budget stops the run and reports a supported fallback. I inspect progress and repeated calls, compare prompt/tool changes, and track long-tail cost rather than allowing an expensive loop to continue.
Difficulty is an editorial estimate. Completion means you chose to mark a question as understood.