AI system design
Words made simple
A few useful meanings before you read the explanation.
- AI
- Software techniques for tasks such as recognizing patterns, understanding language or generating content.
- API
- A way for one piece of software to ask another for data or an action. A claims API might return the current status of a claim.
- Token
- A small piece of text processed by a model. It may be a whole word, part of a word, or punctuation.
More words from this lesson
- RAG
- Find relevant information first, then give it to a model to help answer the question. The answer is only as reliable as the information and checks used.
- Agent
- An application that uses a model to choose its next action within defined limits. It can use tools and continue through several steps.
- Deterministic
- Following steps or rules defined by code. The model does not decide which step should run next in a deterministic workflow.
- State
- The data describing where a workflow is now, what it already knows, and what still needs to happen.
- Durable
- Able to retain progress and recover across interruptions. Durable workflows require appropriate storage and retry behavior.
Beginner — What questions come before architecture?
- What task should succeed, which data is authoritative, who may access it, what errors are costly, how fast it must respond, expected load, and available budget.
- Then decide whether deterministic code or AI is needed.
Intermediate — How do you scale agent traffic?
- Estimate provider/tool call amplification, use admission controls and tenant quotas, isolate background work, and scale workers to real dependency capacity.
- More API replicas do not remove provider token limits or serialize conflicting thread updates automatically.
Senior — How do you design degraded behavior?
- Define acceptable alternatives per task: verified templates, search results, paused durable work, or staff escalation.
- Preserve data/permission/quality contracts and distinguish missing evidence, service failure, and unknown commit outcomes explicitly.
Scenario — Design an insurance assistant in two minutes.
- Use Next.js and an authenticated Node API, a bounded intent router, policy RAG, and authorized claim tools.
- Put draft validation, exact approval, and idempotent submission in deterministic services/workflows.
- Add scoped memory, durable state where needed, redacted tracing, source/outcome evaluations, and per-tenant budgets.
- Start with one assistant; justify any extra agents or MCP integration through real requirements.
Can you explain it without the notes?
AI system design
Use your own words. Explain an example and one limitation before checking the explanation.
Check your explanation
Beginner — What questions come before architecture?
Turn this concept into a clear answer.
Practice this conceptAdapted from 13_AI.MD · “29.21 AI system design”. Examples are study fragments; framework behavior can depend on your version.