Project 2 — Streaming AI chatbot
Words made simple
A few useful meanings before you read the explanation.
- AI
- Software techniques for tasks such as recognizing patterns, understanding language or generating content.
- Streaming
- Sending an answer in small pieces as it is produced, rather than waiting for the complete response.
- API
- A way for one piece of software to ask another for data or an action. A claims API might return the current status of a claim.
More words from this lesson
- Token
- A small piece of text processed by a model. It may be a whole word, part of a word, or punctuation.
- JSON
- A common text format for exchanging data using named fields, lists, numbers and values. Example: {"status": "pending"}.
- State
- The data describing where a workflow is now, what it already knows, and what still needs to happen.
- Authentication
- Checking who a user or service is, such as by verifying a login session.
- Evaluation
- Checking results against examples and clear criteria to find whether the system works well and where it fails.
Requirements
Show progress/text incrementally, allow cancellation, and distinguish completion from stream failure. Preserve final message state.
Architecture
Provider events → Node public-event adapter → streamed fetch/SSE → React. The adapter filters internals; React maintains partial text and terminal status.
Technologies
Provider streaming SDK, Web Streams or Node response streams, abort signals, React incremental state.
Database design
Add message state generating/completed/failed/cancelled; use stable request/run IDs. Partial drafts must not appear as final messages.
API design
Stream events with bounded payloads and an explicit terminal event. Define how a client retrieves final status after disconnect.
AI design
Same task as project 1; expose only approved text/progress. Buffer content if output policy requires validation before display.
Security
Authenticate before opening the stream; do not stream raw state, tool args, or private traces. Enforce connection and rate limits.
Evaluation
Midstream failure, disconnected browser, cancellation, provider refusal, proxy buffering, and incomplete generation.
Observability
Time to first public token, total completion, active connections, aborts, stream errors, and usage.
Deployment
Verify hosting/proxy streaming support, buffering, idle/request limits, and safe reconnection behavior.
Interview questions
- “Does streaming reduce cost?” Answer: it mainly improves perceived latency; cost falls only if less work is done.
- “Can you execute streamed JSON?” Answer: wait for completion and validation.
Acceptance: The UI never confuses a partial response with a successful complete result.
Can you explain it without the notes?
Project 2 — Streaming AI chatbot
Use your own words. Explain an example and one limitation before checking the explanation.
Check your explanation
Requirements: Show progress/text incrementally, allow cancellation, and distinguish completion from stream failure. Preserve final message state.
Turn this concept into a clear answer.
Practice this conceptAdapted from 13_AI.MD · “Project 2 — Streaming AI chatbot”. Examples are study fragments; framework behavior can depend on your version.