Cost, multimodal, and deployment
Words made simple
A few useful meanings before you read the explanation.
- Multimodal
- Working with more than one kind of input or output, such as text, images or audio.
- Token
- A small piece of text processed by a model. It may be a whole word, part of a word, or punctuation.
- Embedding
- A list of numbers that represents the meaning of some content. Similar meanings can produce nearby lists of numbers.
More words from this lesson
- Retrieval
- Finding and fetching information that is useful for the current question.
- Streaming
- Sending an answer in small pieces as it is produced, rather than waiting for the complete response.
- Evaluation
- Checking results against examples and clear criteria to find whether the system works well and where it fails.
- Quantization
- Representing model numbers with fewer bits to reduce memory use. Quality and speed depend on the method and hardware.
- GPU
- A processor designed to handle many numerical operations in parallel. It is often used for model training and inference.
Cost formula: input tokens × input rate + output tokens × output rate + tools/retrieval/embedding/infrastructure/review, with applicable caching and provider accounting. Prices must be checked for the real provider/model/account.
Optimization order: use code/templates where adequate → select relevant context → bound output and steps → evaluate smaller routes → safe caching → batching/traffic isolation → capacity tuning. Streaming improves visible progress; it does not automatically reduce total cost.
Multimodal: validate media → parse/OCR/vision/audio → structured observations → evidence/field checks → review → business action. Transcripts and visible damage are proposed observations, not authoritative eligibility or payment decisions.
Local/self-hosted: deployment control and possible utilization economics come with model/runtime security, GPU capacity, license, patching, and operating costs. Quantization saves memory with task-specific quality trade-offs. Fine-tune only after diagnosing a stable behavior gap and building governed training/evaluation data.
Can you explain it without the notes?
Cost, multimodal, and deployment
Use your own words. Explain an example and one limitation before checking the explanation.
Check your explanation
Cost formula: input tokens × input rate + output tokens × output rate + tools/retrieval/embedding/infrastructure/review, with applicable caching and provider accounting. Prices must be checked for the real provider/model/account.
Turn this concept into a clear answer.
Practice this conceptAdapted from 13_AI.MD · “Cost, multimodal, and deployment”. Examples are study fragments; framework behavior can depend on your version.