Async/await, type hints, and HTTP clients

Overview

• Python async def creates a coroutine function; await lets other work proceed while an awaitable operation waits.
• Type hints describe expected inputs and outputs for developers and type checkers.
• HTTP clients such as HTTPX provide asynchronous requests, connection pooling, and timeout configuration.

Why this matters

AI services spend much of their time waiting for providers, storage, and internal APIs.

An everyday example

A receptionist answers another call while waiting for a department to reply.

A technical example

An async function awaits a claim-service request instead of blocking the whole request loop.

How it works

1. An event loop schedules coroutines
2. Network operations yield while waiting
3. Completed results resume the coroutine
4. Bounded concurrency limits load.

Architecture and flow

FastAPI request → async HTTP client → claim API → validated response.

The insurance assistant example

Concurrently retrieve two independent read-only policy sources, but fetch authorization context before using either result.

When to use it

I/O-heavy endpoints and workers with async-compatible libraries.

When to avoid it

• Async does not make CPU-heavy OCR or local model inference parallel automatically.
• Do not call synchronous blocking clients inside async routes without deliberate offloading.

Interview answer

“Python async improves I/O concurrency. I use async clients, explicit deadlines, reusable pools, and bounded concurrency; CPU work belongs in workers or suitable processes.”

Benefits, trade-offs, and production considerations

• Prefer lifecycle-managed clients over creating a new connection pool per request.
• Propagate cancellation, distinguish connect/read timeouts, and do not create an unbounded gather over a million documents.
• FastAPI concurrency guide.

import httpx

async def get_claim(client: httpx.AsyncClient, claim_id: str) -> dict:
    # Configured client carries trusted service authentication and base URL.
    response = await client.get(f"/claims/{claim_id}", timeout=5.0)
    response.raise_for_status()
    return response.json()  # Validate the response schema before trusting it.