Golden datasets, test cases, and regression testing

🔴 MUST KNOW · Evaluation means measuring whether the application reliably achieves the intended task.

Overview

• A golden dataset is a curated set of representative inputs with expected outcomes, reference evidence, and scoring criteria.
• Regression testing reruns these cases after changes.
• AI tests often allow multiple acceptable phrasings while requiring the same facts, actions, and boundaries.

Why this matters

A few convincing demos cannot reveal systematic mistakes or changes in model behavior.

An everyday example

A driving test checks many road situations, not only whether the car starts once.

A technical example

A case includes question, tenant fixture, expected intent, relevant policy sources, forbidden tools, and whether abstention is required.

How it works

1. Collect anonymized cases
2. Define outcome rubric
3. Split development and held-out sets
4. Run candidate system
5. Score
6. Inspect failure slices
7. Gate deployment.

Architecture and flow

Cases + fixtures + rubrics → versioned run → scores and failure examples → release decision.

The insurance assistant example

Include claim-status lookup, ambiguous ID, unavailable backend, outdated policy, injected PDF, and another tenant's claim.

When to use it

Before production and after model, prompt, parser, tool, or retrieval changes.

When to avoid it

Do not repeatedly tune against the held-out set until it becomes a disguised training set.

Interview answer

“I evaluate expected outcomes and constraints using representative, versioned cases, and rerun them for every material AI-system change.”

Benefits, trade-offs, and production considerations

• Sample enough per risk category; rerun stochastic cases to measure variance where appropriate.
• A high average can conceal a severe privacy failure.
• Keep unit/integration tests for deterministic code alongside behavior evaluations.