Basic NumPy, Pandas, and Jupyter

Overview

• NumPy provides efficient numerical arrays and vector operations.
• Pandas provides table-like data structures for cleaning and analyzing datasets.
• Jupyter provides interactive notebooks that mix code, results, and explanations.

Why this matters

They make it easy to inspect embeddings, prepare evaluation data, and analyze experiment results.

An everyday example

NumPy is a numerical calculator, Pandas is a programmable spreadsheet, and Jupyter is a lab notebook.

A technical example

Use NumPy to compute a similarity score and Pandas to compare error rates by intent.

How it works

1. Load explicit data
2. Validate columns/shapes
3. Compute transformations
4. Inspect outputs
5. Move reusable logic into tested application modules.

Architecture and flow

Anonymized eval records → Pandas analysis → chart/table → engineering decision.

The insurance assistant example

Discover that claim-status requests succeed while exclusion questions frequently lack the correct retrieved clause.

When to use it

Exploration, dataset preparation, and lightweight offline evaluation analysis.

When to avoid it

Do not run production jobs from manually executed notebook cells or load private production dumps into uncontrolled notebooks.

Interview answer

“I use NumPy for numerical operations, Pandas for tabular analysis, and notebooks for experiments; production logic moves into reproducible services and jobs.”

Benefits, trade-offs, and production considerations

• Notebook cells can execute out of order and hide state.
• Restart and run all cells before sharing findings.
• Avoid repeatedly modifying large DataFrames row by row; vectorized operations are usually faster.

import numpy as np
import pandas as pd

a = np.array([1.0, 0.0])
b = np.array([0.8, 0.2])
cosine = float(np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b)))
results = pd.DataFrame([
    {"intent": "claim_status", "passed": True},
    {"intent": "policy_question", "passed": False},
])
print(cosine)
print(results.groupby("intent")["passed"].mean())

• This demonstrates calculations and grouping.
• Two rows cannot establish a reliable quality estimate; use a representative evaluation set and report sample counts.