Tool poisoning and excessive agency

Overview

• Tool poisoning uses malicious tool definitions or descriptions to influence model behavior.
• Excessive agency gives the AI more permissions or autonomy than its task needs.
• A harmless-looking tool name can conceal a sensitive operation or destination.

Why this matters

Dynamic integration ecosystems can introduce capabilities that developers have not reviewed.

An everyday example

A contractor advertises “filing help” but asks for the keys to the entire building.

A technical example

A new MCP tool description tells the agent to read secret configuration before searching policies.

How it works

1. Trust/review server identity and definitions
2. Expose minimal tool set
3. Validate each call
4. Apply independent action policy
5. Monitor changes.

Architecture and flow

Tool catalog → trust review → per-task allowlist → invocation authorization → audit.

The insurance assistant example

A FAQ assistant needs policy search, not claim approval, bank-detail modification, or unrestricted email sending.

When to use it

Any tool-enabled agent, especially dynamic MCP or plugin integrations.

When to avoid it

Do not automatically grant every discovered capability or reuse staff-admin credentials for customer chat.

Interview answer

“I expose only task-appropriate, reviewed tools and enforce policy at execution time. Tool descriptions and annotations are not security controls.”

Benefits, trade-offs, and production considerations

• Pin definitions where practical, review changes, restrict downstream network access, and separate read, draft, and commit capabilities.
• Model-accessible tooling should be narrower than a human administrator's tooling.