#187 — Your AI agents are only as good as the tools you give them

Fewer than you think. The sweet spot for most production agents is 5–15 well-scoped tools rather than 50+ granular ones. Agents have limited context windows — every tool description you load competes for space with actual task information. Teams that consolidate overlapping tools (e.g., replacing list_users + list_events + create_event with a single schedule_event tool) consistently see fewer wrong tool calls, lower token spend, and faster task completion. Start with your 3–5 highest-impact workflows and scale from there.

The most common culprits are ambiguous naming and overlapping tool scope. When two tools share a similar purpose without clear namespace separation, agents guess — and often guess wrong. Fix it by namespacing by service and resource type (e.g., slack_messages_search vs. asana_tasks_search), and audit whether any two tools could plausibly answer the same user request. If they can, consolidate or differentiate. One real-world example: a web search tool was causing degraded results because the agent was appending 2025 to every query — a subtle bias only caught by reading raw call transcripts, fixed with a single description tweak.

Rewrite your tool descriptions. It sounds trivial — it isn't. Tool descriptions load directly into agent context and steer every decision the model makes. Treating them like prompts, not documentation (making implicit context explicit, adding concrete input examples, eliminating parameter ambiguity) has driven benchmark-level accuracy improvements without any model changes. The key step most teams skip: run a structured eval before and after each description change so you can measure the delta instead of guessing.

Start with 10–20 realistic, multi-step tasks grounded in your actual workflows — not toy examples. A strong eval task looks like: "Customer 9182 was charged three times for one purchase — find all relevant logs and check if other customers were affected." A weak one looks like: "Search logs for customer_id=9182." Pair each task with a verifiable outcome (string match or LLM-as-judge), run them in a programmatic while-loop via direct API calls, and track accuracy, token usage, tool call count, and error rates per parameter — not just whether the agent got the right answer.

Yes — and you should make it explicit in your system prompt. Instructing agents to output reasoning and feedback blocks before each tool call (rather than just the call itself) triggers CoT behaviors that measurably increase task performance across all major frontier models. It also gives you a diagnostic layer: you can read the reasoning transcripts to understand why the agent made specific tool choices, not just what it did. This is often where you catch subtle misunderstandings of tool purpose or parameter format that never surface in accuracy scores alone.

Add a response_format parameter to your tools. A real-world Slack tool implementation produced 206 tokens on "detailed" and just 72 tokens on "concise" — identical downstream capability at one-third the cost. Beyond that: implement pagination and filtering on any tool that could return large datasets, cap response sizes at your runtime's default limit (most major API environments impose one), and instruct agents explicitly to make many small targeted queries rather than one broad sweep. Token efficiency is a design decision, not an afterthought.

Because UUIDs and alphanumeric identifiers are essentially random strings to an LLM — they carry no semantic signal. Replacing arbitrary UUIDs with human-readable names or 0-indexed IDs in tool responses significantly reduces hallucinations in retrieval tasks. Return name, file_type, and image_url — not uuid, mime_type, and 256px_image_url. When downstream tool calls genuinely require technical IDs (e.g., send_message(id=12345)), expose a response_format enum so agents can fetch the detailed version only when needed.

The contract is different. A regular API wrapper is designed for deterministic, developer-controlled inputs. An MCP tool is designed for a non-deterministic agent that may call it with unexpected parameters, skip it entirely, or misunderstand its purpose based on a vague description. That means MCP tools need to be scoped around agent-sized jobs (not API endpoints), return contextually filtered responses (not raw data dumps), and have descriptions written like onboarding notes — not reference docs. The ergonomics that make a tool intuitive for an agent turn out to also make it intuitive for humans.

Separate your eval tasks into a training set (what you actively optimize against) and a held-out test set (what you only check at the end). Optimize your tool descriptions and implementations exclusively against the training set, then validate on held-out tasks before shipping. Without this split, you'll overfit — your agent will perform well on the tasks you tuned for and fail on anything slightly different. Teams running this loop iteratively — eval, optimize, re-eval on held-out — consistently extract performance gains beyond what manual expert tuning achieves.

Yes — and it's one of the highest-leverage moves in the workflow. Concatenate your eval transcripts (tool calls, responses, reasoning blocks, errors) and pass them to your AI coding assistant with a prompt to analyze and refactor. LLMs are particularly good at ensuring self-consistency across large tool sets — catching cases where a change to one tool description creates a contradiction in another. The output often beats what human researchers produce through manual iteration, especially when paired with held-out test sets to validate the improvements aren't just overfitting.

There's no universal answer — it depends on your model and task type. LLMs are trained on next-token prediction and perform better with formats that match their training data distribution. Some models handle XML better for structured retrieval; others handle Markdown better for conversational outputs. The only correct approach is to test response formats as an explicit variable in your eval, measure the accuracy delta, and choose based on your own data. Don't assume what worked in one agentic context will transfer.

Write them like you're onboarding a smart new hire who has never seen your codebase. That means: make implicit context explicit (specialized query formats, niche terminology, how resources relate to each other), use unambiguous parameter names (user_id not user, start_date not date), and include concrete examples of correctly formatted inputs directly in the description. The test: could someone — or something — follow this description with zero prior context and still call the tool correctly on the first try? If not, it needs more specificity.

Tool annotations are metadata flags in the MCP spec that disclose a tool's behavior to the client and agent — specifically whether a tool requires open-world access (e.g., reads external data) or makes destructive changes (e.g., deletes records, sends messages). They're not enforced at runtime but give agents and orchestration layers the signal they need to apply appropriate caution. Use them on any tool that writes, modifies, or deletes data — this is especially important in multi-agent setups where one agent's action can cascade into another's context.

Write actionable, specific error messages — not stack traces or opaque error codes. When a tool call fails on input validation, the error message loads directly into the agent's context and steers its next action. A bad error message ("Error: 422 Unprocessable Entity") tells the agent nothing. A good one tells it exactly what went wrong and gives a concrete example of correct input format. Think of your error responses as micro-prompts: they should guide the agent toward the right retry strategy, not just log the failure.

Interleaved thinking is a model capability where the agent outputs explicit reasoning between each tool call — not just at the start of a task. In practice, it means the agent can re-evaluate its strategy mid-task after seeing a tool response, rather than committing to a plan upfront. This dramatically improves performance on multi-step tasks where early tool calls reveal information that should change subsequent ones. For evals, it also gives you a detailed diagnostic view: you can read exactly why the agent chose each tool, what it expected to find, and how it interpreted the result — far more useful than a final answer alone.

Implement pagination with sensible, conservative defaults — not the maximum your API supports. Agents don't benefit from receiving 500 results when 10 would answer the question; they just burn context on irrelevant data. Expose pagination parameters explicitly (e.g., page, limit, offset) and steer agents in your system prompt to start narrow and paginate only if needed. Pair truncated responses with a clear signal — like a has_more: true field — so the agent knows to paginate rather than assume the first page is exhaustive.

The most widely cited benchmark for agentic tool use is SWE-bench Verified, which measures an agent's ability to resolve real GitHub issues using tools like file editing, terminal execution, and code search. For your own tools, build a domain-specific eval that mirrors your actual workflows — internal data, realistic prompts, multi-step tasks. Track accuracy, token consumption, runtime, and tool call count. Improvement on your own eval matters more than third-party benchmarks; the goal is compound improvement on your specific use case, not a leaderboard score.

Often yes — but the reason is safety, not just separation of concerns. Read tools are low-risk and can be called freely; write tools modify state and may be irreversible. Separating them lets you apply different permission levels, surface tool annotations (read-only vs. destructive), and give agents explicit guidance about when to confirm before acting. In practice, the pattern that works well is a read tool that retrieves and validates context first, followed by a write tool that acts on it — with an explicit confirmation step in between for any destructive operation.

In multi-agent systems, tools need to be stateless, idempotent where possible, and clearly scoped — because multiple agents may call the same tool concurrently or in sequence with different assumptions about shared state. Tool descriptions need to be even more explicit about side effects, since a downstream agent receiving a tool response may have no context about what triggered the original call. Namespace your tools by agent role or domain if different agents have different permission levels, and design return formats that are self-explanatory without requiring prior conversation context.