Free · structured · practical
Search 中文 Start at lesson 1
Courses/The Art of Tool Design/Using Agents to Optimize Agent Tools
Part 4 · Engineering Patterns for Reliable Agents

Using Agents to Optimize Agent Tools

Claude Code in practice: using AI to write tool descriptions, run evals, and auto-iterate

THE QUESTION THIS PAGE ANSWERS

ANSWER FIRST

What is the key idea behind “Using Agents to Optimize Agent Tools”?

Claude Code in practice: using AI to write tool descriptions, run evals, and auto-iterate

DECISION RULE

Make the claim earn its place. Use this page as a decision aid, not a definition to memorize. Connect the idea to one real task, one observable result, and one failure that would change your mind.

TRY NEXT

Write one question you could answer with evidence after trying this idea.

WATCH FOR

A conclusion that sounds complete but leaves the key assumption untested.

Core Idea
Traditional approach: humans write tools → humans test → humans improve. Long cycle, slow feedback, relies on developer intuition.
New approach: let Claude Code write tools → use evaluations to measure automatically → let Claude Code read eval results and optimize automatically. The Agent becomes the product manager of its own tools.
Three-Step Workflow: Prototype → Evaluate → Optimize
Prototype
Evaluate
Optimize
Evaluation not satisfactory? Repeat the loop until it meets the bar
01

Prototype

Use Claude Code to quickly generate a tool prototype. Describe the tool functionality you want and let it generate the MCP tool code skeleton.
Input: "Write me a Jira tool that can create issues, list issues, and update issue status"

Output: Claude Code generates complete MCP tool code, including tool definitions, parameter validation, and API call logic
02

Evaluate

Build an evaluation system to systematically measure tool performance. Use data to prove quality — "it seems to work" is not good enough.
Evaluation dimensions:
- Did the Agent select the right tool?
- Were parameters filled in correctly?
- Was the return value correctly interpreted?
- What is the end-to-end task completion rate?
03

Optimize

Let Claude Code read the evaluation results, automatically analyze failure reasons, and improve tool descriptions and implementation.
Claude Code analysis: "Agent confused search and list in 23% of cases because the descriptions were too similar"

Auto-fix: Rewrite tool descriptions, add differentiating explanations and usage examples
Five Tool Design Principles
1

Choose the Right Tools: Less Is More

Don't implement too many tools. If human developers can't tell whether to use search, find, or lookup, the Agent can't either.
Principle: If two tools have more than 50% overlapping use cases, merge them. Better to have one tool with more parameters than two easily confused tools.
2

Namespacing: Group Management

Group related tools with a prefix so the Agent can immediately see how tools relate to each other.
Good naming: jira_create_issue / jira_list_issues / jira_update_status
Bad naming: create_issue / list_tasks / update
3

Return Meaningful Context

Tool returns should not just say "success" — return the information the Agent needs for its next step.
Bad: {"status": "success"}
Good: {"status": "success", "issue_id": "PROJ-123", "url": "https://...", "assignee": "sample-user"}
4

Token Efficiency: Trim Your Returns

Trim large result sets. Returning 1,000 records consumes massive Tokens, while the Agent only needs the first 10.
Strategies: Summarize (return only statistics), Truncate (default to top N), Paginate (support page parameters), Filter (support conditions)
5

Engineer Your Tool Descriptions as Prompts

Tool descriptions are not just documentation — they are part of the Prompt. Tell the Agent when to use this tool, and more importantly, when NOT to use it.
Good description template: "[Tool name] is used for [specific purpose]. Use this tool when you need [scenario A] or [scenario B]. Do NOT use it in [scenario C] — use [another tool] instead. Example: [specific input/output]"
Namespacing in Practice: Give Agents a Tool Map

Tool Namespace Groups

jira_ -- Project Management
jira_create_issue jira_list_issues jira_update_status jira_add_comment
git_ -- Version Control
git_diff git_commit git_log git_create_branch
db_ -- Database
db_query db_insert db_update db_schema
The value of namespacing: when an Agent sees a group of tools with the jira_ prefix, it immediately knows they are related and operate on the same system. This dramatically reduces the probability of selecting the wrong tool.
Token Efficiency: The Science of Return Values

Full Return

[ {"id": 1, "title": "Fix login bug", "desc": "Users cannot login...", "created": "2025-01-15T...", "updated": "2025-01-16T...", "assignee": {"name": "sample-user", ...}, "labels": [...], "comments": [...]}, {"id": 2, ...}, ... // 847 records total ]
~52,000 Tokens -- Agent cannot process this at all

Trimmed Return

{ "total": 847, "showing": 10, "page": 1, "results": [ {"id": 1, "title": "Fix login", "status": "open", "assignee": "sample-user"}, {"id": 2, ...}, ... // top 10, core fields only ], "hint": "Use page=2 for more" }
~800 Tokens -- High information density, Agent handles it easily
Real Example: The Gap in Tool Descriptions

search_issues Tool Description Comparison

BEFORE -- Lazy Description
{ "name": "search_issues", "description": "Search for issues in the project tracker." }
Agent doesn't know the search syntax, return format, or how this differs from list_issues
AFTER -- Engineered Description
{ "name": "search_issues", "description": "Full-text search across issue titles and descriptions. Use when the user mentions specific keywords. Returns max 20 results sorted by relevance. For browsing by status/label, use list_issues instead. Example: search_issues({ query: 'login timeout', status: 'open' })" }
Clear semantics, usage boundaries, examples, and differentiation from similar tools
Key insight from the optimization loop: after Claude Code runs the evaluations, it can precisely say "43% of errors are because the Agent confused search and list," then automatically modify the tool description to fix the problem. This is far faster than humans debugging by intuition.
Tool quality determines the ceiling of Agent quality. Use the Prototype → Evaluate → Optimize loop to systematically improve tool quality. Remember the five principles: right tools, namespacing, meaningful returns, Token efficiency, engineered descriptions. Let the Agent be the product manager of its own tools.

How “Core Idea” changes an answer

“Claude Code in practice: using AI to write tool descriptions, run evals, and auto-iterate” shows that a model does not process the “word count” we see. It processes Token pieces. Tokenization affects input length, how much context fits, and how much computation a request consumes.

Length, information, and context are different

As “Claude Code in practice: using AI to write tool descriptions, run evals, and auto-iterate” grows, separate three questions: how many Tokens the text becomes, which pieces can change the current decision, and whether older material has fallen outside the context window. Removing repetition is often more useful than simply making the window larger.

Keep what can change the decision

Use “Claude Code in practice: using AI to write tool descriptions, run evals, and auto-iterate” as an A/B test: keep the same question while removing repeated background, compressing format, and trimming irrelevant history. Compare answer quality, latency, and Token count.

From “Core Idea” to “Three-Step Workflow: Prototype → Evaluate → Optimize”

“Core Idea” grounds the problem in “Traditional approach: humans write tools → humans test → humans improve. Long cycle, slow feedback, relies on developer intuition. New approach: let Claude Code write tools → use evaluations to m…”. “Three-Step Workflow: Prototype → Evaluate → Optimize” then moves it toward “Prototype Evaluate Optimize Evaluation not satisfactory? Repeat the loop until it meets the bar 01”. Together, they show that the lesson is not just a conclusion to remember, but a claim with conditions.

Carry the judgment into the next situation

For long text, keep what can change the conclusion before compressing format and history. A larger context is worth its cost only when the added information is useful.

  • “Core Idea”: Traditional approach: humans write tools → humans test → humans improve. Long cycle, slow feedback, relies on developer intuition. New approach: let Claude Code write tools → use evaluations to m…
  • “Three-Step Workflow: Prototype → Evaluate → Optimize”: Prototype Evaluate Optimize Evaluation not satisfactory? Repeat the loop until it meets the bar 01
  • “Real Example: The Gap in Tool Descriptions”: search_issues Tool Description Comparison BEFORE -- Lazy Description { "name": "search_issues", "description": "Search for issues in the project tracker." } Agent doesn't know the search syntax, return format…

The final “Real Example: The Gap in Tool Descriptions” brings the discussion to “search_issues Tool Description Comparison BEFORE -- Lazy Description { "name": "search_issues", "description": "Search for issues in the project tracker." } Agent doesn't know the search syntax, return format…”. The useful thing to carry forward is knowing which judgments must be revisited when input, scale, or risk changes.

Mark as learned Your reading progress updates automatically
← PreviousNext →

Keep reading

The next useful article in the thread.

The Art of Tool Design · Hands-onMilvus as an Agent Knowledge ToolWrap vector retrieval as search_knowledge: ToolMessage, memory separation, and testing both calls and non-callsThe Art of Tool Design · Deep DiveThink Tool: Making AI Think Before ActingGive the Agent a pause-to-think space in complex tool chains — τ-bench performance up 54%The Art of Tool Design · Design PatternACI: Agent-Computer InterfaceTools are the contract between Agents and the world. Design Agent interfaces like you design human interfacesAgent Evaluation · Hands-onThree Graders: Code, Model, HumanStatic assertions vs LLM-as-Judge vs human calibration — which fits which scenario

ARTICLE DISCUSSION

Leave one useful thought here.

Keep the idea that clicked, the question that stayed open, or a small note for the next learner.

↗
Discussing Using Agents to Optimize Agent Tools Engineering Patterns for Reliable Agents
3discussionsArticle discussion · synced with the Circle
View in the learning circle
AM
Asha MorganContent editor
INSIGHTField note

I turned one judgment from this article into a small experiment I could run today. Knowing what to observe next is more useful than simply remembering the conclusion.

ARTICLE DISCUSSION7 helpful
LH
Lin HarperIndie developer
INSIGHTInsight

After reading this, I first looked for the conditions behind the idea instead of copying the method into a project. That order made the later trade-offs much clearer.

ARTICLE DISCUSSION5 helpful
KM
Kiki MooreProduct operations
QUESTIONQuestion

When this judgment reaches real work, which constraint should be added first? I am curious which step matters most between reading and the first practical attempt.

ARTICLE DISCUSSION4 helpful