Four Agent Cost Traps and the Circuit Breaker
Tool-return explosion, thinking tax, infinite loops, history snowball: one shippable strategy per trap, plus three red lines
THE QUESTION THIS PAGE ANSWERS
ANSWER FIRSTWhat is the key idea behind “Four Agent Cost Traps and the Circuit Breaker”?
Tool-return explosion, thinking tax, infinite loops, history snowball: one shippable strategy per trap, plus three red lines
Read cost as a shape, not a single number. Break a request into input, output, retries, tools, and waiting time. The shape of usage usually tells you which design choice is expensive and where a smaller change can help.
Measure one real request before you optimize an imagined average.
A cheaper call that quietly creates more retries, latency, or review work.
User says “pull every user's orders from the database,” the Agent's SQL tool returns 10,000 rows ≈ 500,000 Tokens. Those 500k Tokens get stuffed into the next turn's Input: you trip the expensive tier, maybe blow the context window, and the model “gets lost” under overload so output quality drops. The fix is a truncation guard on every tool:
Reasoning-enabled models—such as DeepSeek-R1, OpenAI o-series models, and reasoning variants from other providers—may spend extra tokens on work users never see. Providers count those tokens differently, so check the current rate card instead of assuming a fixed multiplier. In this worked example, the visible answer stays at 450 Tokens while hidden reasoning adds 2,000 Tokens, making the billable output more than five times larger.
| Task type | Thinking mode | Why |
|---|---|---|
| Simple retrieval | ❌ Off | No deep reasoning needed |
| Data cleaning | ❌ Off | Rules are clear—no “thinking” required |
| Complex reasoning | ✅ On | Worth paying for accuracy |
| Code generation | ⚠️ It depends | Off for simple functions; on for complex architecture |
Advanced fix: use a small routing model as front-door triage—spend a fraction of a cent first to decide whether this request needs deep reasoning, then route to the right mode. That's the concrete shape of lesson 3's “T2 backs up T0.”
Agent fixing a bug: fix A → error B → fix B → error A (back to square one) → … still spinning at turn 15. If Input grows 1,000 Tokens per turn, 20 turns cost 13×; worse, the user waited 5 minutes with nothing done. The fix is a forced circuit breaker—graceful exit when any of three conditions hits:
On graceful exit, return rounds_executed, tokens_consumed, and partial_result—a half-finished artifact beats a black hole.
The standard (wrong) approach stuffs the full history into Input every turn. The better approach is fixed prefix + compressed history + last N turns: never compress the System Prompt (preserve the cache prefix), keep the last 3 turns verbatim, and squash older history into one summary sentence with a small model.
| Approach | Turn-10 Input | Notes |
|---|---|---|
| Unbounded growth | ~50,000 Tokens | Includes full history |
| Sliding window (last 5 turns) | ~12,000 Tokens | Loses early context |
| Fixed + summary + last 3 turns | ~6,000 Tokens | Keeps what matters, controls length |
| Control point | Strategy | Expected gain |
|---|---|---|
| Tool returns | Truncate + summarize, cap 2k Tokens | Stop single-turn explosions |
| History management | Fixed prefix + compress old history | Cut Input 50%+ |
| Loop control | Circuit-breaker mechanism (turns / Tokens / loop detect) | Stop bottomless pits |
| Thinking mode | Enable by task tier | Cut Output cost ~4× |
| Model selection | Small models for simple subtasks | Lower unit price |
| Cache use | Fixed System Prompt, hit KV Cache | Cut Input cost ~90% |
| Red line | Suggested threshold | Consequence | Response |
|---|---|---|---|
| Per-turn Input | < 32k Tokens | Jump into expensive tier | History compression + tool truncation |
| Total turns | < 10 turns | Cost grows exponentially | Circuit-breaker mechanism |
| I/O Ratio | Watch > 50:1 | Agent is “spinning” | Optimize workflow or degrade the task |
Cap tool returns at 2k: truncate + summarize + tell the model to narrow scope—stop single-turn Input explosions.
Tier thinking mode by task: the invisible inner monologue still bills as Output, at 4× the unit price.
Circuit breakers are the Agent's fuse: hit any of turns, Token budget, or loop detection → graceful exit.
Manage history with “fixed + summary + last 3 turns,” half the cost of a blunt sliding window without amnesia.
Source: Adapted from the author's internal team share “AI Token Cost Engineering Strategies,” section “Billing Mechanics for Agentic Apps.” Product angles on Agent freezes and fool-proofing are covered in Hands-On Practice; for context compression also see Harness Core · Context Overflow.
The complete interaction cost of “Trap 1 · Tool-return information explosion”
“User says “pull every user's orders from the database,” the Agent's SQL tool returns 10,000 rows ≈ 500,000 Tokens .” is a reminder that AI cost is not one price multiplied by one call. Input, output, retries, tools, waiting time, and human cleanup together decide what a task really costs.
Find what the bill repeats
The key variables behind “Reasoning-enabled models—such as DeepSeek-R1, OpenAI o-series models, and reasoning variants from other providers—may spend extra tokens on work users never see.” are usually repeated context, oversized output, retries after failure, and calls that do not produce useful progress. Removing wasted Tokens can reduce cost, latency, and concurrency pressure at the same time.
A cheaper call can make the whole workflow more expensive
Start with “Manage history with “fixed + summary + last 3 turns,” half the cost of a blunt sliding window without amnesia” and keep a small table for input, output, retries, tools, and human review. Compare quality before and after optimizing instead of looking at one price in isolation.
From “Trap 1 · Tool-return information explosion” to “Trap 2 · The invisible bill of thinking Tokens”
“Trap 1 · Tool-return information explosion” grounds the problem in “User says “pull every user's orders from the database,” the Agent's SQL tool returns 10,000 rows ≈ 500,000 Tokens . Those 500k Tokens get stuffed into the next turn's Input: you trip the expensive tier, maybe b…”. “Trap 2 · The invisible bill of thinking Tokens” then moves it toward “Reasoning-enabled models—such as DeepSeek-R1, OpenAI o-series models, and reasoning variants from other providers—may spend extra tokens on work users never see. Providers count those tokens differently, so che…”. Together, they show that the lesson is not just a conclusion to remember, but a claim with conditions.
Carry the judgment into the next situation
When analyzing cost, map the complete interaction first, then find repeated input, wasted output, and retries. A cheap individual call does not make the whole task cheap.
- “Trap 1 · Tool-return information explosion”: User says “pull every user's orders from the database,” the Agent's SQL tool returns 10,000 rows ≈ 500,000 Tokens . Those 500k Tokens get stuffed into the next turn's Input: you trip the expensive tier, maybe b…
- “Trap 2 · The invisible bill of thinking Tokens”: Reasoning-enabled models—such as DeepSeek-R1, OpenAI o-series models, and reasoning variants from other providers—may spend extra tokens on work users never see. Providers count those tokens differently, so che…
- “The closing point”: Tier thinking mode by task: the invisible inner monologue still bills as Output, at 4× the unit price
The final “The closing point” brings the discussion to “Tier thinking mode by task: the invisible inner monologue still bills as Output, at 4× the unit price”. The useful thing to carry forward is knowing which judgments must be revisited when input, scale, or risk changes.
I turned one judgment from this article into a small experiment I could run today. Knowing what to observe next is more useful than simply remembering the conclusion.
After reading this, I first looked for the conditions behind the idea instead of copying the method into a project. That order made the later trade-offs much clearer.
When this judgment reaches real work, which constraint should be added first? I am curious which step matters most between reading and the first practical attempt.
No discussion on this article yet.