Input-Dominated: a 62:1 I/O Ratio
Every Agent turn re-reads the full history. Step through an Excel task turn by turn and watch Input roll up to 31,460 Tokens
THE QUESTION THIS PAGE ANSWERS
ANSWER FIRSTWhat is the key idea behind “Input-Dominated: a 62:1 I/O Ratio”?
Every Agent turn re-reads the full history. Step through an Excel task turn by turn and watch Input roll up to 31,460 Tokens
Read cost as a shape, not a single number. Break a request into input, output, retries, tools, and waiting time. The shape of usage usually tells you which design choice is expensive and where a smaller change can help.
Measure one real request before you optimize an imagined average.
A cheaper call that quietly creates more retries, latency, or review work.
A plain chatbot asks “tell me a joke” (10 Tokens) and answers with 150 Tokens—I/O Ratio ≈ 1:15, output-dominated. An Agent doing a code-fix task: System Prompt, tool returns, and past outputs all get re-read every turn. After three turns, total Input 16,790 and total Output 270—I/O Ratio 62:1, input-dominated.
Task: “Analyze this Excel sheet, find the top-selling product, then generate a chart.” Hit “Next turn” and watch cumulative Input snowball each round.
Suppose a task takes N turns, System Prompt length S, and each turn adds about Δ (output + tool return). Turn N's Input ≈ S + Q + Δ×(N-1), and total Input is the sum across all turns—hiding the arithmetic series 1+2+3+…+(N-1). At +500 Tokens per turn, five loops already push total Input past 15,000; double the turns and cost nearly quadruples.
| Agent framework (SWE-bench measured) | Avg. I/O Ratio | Notes |
|---|---|---|
| Simple RAG Agent | 10:1 ~ 20:1 | Retrieve + answer |
| OpenHands | 20:1 ~ 50:1 | Code-fix tasks |
| AutoGPT-class | 30:1 ~ 100:1 | Open-ended tasks, many loops |
That's why KV Cache (lesson 11) is decisive for Agents: if every turn re-reads the same prefix, a cache hit is a 5× price gap. It also explains why you should watch I/O Ratio—above 50:1 usually means the Agent is spinning / idle looping, so optimize the workflow or degrade the task.
Agents are input-dominated: every turn's Input carries the full history; total volume grows roughly with the square of turn count.
One run looks cheap; scale is where it bites: a $0.02 illustrative task × failed retries × a million calls is the bill-shock moment.
Treat I/O Ratio as a health metric: >50:1 means the Agent is spinning—fix the workflow before you chase savings.
Source: Adapted from the author's internal team share “AI Token Cost Engineering Strategies,” section “Billing Mechanics for Agentic Apps.” The dollar amounts are illustrative; real bills vary by provider, cache status, region, and model. Industry data references SWE-bench studies of Agent Token consumption (see Finale reading list). For Agent loop fundamentals see Hands-On Practice.
The complete interaction cost of “Chatbot vs Agent: totally different billing structures”
“A plain chatbot asks “tell me a joke” (10 Tokens) and answers with 150 Tokens— I/O Ratio ≈ 1:15, output-dominated .” is a reminder that AI cost is not one price multiplied by one call. Input, output, retries, tools, waiting time, and human cleanup together decide what a task really costs.
Find what the bill repeats
The key variables behind “Task: “Analyze this Excel sheet, find the top-selling product, then generate a chart.” Hit “Next turn” and watch cumulative Input snowball each round” are usually repeated context, oversized output, retries after failure, and calls that do not produce useful progress. Removing wasted Tokens can reduce cost, latency, and concurrency pressure at the same time.
A cheaper call can make the whole workflow more expensive
Start with “Treat I/O Ratio as a health metric: >50:1 means the Agent is spinning—fix the workflow before you chase savings” and keep a small table for input, output, retries, tools, and human review. Compare quality before and after optimizing instead of looking at one price in isolation.
From “Chatbot vs Agent: totally different billing structures” to “Interactive Demo · The real bill for a “simple” task”
“Chatbot vs Agent: totally different billing structures” grounds the problem in “A plain chatbot asks “tell me a joke” (10 Tokens) and answers with 150 Tokens— I/O Ratio ≈ 1:15, output-dominated . An Agent doing a code-fix task: System Prompt, tool returns, and past outputs all get re-read…”. “Interactive Demo · The real bill for a “simple” task” then moves it toward “Task: “Analyze this Excel sheet, find the top-selling product, then generate a chart.” Hit “Next turn” and watch cumulative Input snowball each round”. Together, they show that the lesson is not just a conclusion to remember, but a claim with conditions.
Carry the judgment into the next situation
When analyzing cost, map the complete interaction first, then find repeated input, wasted output, and retries. A cheap individual call does not make the whole task cheap.
- “Chatbot vs Agent: totally different billing structures”: A plain chatbot asks “tell me a joke” (10 Tokens) and answers with 150 Tokens— I/O Ratio ≈ 1:15, output-dominated . An Agent doing a code-fix task: System Prompt, tool returns, and past outputs all get re-read…
- “Interactive Demo · The real bill for a “simple” task”: Task: “Analyze this Excel sheet, find the top-selling product, then generate a chart.” Hit “Next turn” and watch cumulative Input snowball each round
- “The closing point”: Treat I/O Ratio as a health metric: >50:1 means the Agent is spinning—fix the workflow before you chase savings
The final “The closing point” brings the discussion to “Treat I/O Ratio as a health metric: >50:1 means the Agent is spinning—fix the workflow before you chase savings”. The useful thing to carry forward is knowing which judgments must be revisited when input, scale, or risk changes.
I turned one judgment from this article into a small experiment I could run today. Knowing what to observe next is more useful than simply remembering the conclusion.
After reading this, I first looked for the conditions behind the idea instead of copying the method into a project. That order made the later trade-offs much clearer.
When this judgment reaches real work, which constraint should be added first? I am curious which step matters most between reading and the first practical attempt.
No discussion on this article yet.