The Hidden Cost of AI Coding Agents: Context, Retries, and Verification

By Neel Shah · October 22, 2026

When teams compare AI coding agents, they compare the sticker price: dollars per million tokens. That number is almost irrelevant. The real cost of an agent isn't what it charges to generate code — it's the total cost of getting a correct, tested, reviewable change merged. Tokens are the cheapest input in that equation. The expensive parts are the context it re-reads, the retries it burns, and the human time it takes to verify the output is actually right.

Two pieces of research published recently put hard numbers on this, and they're worth sitting with because they're counterintuitive. So let me break the hidden cost into its three parts, show what the data says, and end with a metric you can actually use to compare agents: cost per verified change.

Context: the agent reads more than it needs

An agent's context is a snowball. Repository files, previous messages, tool outputs, test logs — all of it accumulates and gets re-fed to the model on every single step. This is the part people underestimate, because output tokens are what you see in the chat, but they're not what you pay for.

Microsoft Research's How Do AI Agents Spend Your Money? measured this across 500 SWE-bench problems and found that input tokens — not output — dominate cost, even with caching, precisely because the agent repeatedly re-feeds accumulated context. The variance is staggering: the most expensive problem cost about 7 million more tokens than the cheapest, and the same problem run multiple times could differ by up to 30× in total tokens. Expert-rated task difficulty barely predicted it — 11% of tasks rated "over an hour" used fewer tokens than the average "under 15 minutes" task.

The intuitive fix is to give the agent more context up front — a repo-level AGENTS.md file describing your conventions. Here's where the second study complicates the story. ETH Zurich's SRI Lab Evaluating AGENTS.md tested this directly, and the results argue against the reflex:

• LLM-generated context files raised execution cost by over 20% (20% on SWE-bench Lite, 23% on AGENTbench) while delivering no average lift — actually a slight drop in success in 5 of 8 settings.
• Developer-written files gave only a marginal ~4% gain, still at added cost.

But — and this is the nuance that makes it useful rather than a blanket "don't bother" — when the researchers removed all other documentation, LLM-generated context files did help (+2.7%). The takeaway isn't "context is useless." It's that redundant context is pure cost: in a repo that already has good docs and types, a context file mostly makes the agent run more tests and explore more without getting more right. In a doc-poor repo, it can earn its keep.

The practical principle: focused context beats broad context. A clear plan and the three files that matter outperform dumping the repository into the window and paying to re-read it on every step. (Note the study is Python-only and sampled each agent once — treat it as a strong signal to test on your stack, not a universal law.)

Retries: the same task can quietly get expensive

Failed tool calls, repeated test runs, re-exploring files the agent already saw, and reattempting a broken approach all add latency and cost — and they're invisible in a per-token price. The Microsoft study saw exactly this: higher-cost runs showed repeated file views and edits climbing sharply, and failed runs consumed meaningfully more tokens than successful ones.

The most important finding for anyone paying the bill: accuracy often peaks at an intermediate cost and then saturates. Spending beyond that point mostly doesn't buy you a better result — the agent is burning tokens going in circles. NVIDIA's engineering analysis of agentic systems illustrates the same dynamic from the architecture side: context grows across many requests and sub-agent invocations, and without bounds it compounds.

The distinction that matters is between a useful retry and a loop. A useful retry incorporates new information — a test failure that tells the agent what to fix. A loop re-attempts the same approach against the same error, making no progress while the context (and the bill) grows. The mitigation is unglamorous but real: explicit stopping conditions, a retry limit, and a plan the agent checks against, so a stuck agent fails fast instead of failing expensively.

Verification: generated code is not a completed task

This is the cost that never appears in any dashboard. An agent emitting a diff is not the same as a task being done. The change still has to pass tests, satisfy the actual requirements, avoid regressions, and produce a diff a human can review. Every one of those is real work, and most of it lands on a person.

If you've read my earlier posts, this is the same theme from a different angle: generation got cheap, verification is the bottleneck. The cost of an agent has to include the compute to run its tests and the human minutes to confirm the diff is correct and safe. A change that looks plausible, passes shallow checks, and is subtly wrong is the most expensive output an agent can produce — because the cost of catching it (or not) dwarfs the tokens that made it.

So any honest accounting of an agent's cost has three line items, not one:

1. Agent cost — the tokens, including all that re-fed context and every retry.
2. Verification cost — the compute to run tests, lint, and checks.
3. Human review cost — the minutes a person spends confirming it's right.

Run the experiment yourself

Opinions about agent efficiency are cheap. Here's a small, reproducible test that produces real numbers for your setup. The task: ask an agent to add input validation and tests to a simple TypeScript API endpoint — small enough to run many times, real enough to be representative.

Run it under two configurations, holding the model, task, and environment constant:

Run each config several times (results vary run-to-run — that's the point), keep the evaluation criteria identical, and report failures as well as successes. Resist assuming Config B wins; the honest, more interesting outcome is finding where the extra discipline pays off and where it just adds overhead. More context and more verification aren't free — sometimes they buy reliability, sometimes they only buy cost.

The metric that matters

Collapse it all into one number:

                    agent cost + verification cost + human review cost
cost per verified change = ──────────────────────────────────────────────────
                         number of changes that pass the acceptance criteria

This is the metric to compare agents on — not $/token, not raw task-completion rate. A "cheaper" agent that produces plausible-but-wrong diffs has a higher cost per verified change than a pricier one that gets it right the first time, because the denominator is what you actually care about and the numerator includes the human time to sort it out.

Takeaways

• The cost of an AI coding agent is not $/token — it's the cost of a correct, tested, reviewable change.
• Context: input tokens from re-fed context dominate cost; the same task can vary up to 30×. Focused context beats broad — and LLM-generated AGENTS.md files added 20%+ cost for no average gain, except in doc-poor repos.
• Retries: accuracy peaks at intermediate cost and saturates — distinguish useful retries from progress-free loops, and cap them.
• Verification: a diff isn't a done task. Count test compute and human review in the real cost.
• Compare agents on cost per verified change, and run the two-config experiment on your own stack rather than trusting any benchmark.

The agents are genuinely good and getting better. The discipline that separates a force multiplier from a money pit isn't picking the cheapest token price — it's measuring the thing that actually costs you: getting to a change you'd confidently merge.

Sources & further reading

• Microsoft Research — How Do AI Agents Spend Your Money? Analyzing and Predicting Token Consumption in Agentic Coding Tasks
• SRI Lab, ETH Zurich — Evaluating AGENTS.md: Are Repository-Level Context Files Helpful for Coding Agents?
• NVIDIA — Building for the Rising Complexity of Agentic Systems
• GitHub Docs — Optimizing your use of AI (splitting research, planning, and implementation into focused phases)