Why Your AI Agent Burns Tokens and Does Nothing Useful

An agent that "tries again" can spend more on the retries than the task was worth. The user asked for one thing. The loop called a tool, reloaded the whole conversation, called the tool again, and kept the transcript. The invoice is the transcript times the number of steps. Nothing in the product UI shows that, so the first person to notice is whoever opens the provider bill.

The hosting and retainer side of AI app cost is in what founders should budget to maintain an AI app. This note is the agent loop itself: tokens spent because the run did not stop. The way you see it per run is in how to debug an agent that works until it doesn't.

What a step actually sends

An agent loop is simple. The model returns a tool call or a final answer. If it returned a tool call, your code runs the tool and sends the result back. The model goes again. Anthropic's description of the agent loop is that cycle, and it will continue until the model stops on its own unless you set a limit. For a narrow task that is fine. For "improve this codebase" or "keep researching until you are sure," it is how a run lasts until someone notices the bill.

Each request includes the prior messages. Step one is the user text and the tool list. Step eight is the user text, eight tool results, the model's previous chatter, and the tool list again. Input tokens climb even when the new work is a one-line tool result. Output tokens climb if the model restates the plan every time. A loop of twenty steps is not twenty times a small call. It is a small call plus a larger one plus a larger one.

Prompt caching changes the price of the repeated prefix. It does not change the fact that the prefix is sent, and it does not apply to the new tail. A cache miss, because you edited the system prompt or reordered tools, drops you back to full input price. If you do not record cache-read tokens separately you will not know whether the cache is working. Record them as a subset of input tokens, not as an extra pile to add. OpenTelemetry's token conventions say the cache and reasoning counters are subsets of the input and output totals. Summing every counter is how an internal dashboard invents a spend the provider will not match.

Caps that exist, and caps that only advise

The Agent SDK exposes two limits. max_turns / maxTurns caps how many turns the loop may take. max_budget_usd / maxBudgetUsd caps estimated spend. The default for both is no limit. When a limit hits, the result subtype is error_max_turns or error_max_budget_usd, and there is no normal final answer. That is what you want in production. A run that hits the cap should look like a failure you handle, not like a successful empty completion.

Count turns the way the docs count them. Turns that use tools count, and you need room for a final text response or the run ends on the cap without an answer. The docs' example: max_turns=2 can stop a loop before the edit step if you forgot that the last answer needs a turn. Size the cap to the tool calls the task needs, then add the finishing turn. Do not size it to "a big number so users stop complaining."

The budget is checked after a model response arrives, because the cost comes from the usage the API returns with that response. The response that crosses the cap still completes and still counts. Subagents that are already running can spend a bit more before they stop. Spend can pass the cap by up to one response, plus those in-flight subagents. Leave headroom. A cap of the exact dollar you are willing to lose will lose a bit more. Subagent spend counts toward the same total. A parent with a budget and children with no budget is a parent with no budget.

A /clear starts the budget over. A queued message when streaming can start a new turn whose max-turn count resets, while the budget keeps accumulating. Read that twice if you are building a chat box. The turn cap is not a conversation cap unless you make the conversation the thing you cap.

Anthropic's separate task budget is a different mechanism, and it is softer. You tell the model a token total for the agentic loop, including thinking, tool calls, tool results, and output. The API injects a countdown. The model is supposed to pace itself and finish cleanly as the budget runs down. The docs are explicit: this is a hint, not a hard cap. The model may exceed it to finish an action it would be worse to interrupt. The enforced ceiling on output tokens for a single request is still max_tokens, which stops generation with stop_reason: max_tokens.

Use the soft budget if you want the model to wrap up. Use max_tokens and the SDK's max turns and max dollars if you want the process to stop. A prompt that says "be concise" is neither.

A new user message that contains no tool results starts a new turn and a fresh task budget. History from earlier turns can still sit in the context and still be billed as input. Server-side compaction during a turn does not reset the budget: tokens already consumed in the turn still count. Compaction is not a refund.

The patterns that waste the money

The loop. A tool errors, the model calls it again with the same arguments, the error comes back, the context grows. Twenty identical failures cost more than one, and they do not move the task. Stop on repeated identical calls. That rule belongs in your runner. The model will not reliably invent it.

The reload. Every step sends the full tool catalog and the full history. If you have thirty tools and the task needs two, you are paying for twenty-eight descriptions on every step. Cut the tools per task. The security benefit and the cost benefit are the same decision.

The upgrade. A router sends a "quick" classification to the largest model, or a fallback promotes every call because the first model returned a retryable error that is in fact permanent. The trace shows one model name on the request and another on the response, repeated. Pin the model per step. Alert when the fallback share moves.

The unbounded retry. A 429 or a 529 with exponential backoff is reasonable. A retry of a 400, or a retry that also restarts the whole agent, is not. Cap retries. Log which error was retried. A retry budget that is independent of the token budget will defeat the token budget.

The tool result that pastes the world. A search tool returns every row. The next input-token count jumps by the size of your table. Paginate tool results. Give the model a summary and an id, and fetch the record only if it asks for one. This is the single largest accidental line item in agent products that "added search."

The missing user-level cap. The run cap stops one loop. It does not stop a user who starts a hundred loops. Put a budget on the account or the API key for the day. When it hits, stop starting new runs. Show the user that it stopped. Do not keep the runs going and argue about the bill later.

How to read the invoice

Take a week of inference spans, not parent spans. Group by model and by agent name. Sum input and output tokens. Apply the provider's current prices for that model, and treat cache-read tokens at the cache-read price if you recorded them separately. Compare the total to the invoice. If you are far off, you are either missing a caller, using a stale price, or double-counting parents.

Then sort runs by cost. The top ten runs will show the pattern: a loop, a huge tool result, or a model you did not think was in production. Fix that pattern. A blanket "use a cheaper model" change, applied before you have looked at the top ten, often makes the agent take more steps and spend the same money with worse answers.

New Relic's 2026 coding survey is about generated code in production, not about this invoice, but one figure belongs next to the bill: 96% of those leaders said observability was very or extremely important for AI-generated code, and 78% were prompting tools to emit telemetry. Token spans are that telemetry for an agent. Without them you have a monthly total and a theory.

Anthropic's own migration write-up is a useful scale marker, not a target. The Bun language port they describe consumed 5.9 billion uncached input tokens and 690 million output tokens, on the order of $165,000 at API pricing, for a million-line migration with a test suite as the judge. A production feature agent that spends like a migration, without a judge, is on fire. Your cap should be set for a user task, not for a migration, unless you are running a migration on purpose and have said so.

What to set before the next deploy

1. A max turn count sized to the task, with room for a final answer, and a tested behavior when the cap hits. The user should see a failure or a partial result you chose, not a spinner.
2. A max dollar or token budget per run, with headroom for the last response. Handle error_max_budget_usd in code.
3. max_tokens on each model call so one response cannot run away. Remember it truncates. Do not treat a truncated tool call as a tool call you should execute.
4. A per-account daily ceiling that stops new runs.
5. A tool list per task, not the full catalog on every call.
6. Tool results with a size limit.
7. A ban on retrying the same tool arguments, and a small retry cap for the errors you do retry.
8. Spans for each model call with input, output, and cache-read tokens, so next week's top ten runs are a query and not an archaeology project.

Check the prices when you change models. A blog's rate table is not your bill. The provider's pricing page on the day you deploy is.

What to stop doing

Do not leave max turns unset because the demo was short. Do not rely on a task budget hint as your only stop. Do not paste entire documents into the context on every turn when the model needed a paragraph. Do not add a "research thoroughly" instruction to a user-facing agent that has a credit card on the API key and no cap. That instruction is a blank check.

Do not hide the cap from the product. A stopped run is a product state. Log it, count it, and decide whether the cap is too low because good tasks hit it, or too high because bad tasks never do. Both happen. The trace tells you which.

A budget a product can explain

Put the cap where a user can understand it. "This task stopped because it reached the limit" is a better product than a spinner that eventually errors inside the provider. Show the limit in the same units you enforce, turns or dollars, and log the subtype that fired. If users hit the cap on tasks you consider legitimate, raise the cap for that task class or split the task. If they never hit it and the invoice is still ugly, the cap is above the waste and is not doing anything. Lower it until a wasteful run stops and a normal run does not.

Separate background work from interactive work. A nightly job can have a larger budget and a queue. A chat box cannot. Sharing one uncapped key between them means the nightly job and a looping chat compete, and you will not know which one to kill. Two keys, two budgets, two dashboards. The implementation is a configuration change. The alternative is a postmortem that starts with "we did not know which product spent it."

Review the cap when you add a tool. A new tool that returns large payloads will push normal runs over a cap that used to fit. That is a signal to shrink the payload, not an automatic reason to double the budget. Write the review into the same checklist as the security review of the tool. A tool that can spend and a tool that can leak are often the same integration, added on the same afternoon, by the same pull request.

How Techtribe cuts agent spend that is not doing work

We trace a week of runs, name the pattern in the expensive ones, and put a turn cap and a budget on the runner so the pattern cannot repeat overnight. The change is usually a tool result, a retry, or a missing stop, not a new model. We reconcile the estimate against the invoice so the number you watch is the number you pay.

If the agent is part of a product you are about to put in front of customers, cost and latency monitoring is in a Production MVP. If it is already live and the bill moved, a Dedicated Team can own the caps while the product keeps shipping. Building the agent itself is AI agent development. Bring the last invoice and one conversation that felt expensive. If you do not have per-run numbers, the first deliverable is those numbers.

The fixed scope for taking a generated app to production is a page per tool: Lovable, Bolt, v0, and Replit.