How to Control AI Agent Model and API Costs

Where does agent spend actually go?

Most teams assume the output is the bill. It is not. An agent resends the entire conversation on every turn, so a twenty step run pays for step one twenty times over. Add tool results, file reads, and search output, and input volume usually runs several times output volume. That is why caching and input hygiene beat model shopping.

• Resent history: every turn re-bills the whole transcript so far
• Tool results: a fetched page or a large file lands in context and stays
• Retries: a failed tool call bills the turn that broke and the turn that repairs it
• Abandoned runs: a loop that hits a wall still charged for every step it took

What do the frontier models cost right now?

Read the spread, not the row. Output on the strongest model runs $50.00 per million while input on the cheapest runs $0.20, so where a token lands matters far more than which vendor you signed with. Both vendors discount asynchronous batch work by half, and both bill a cache write above the plain input rate, so caching only pays back when the prefix is genuinely reused.

Which lever should you pull first?

Caching comes first because it is free. Prompt caching is a prefix match: the system prompt, the tool list, and the skill files are identical on every turn, so they can be served from cache at a fraction of the input rate. One clock reading or one unsorted blob of JSON near the top of that prefix invalidates everything after it, silently. No error is raised. The bill simply does not move.

resp = client.messages.create(
    model="claude-sonnet-5",
    max_tokens=2000,
    cache_control={"type": "ephemeral"},
    system=STABLE_PROMPT,   # no clocks, no request ids, no random order
    messages=history,
)

print(resp.usage.cache_read_input_tokens)  # 0 on repeats means it is broken
print(resp.usage.input_tokens)             # uncached, billed at full rate

How do you route a task to a cheaper model?

Routing is not a global setting. It is a per job decision, made once and written into the workflow spec so an operator can see why a task escalates. Classification, extraction, formatting, tagging, and routine drafting run on the cheap tier all day. Ambiguous synthesis, security review, and anything a client puts their name on runs on the strong tier, then drops back down for the mechanical follow-up.

Two cautions before you build a cascade. Caches are scoped to a model, so every hop across models forfeits the cache you were paying to build. And a cheap model that needs three passes and a human rewrite is not cheap. Judge the finished task, not the request.

Bound the loop before you issue a key

Unbounded is the failure mode that produces the number nobody wants to explain. An agent that can retry forever will. Put ceilings in the spec and make the agent stop with partial results when it hits one. A run that halts at eighty percent and says so is recoverable. A run that quietly grinds for four hours is not.

• A hard cap on turns per run, enforced, not a warning in the prompt
• A cap on tool calls, pages fetched, and files read in one run
• A wall clock limit on every scheduled or background job
• A monthly dollar ceiling on the key, set at the provider console
• Approval before the agent can send, publish, deploy, delete, or spend

budget:
  max_turns: 12
  max_tool_calls: 40
  max_wall_clock_minutes: 15
  on_exhausted: stop_and_report_partial

gates:
  requires_approval:
    - send        # email, DM, anything outbound
    - publish     # site, listing, social
    - deploy
    - delete
    - spend       # any paid API or new credential

What should you actually measure?

Total tokens is a vanity metric. Track cost per finished task: a run that produced something a person accepted without rework. Divide monthly spend by accepted outputs and you get a figure you can hold against what the same task used to cost in staff time. That is the number that decides whether a workflow expands or gets retired.

• Cost per accepted output, broken out by workflow
• Correction rate: how often a person rewrites the result
• Share of runs that hit a ceiling instead of finishing
• Cache hit rate on the two or three workflows you run most

Most teams do not need a metered key in month one. Start on the subscriptions, local models, and free allowances already paid for, prove one workflow end to end, then buy metered access for that workflow specifically. Keep the external API budget at zero until a workflow has earned the exception, and require a decision from a person before any ceiling moves.