Token Cost Management for AI-Assisted Modernisation Programmes

Inference spend is a new cost category for modernisation budgets

Engineering leaders know how to manage cloud spend and people costs. An AI-assisted programme adds a third category, inference, which is small next to salaries but behaves differently from anything else in the budget. It scales with conversation length rather than headcount, a single badly structured workflow can make it spike, and finance teams will ask about the first spike.

The cost drivers are few, and the controls are well understood. Most published advice, though, is developer-level thrift (shorter prompts, smaller models) rather than programme-level policy. This guide covers the policy level: what to control, in what order, and what to leave alone.

What drives the bill?

Three drivers account for most of the bill, and they are not equally important.

Repeated input. Every turn of an agentic session re-sends the conversation so far, including the system prompt, project context, file contents, and tool results. On a long modernisation session that prefix is large and changes little from turn to turn, so without caching you pay for it in full every time. It is usually the biggest single component of spend, and prompt caching addresses it directly.

Model tier. The price spread between the smallest and largest models in a family is roughly an order of magnitude, on both input and output. Routing high-volume mechanical work through the top tier is the most common avoidable expense.

Output and thinking. Output tokens cost more than input tokens, and extended thinking counts as output. Deep reasoning is worth paying for on hard problems and wasted on boilerplate, which is another reason to choose the tier by task rather than by habit.

Check current per-model rates on Anthropic’s pricing page rather than relying on any blog post, including this one, because rates and cache pricing change with model releases.

How should a programme set model-tier policy?

Write the policy down, assign work classes to tiers, and review it against outcomes. The three-tier split below has emerged as the published consensus, and it matches our own delivery experience:

Three programme-level rules turn the policy into practice:

1. Defaults live in configuration. Set the default model per repository and per subagent in the tooling configuration, so the policy applies without anyone having to remember it. Subagent definitions can pin cheaper models for bounded tasks while the lead session stays on the workhorse tier.
2. Escalation needs a reason. Moving a task up a tier is fine, but the policy should name who decides and on what grounds, so the top tier does not drift into being the default.
3. Review quarterly. Model capability moves quickly, and the boundary between tiers moves with it, so a task that needed the top tier last quarter may not need it now. We publish our own quarterly review of tooling and models in the AI Velocity Report.

Many teams arrive at this split on instinct, and the instinct is sound. Moving it from developer preference into configuration and governance is what makes it hold across a programme.

Why is prompt caching the highest-leverage control?

Prompt caching acts directly on the largest cost driver, repeated input. It writes the stable prefix of a request (system prompt, project context, schemas, long documents) to a cache once, then reads it back at a much lower rate on later requests. Cache durations of 5 minutes and 1 hour are available. Agentic tools such as Claude Code use caching automatically, while API-based workloads need to be designed for it explicitly. See Anthropic’s prompt caching documentation for the mechanics and current rates.

The design rules stay the same even as rates change:

• Order prompts cache-first. Stable content (system instructions, CLAUDE.md-style context, database schemas, reference documents) goes in front, and variable content (the current question, the current file) goes behind it. A variable element early in the prompt invalidates the cache for everything after it.
• Keep the stable prefix unchanged. Timestamps, request identifiers, or per-user content injected into a system prompt will prevent cache hits, and the only sign is in the usage data.
• Match cache duration to session shape. Long modernisation sessions and batch pipelines benefit from the longer duration, while sporadic interactive use may not.

For platform teams, note that on some platforms cached reads also fall outside input rate limits, so caching buys throughput as well as a lower price. Microsoft’s Foundry documentation, for example, excludes cache reads from input tokens per minute.

How does context engineering reduce spend?

Most wasted tokens come from wasted turns. When the agent produces the wrong thing, the session pays for the misunderstanding twice, once to make the mistake and once to correct it, and context engineering reduces how often that happens.

• CLAUDE.md and AGENTS.md carry the conventions once, so sessions do not each re-explain the target architecture, naming rules, and forbidden patterns. Our .NET configuration guide covers the setup.
• Skills package repeatable workflows, so a migration step is invoked rather than described again each time. Our skills post covers the authoring principles.
• Scoped context costs less than bulk context. Feeding an agent a large database’s whole schema on every request costs more, and gives worse accuracy, than serving the few tables the task needs. A scoped, read-only Model Context Protocol (MCP) source does this, as covered in our scoped database context post. The wider strategies are in scaling coding agents across large codebases.
• Deterministic checks prevent expensive loops. A hook that fails a build immediately after a bad edit stops the agent running five more turns on top of the mistake. We treat harness engineering as a cost control as well as a quality control, and the harness engineering guide covers it in depth.

When is a multi-model environment worth the complexity?

In most cases it is later than the tooling market suggests. Gateways and routers that spread requests across providers exist and are sometimes the right choice, but their costs are easy to underestimate:

• another piece of infrastructure to run
• another supplier, or several, to take through vendor assessment
• divergent failure modes and model-specific prompt behaviour
• a wider governance surface for your compliance team to reason about

For a regulated business, each additional provider brings a data protection assessment as well as an API key. Our Claude GDPR guide shows what one provider’s assessment involves.

We recommend this sequence:

1. Exhaust single-vendor tiering first. Large, medium, and small models from one provider cover most of the cost curve with one contract, one data processing agreement (DPA), and one set of behaviours.
2. Add a second provider for a measured, recurring workload rather than an assumed one. The credible case is a specific job with evidence behind it, such as a workload whose cost or quality profile is demonstrably better elsewhere at sustained volume.
3. If you do route, route deterministically. Task-class routing set in configuration can be audited. Per-request ‘smart’ routing puts another model in charge of unlogged decisions about where your data goes.

We design environments around the work rather than around loyalty to a vendor, and we publish our tooling decisions and their outcomes quarterly in the AI Velocity Report. For most delivery workloads, the evidence so far favours using more tiers from one provider over adding more providers.

How do you meter and govern spend?

Metering works like cloud cost management on a smaller scale:

1. Instrument from week one. Track usage per developer, per repository, or per workstream, whichever maps to how the programme reports. The tooling exposes usage data, so capture it before anyone asks for it.
2. Set expected ranges rather than hard caps. An agent blocked mid-migration costs more in engineer time than the tokens it wanted. A range prompts a conversation when spend moves outside it, whereas a cap stops the work.
3. Review weekly for the first month, then monthly. Early sessions reveal workflow mistakes such as uncached batch jobs, top-tier defaults, and context bloat while they are still cheap to fix.
4. Attribute spend to outcomes. Cost per merged change is a more useful number than cost per day. It also makes the budget conversation easier, because inference spend is small next to the engineering time it replaces. The business-level framing is covered in what AI-augmented development means for your budget.

Next steps

If you are planning an AI-assisted modernisation programme, build the cost controls into the environment design from day one. That means tier policy in configuration, caching-aware prompt structure, context engineering, and metering. Retrofitting them after the first surprising invoice is harder, and it undermines confidence in the programme.

For the wider environment setup, see the .NET configuration guide and the harness engineering guide. Our Claude Code for .NET developers guide covers the project files and everyday workflows that these controls sit alongside. For the modernisation programmes these controls serve, start with the .NET modernisation playbook.

Want the controls designed in from the start? Book a free consultation.