Technology & AI 20 November 2026 6 min read

FinOps for AI: Getting Inference Costs Under Control

Engineer working on a laptop in a server room with cloud and automation icons

For years the cloud bill had a familiar shape: compute, storage, data transfer and a long tail of managed services. AI has added lines that behave differently. Token charges from model APIs, GPU instances that cost the same whether busy or idle, vector databases, and AI features folded quietly into SaaS renewals. Many finance teams first notice them when a monthly invoice jumps and nobody can say which product caused it.

The FinOps Foundation's State of FinOps 2026 report, published in February 2026 and based on 1,192 respondents, found that 98% of them now manage AI spend, up from 31% two years earlier. Practitioners named FinOps for AI as their top forward-looking priority. In June 2026 the Linux Foundation went a step further and launched a Tokenomics Foundation to work on practices for token-based AI costs. The discipline is catching up because it has to.

The principles have not changed. Spend should be visible, owned and tied to value. What has changed is the unit of consumption, and how quickly a single design decision can move the bill.

Why AI spend behaves differently

AI costs are driven by behaviour, not provisioning. A developer who adds three pages of context to every prompt, or an agent that loops through ten tool calls where two would do, multiplies cost without any infrastructure change that anyone would normally review.

Pricing also varies widely between models for the same task, and the most capable model is rarely the cheapest adequate one. And the spend is scattered: some on hyperscaler invoices, some on direct contracts with model providers, some on corporate cards, and some buried inside software subscriptions where it never appears as AI at all.

That scattering is why the first job is visibility. Optimisation comes later.

Make the spend visible and give it an owner

Start by finding all of it. Bring model API invoices, GPU and AI platform services from your cloud bills, and the AI line items in SaaS contracts into one view. Expect surprises: proofs of concept that were never switched off, a team paying list price on an account nobody in procurement knows about, two business units buying the same capability twice.

Then allocate it. Ordinary tagging gets you some of the way for GPU instances and managed AI services. Model APIs usually need more, because a single API key often serves several applications.

  • One key or project per application. Most providers let you separate usage this way. It is the cheapest allocation control you will ever implement.
  • A thin internal gateway. Route model calls through a service that records the application, feature, team and, where it matters, the customer for every request, along with tokens in and out.
  • Shared platform costs split by usage. GPU clusters and vector stores that serve several products need an agreed split, based on measured consumption rather than headcount or guesswork.

The argument we made about cloud cost as a strategy problem applies with more force here: the person who decides how a product uses AI should see what it costs within days, not at the next quarterly review.

Set unit economics the business recognises

Total AI spend is a poor management number. It rises when adoption succeeds, which is rather the point. What you want is cost per unit of value delivered.

Good units are specific to the use case: cost per resolved support ticket, per invoice processed, per contract reviewed, per qualified lead. Each needs two halves. The cost side comes from the allocation work above. The outcome side has to come from the business process, and it has to count outcomes rather than attempts. A ticket the AI "handled" that came back two days later as a complaint was not resolved.

Picture a hypothetical service desk where the AI assistant costs a few pence per conversation but fully resolves only a minority of them. The useful comparison is the fully loaded cost per resolved ticket, AI and people together, against the people-only baseline. That framing prevents two common mistakes: celebrating a low token bill for an assistant that resolves very little, and panicking about a rising bill for one that is quietly carrying most of the volume. Our guide to measuring AI ROI goes further on connecting these figures to the P&L.

If you cannot say what a resolved ticket costs, you cannot say whether your AI is cheap or expensive.

The engineering levers that move the bill

Once costs are visible and tied to units, the optimisation work tends to become obvious, and much of it is architectural.

  • Route to the smallest model that does the job. Classification, extraction and routing rarely need a frontier model. A router that sends simple requests to a small model and escalates only the hard cases is often the largest single saving, as we discuss in choosing AI models. Test quality on your own cases before and after the switch.
  • Cache what repeats. Long system prompts, policy documents and product catalogues sent with every request suit prompt caching. Anthropic, for example, prices cache reads on most of its models at a tenth of the normal input rate, although writing to the cache costs more than a standard input token, so it pays only for content that is reused.
  • Batch what can wait. Overnight document processing, back-office enrichment and evaluation runs do not need instant answers. OpenAI's Batch API charges half the synchronous price for work that can finish within a 24-hour window.
  • Trim the context. Retrieval that sends twenty passages when three would do, and conversation histories that are never summarised, are paid for on every call.
  • Cap the loops. Agents should have a maximum number of steps and tool calls per task, with a handover to a person when they reach it.

For self-hosted models the levers move towards GPU utilisation: right-sizing instances, sharing clusters across workloads, scaling down outside working hours where latency allows, and buying commitments only once the baseline is stable.

Budgets and guardrails that do not block the work

Hard spending caps on AI are blunt instruments. Set too low, they switch off a customer-facing feature at the worst possible moment. Set too high, they never trigger.

A layered approach works better. Give each application a forecast and a budget, with alerts at sensible thresholds that go to the product owner and not only to finance. Add rate limits and per-user or per-tenant quotas at the gateway, so one runaway integration cannot consume a month's budget over a weekend. Make a cost estimate at expected volume part of approving any AI use case for production. And review unit cost monthly next to quality metrics, because a model change that saves money while quietly lowering accuracy is not a saving.

Expect the numbers to move. Model prices change, new models appear, and usage shifts as people learn what a tool can do. Treat the first few months of unit economics as provisional, and revisit routing and caching decisions whenever prices or volumes change materially.

Common questions

What is FinOps for AI?

FinOps for AI applies the established FinOps practice of making technology spend visible, allocated and accountable to AI workloads: model API tokens, GPUs, AI platform services and AI features in SaaS. The emphasis shifts towards per-request allocation and unit economics, because AI cost is driven by usage patterns as much as by infrastructure.

How do you reduce LLM inference costs?

The biggest levers are usually routing simple requests to smaller models, caching repeated context, batching work that is not time-sensitive, trimming unnecessary context and limiting agent loops. Measure quality alongside cost so that savings do not come from worse answers.

How should AI costs be allocated to products?

Give each application its own API key or project, and route model calls through a gateway that records application, feature, team and customer for each request. Split shared GPU and platform costs by measured usage. That data then supports unit metrics such as cost per resolved ticket or per document processed.

Where Ganexa can help

Ganexa helps CIOs and finance leaders bring AI spend into the same discipline as the rest of the technology budget, from allocation design and unit economics per use case to model routing and the guardrails that keep costs predictable as adoption grows. Our AI strategy and implementation work covers both the architecture and the operating model. To look at where your AI spend is really going, book a free consultation.

FinOpsAI CostsCloud CostUnit EconomicsLLMs