When enterprise teams talk about reducing AI costs, the conversation usually starts with model prices. Which provider has cut input-token charges? Can a smaller model handle the task? Should routine work be routed away from the frontier model?

Those are sensible questions. They are also downstream of a more important one: why are we sending the model so many tokens in the first place?

Most discussions about AI cost begin with the model price table. That is understandable, visible and often the wrong place to start. Recent research ran the same 22 enterprise tasks on six foundation models and changed only the orchestration harness around them. Cost fell by 33–61% for every model. Tokens per task fell 38%, median latency fell 44%, and aggregate completion quality remained broadly level.

The orchestration layer moved cost more than switching between the cheapest and most expensive model in the test. That should change how enterprises think about AI efficiency.

Falling token prices can hide rising task costs

Token prices have fallen quickly, which creates the impression that model usage will naturally become cheaper. But agentic systems are also consuming more tokens per task: longer reasoning traces, larger tool catalogues, more retrieved context, more turns and more complete conversation histories replayed on every step.

The unit price can fall while the total bill rises. The useful unit of measurement is therefore not cost per token. It is cost per successfully completed task.

An inexpensive model in a wasteful loop can cost more than a frontier model inside a disciplined workflow. It can also deliver a worse result and require more human checking.

Most input tokens are an application decision

A model controls the length of its own answer. It does not decide most of what appears on the input side.

The application assembles the system instructions, conversation history, tool schemas, retrieved documents, intermediate results and the current user request. Four of those categories are constructed by software before the model sees anything.

This is why orchestration is a P&L concern rather than plumbing. The layer around the model decides whether every turn receives the entire transcript, every available tool and every piece of related data, or only the information required for the current step.

The naive agent loop gets more expensive as it gets longer

A simple agent often replays its full history on each turn. As the conversation grows, each new step includes all the tokens from the previous steps. The total input grows much faster than the useful work.

A larger context window makes this possible. It does not make it sensible. A bigger skip is not an instruction to fill it.

A better orchestration layer separates stable context from volatile context. Stable instructions and tool definitions can be cached. Older conversation can be compacted into a structured state. Large tool outputs can be stored outside the prompt and referred to when needed. A workflow that is waiting for a person or external event can suspend rather than poll a model repeatedly.

The aim is not crude prompt shortening. It is to preserve the information that changes the decision and stop paying to resend everything else.

A workflow can choose the cheapest reliable path

Cost control is not only about context. Different parts of a task require different capabilities.

A bounded classification may be handled by a fast, inexpensive model. Complex synthesis across contradictory evidence may justify a frontier model. Identifier validation, arithmetic, permissions and completion checks should often be handled deterministically rather than delegated to a model at all.

ProdPad Conductor can make those choices step by step. It can also stop a workflow once the completion rule is satisfied, repair a failed structured response without restarting the entire task and prevent an agent from repeatedly exploring the same dead end.

This is more useful than assigning one model to the whole process and hoping its average cost is acceptable.

Employee-built agents create invisible duplication

The cost problem becomes harder when every team or employee creates separate agents. Each may carry its own long prompt, duplicate the same product context, define overlapping tools and repeat work another agent has already performed.

Anthropic has made the same point from another direction in its work on MCP and code execution: loading every tool definition and passing every intermediate result through the context window increases both latency and cost. Loading tools on demand and filtering results before they reach the model is orchestration hygiene, not a cheaper-model trick.

Individually, none of those agents looks particularly expensive. Collectively, they become shadow IT with a token bill: duplicated context, duplicated tools, duplicated mistakes and no shared way to tell whether the work was actually completed.

Enterprises need room for experimentation. They also need a path from experimentation to a shared orchestration layer where successful workflows can be governed, measured and improved once rather than copied endlessly.

Measure completions per million tokens

Teams that report only answer quality or model usage will naturally token-max. More context and more reasoning can appear safer because the compute cost is somebody else’s line item.

A better set of measures connects spend to the outcome:

• successful task completions per million tokens
• quality or acceptance rate per pound or dollar
• cost and elapsed time per completed workflow
• retry, repair and human-intervention rates
• cost by workflow step and model

These measures reveal whether a more expensive model genuinely reduces total workflow cost, or merely moves the spend to a more visible line.

ProdPad’s AI should reduce the cost of product work, not create another bill to manage

Product management is particularly vulnerable to token waste because the context is broad and connected: feedback, ideas, objectives, roadmaps, decisions and conversations all appear relevant.

They are not all relevant to every step.

ProdPad Conductor uses workflow state and the product graph to retrieve the context required for the current task, select the appropriate model or tool, validate the result and stop when the work is complete. That gives product teams the benefit of rich organisational context without blindly pouring the whole product system into every prompt.

The model price still matters. But the larger enterprise lever is architectural: cache the stable, compact the old, retrieve the relevant, route the bounded and stop the failing.

The cheapest token is not the discounted token. It is the token the workflow never needed to send.

Shadow AI is also shadow spend

IBM’s newer definition of shadow AI goes beyond somebody pasting a document into an unapproved chatbot. It includes teams creating agents, connecting models, adding MCP servers and deploying autonomous workflows faster than the organisation can discover or govern them.

That is also a cost problem. Each hidden agent can duplicate context, repeat tool definitions, choose its own model and retry failures without a common stopping rule. The individual bill may look trivial. The combined agentic estate can become expensive long before finance can explain what the money produced.

The answer is not to ban experiments. It is to give successful experiments a route into shared orchestration, where context, routing, traces, completion and cost can be managed once rather than reinvented in every team.