Supporting Multiple Models Doesn’t Make You Model-Agnostic

Many AI products now describe themselves as model-agnostic. Usually this means they can call more than one model API.

That is useful supplier optionality. It is not yet model independence.

The market often uses “model-agnostic” or “vendor-agnostic” for the weaker claim that several providers can be plugged in. Nadella’s “choice” test is more demanding: could one model disappear without taking the company veteran with it? That is the standard worth using because it tests architectural independence, not the size of a dropdown menu.

An application can offer a dropdown containing OpenAI, Anthropic and Google while remaining deeply dependent on one model’s behaviour. If the workflow is encoded in provider-specific prompts, tool semantics, conversation state and agent conventions, switching the model can change the product rather than merely changing the execution engine.

A model dropdown is not model independence

The weakest form of model choice is manual selection: the user or administrator chooses a provider and all work goes there.

Routing by capability, speed, cost and risk is better. But even clever routing is only one part of model independence. The surrounding workflow must continue to function when the selected model changes.

That requires a stable contract between the application and the models: normalised tool definitions, structured outputs, validation, external workflow state and evaluations that compare results against the business outcome rather than against a preferred model’s wording.

The company knowledge and operating logic must live outside the model

Every enterprise AI product has some capability that should survive a supplier change. In ProdPad, that includes understanding the product graph, recognising workflow progress, enforcing permissions, deciding which actions are available and knowing what counts as completion.

If those capabilities exist only as instructions inside one model’s context, the enterprise does not really own them. It has rented a behaviour that may change with a model update, pricing change or provider decision.

ProdPad Conductor keeps that company knowledge and operating logic in the orchestration layer. Models can interpret intent, synthesise evidence and generate language. They do not own the state or rules of the product workflow.

Different steps deserve different models

Model independence is not only an exit strategy. It allows the application to use the model menu intelligently.

A fast low-cost model may classify a request or select from a bounded set of choices. A stronger model may handle ambiguous synthesis. A specialist model may be better for a particular language or document type. Deterministic software may be the right answer for validation, arithmetic or permissions.

The orchestration layer can make those decisions based on metadata and measured performance rather than brand loyalty. It can also change the routing policy without redesigning the user experience.

Evaluations make model choice operational

“Eval-driven development” and “agent improvement loop” are becoming the standard language for this. Traces show what happened. Feedback identifies what was wrong. Evals turn those failures into repeatable tests. Only then can model routing and harness changes be compared against evidence rather than taste.

Without your own evaluations, model selection quickly becomes pub argument territory. One model feels more articulate, another tops a public benchmark, and a third is cheaper on the price table.

Enterprise evaluations should test the actual workflow. Did the model use the right evidence? Did it return a valid structure? Did the user accept the recommendation? Did the workflow complete without repair? What did the task cost and how long did it take?

The same evaluation set can then compare models and routing policies. This is how model choice becomes an optimisation problem rather than a quarterly argument about which provider is ahead.

Provider-specific features still have value

Model independence does not require reducing every provider to the lowest common denominator.

A capable orchestration layer can use native tool calling, prompt caching, structured output or other provider-specific features behind a common interface. The important distinction is that those optimisations improve execution without becoming the source of the product’s business rules or memory.

The application should be able to exploit a provider’s strengths without making the entire workflow inseparable from them.

Model changes should not erase the company veteran

Nadella distinguishes between the generalist model and the “company veteran” capability built from enterprise context and learning. That veteran should remain even when the generalist changes.

In practical terms, the veteran is not a single fine-tuned model. It is the combination of product data, graph relationships, workflow state, evaluations, corrections, permissions, tool integrations and accumulated decisions that make the system useful inside one organisation.

Those are orchestration assets. Keeping them outside the model protects both continuity and bargaining power.

What model independence means in ProdPad

Conductor maintains a registry of available models and the metadata required to choose between them: capability, speed, cost and supported behaviours. The workflow can request the kind of execution it needs while the orchestration layer translates that request into provider-specific settings.

Responses are validated against a common contract. Workflow state and entity references live in ProdPad. Tools are exposed according to the current step rather than according to whatever an unconstrained agent would like to call.

That means a model can be replaced, upgraded or reserved for different work without moving the product organisation’s memory and operating rules with it.

Supporting several models is useful. Being able to lose any one of them without losing the way your company works is model independence.