Your Enterprise AI Trust Boundary Is Too Small

It is worth distinguishing this from the “trust layer” language used by platforms such as Salesforce. A trust layer enforces protections around an interaction: identity, permissions, prompt defence, data masking and output filtering. The trust boundary is the larger organisational boundary around what may leave the company at all, including traces, evals, corrections and the learning derived from them. You need both, but they solve different problems.

Enterprise AI security discussions usually begin with data: where it is stored, whether it is used for training, who can access it and how long the provider retains it.

Those questions remain necessary. They are no longer sufficient.

Most enterprise AI security discussions start and end with the documents: where they are stored, whether they are used for training and who can retrieve them. That is necessary, but it misses the more interesting thing the system is collecting. As AI becomes part of daily work, the organisation also creates a record of how it thinks and acts. Prompts reveal what people are trying to accomplish. Tool calls reveal how work is performed. Corrections reveal what experts know. Evaluations reveal what the organisation considers good. Workflow state and memory reveal how decisions accumulate over time.

That is the knowledge an enterprise AI system needs in order to become genuinely useful. It is also the knowledge that makes the enterprise distinctive.

Protecting data is not the same as protecting learning

A conventional security boundary can ensure that a document is encrypted and not used to train a public model. Yet the surrounding application may still scatter prompts, traces, evaluations, corrections and workflow memory across several providers and employee-created agents.

The files can remain perfectly protected while the organisation quietly gives away control of the learning loop around them.

Satya Nadella calls this the Reverse Information Paradox: the buyer pays for intelligence, then risks giving away the proprietary knowledge required to make that intelligence useful. His proposed answer is an enterprise trust boundary that contains not just data but memory, traces, feedback, decisions, evaluations and adapted capability.

That is an important distinction. In the cloud era, companies accumulated data. In the AI era, they also accumulate learning.

The trust boundary must include the orchestration layer

The model does not create the whole learning loop. The orchestration layer decides what context reaches it, which tools it may use, how the result is evaluated, what corrections are retained and how that experience changes the next workflow.

Protecting the database while renting all of those decisions is rather like locking the filing cabinet and handing somebody else the keys to the factory.

The orchestration layer therefore belongs inside the trust boundary. It should preserve:

• workflow state and completion rules
• evaluations and definitions of a good outcome
• traces, corrections and human interventions
• organisational memory and entity relationships
• permissions and authority by workflow step
• model-performance, cost and reliability history

Those assets should remain useful even when the enterprise changes model providers.

Corrections are institutional knowledge

When an experienced product leader corrects an AI-generated recommendation, the value is not simply that one answer becomes better. The correction may reveal that a particular customer segment matters more, that an apparent request conflicts with strategy, or that an internal term has a precise meaning the public model could not know.

If the correction disappears into a transcript, the organisation pays to rediscover it. If it becomes structured workflow knowledge, the next decision can begin from a better position.

This does not automatically mean fine-tuning a model. In practice, the better fix is often much more prosaic: change the retrieval, the workflow rule, the validation, the context or the point where the system asks a person. The organisation learns by improving the system around the model as well as the model itself.

Private evaluations define what good means here

Generic benchmarks can show that a model is capable. They cannot tell a company whether the model made a good decision in its specific context.

Enterprise evaluations should reflect real workflows, policies, data and trade-offs. Did the system use the right evidence? Did it respect authority? Did it distinguish an isolated request from a recurring pattern? Did it complete the right action and stop at the right point?

These evaluations are proprietary because they encode how the organisation judges work. They should be portable across models and retained with the orchestration layer rather than tied to one provider’s agent environment.

Choice is part of trust

An enterprise does not fully control its learning loop if removing one model also removes its ability to operate.

True choice means that workflow state, tools, evaluations, memory and business rules remain outside the model. A different provider may reason differently, but it can still participate in the same governed process and be measured against the same definition of success.

That makes model independence a resilience property as well as a procurement advantage.

ProdPad already contains the beginnings of a product organisation’s learning loop

Product organisations create decisions continuously: why an idea matters, which evidence supports it, how it relates to strategy, what was rejected, what changed and what happened afterwards.

ProdPad already provides a connected product system in which those decisions can live. ProdPad Conductor adds the orchestration required to use that context safely: selecting what is relevant, applying workflow rules, validating actions and retaining structured state instead of relying on a model to remember a conversation.

That is materially different from sending company documents to a chatbot. The objective is not merely to answer questions about the product organisation. It is to help the organisation work while ensuring that what the system learns from that work remains part of the organisation’s own capability.

A real enterprise AI trust boundary protects not only what the company knows, but how the company learns.

Governance becomes real only when it reaches the running workflow

IBM now distinguishes periodic AI governance from continuous AI assurance. That is a useful correction. A policy can record what an agent was approved to do. Assurance asks whether the live system is still doing it after the model, tools, prompts, data and connected agents have changed.

This is where orchestration-led governance matters. Permissions, approval points, validation and escalation cannot exist only in a governance catalogue. They have to be exercised by the layer that actually assembles context, exposes tools and authorises actions.

The trust boundary therefore needs both ownership and enforcement: ownership of the traces, evaluations and learning loop, and runtime controls that make those rules true while the work is happening.