The missing layer in the AI stack: operational integrity

The stack as it stands

Compute. NVIDIA dominates GPU hardware. AWS, Azure, and Google compete on cloud GPU instances. The compute layer is commoditising, but the demand curve is steep.

Models. OpenAI, Anthropic, Google, Meta, and Mistral compete on foundation models. Capabilities improve quarterly. The model layer is intensely competitive and increasingly interchangeable for many use cases.

Context. Pinecone, Weaviate, Chroma, and dozens of others provide vector storage. LangChain, LlamaIndex, and similar frameworks handle retrieval and augmentation. The context layer connects models to organisational data.

Orchestration. LangGraph, CrewAI, Temporal, and emerging frameworks manage multi-step agent workflows. They handle state, branching, tool invocation, and error recovery. The orchestration layer coordinates agent execution.

Agents. Goal-oriented actors combining model reasoning, context retrieval, and tool use. The agent layer is where autonomous behaviour emerges.

Applications. Copilots, AI SaaS products, enterprise assistants. The application layer is what users interact with.

Each layer has active competition. Billions in funding. Clear market dynamics. One layer between agent execution and applications has almost no competition, because almost nobody is building it yet.

The gap

Between agent execution and the application layer, there is no standard infrastructure for operational integrity.

No system ensures that what agents did is recorded in a structured, verifiable format. No protocol standardises how agent decisions are documented. No chain of custody exists for AI operational actions.

Everyone assumes logs are enough. They are not.

Logs are unstructured text. They are mutable. They capture function calls, not decisions. They do not link causally. Two log entries written 50ms apart might be completely unrelated or the reason your system broke. Reconstructing an agent's decision process from application logs is like reconstructing a trial from court transcripts written in shorthand by three different clerks who each captured different parts.

The gap is not observability. Datadog, Honeycomb, and OpenTelemetry handle metrics, traces, and spans. The gap is accountability: structured evidence of what information an agent considered, what decision it made, what action it took, and whether the chain of events is verifiable.

Historical parallels

This gap has appeared before in other parts of the infrastructure stack.

Each time, a neutral coordination layer emerged when the underlying technology became complex enough that ad-hoc approaches stopped working. The pattern is consistent: fragmentation, pain, standardisation, adoption. AI operations are entering the pain phase now.

A protocol-based approach

The most durable infrastructure tends to be protocol-based, not product-based.

HTTP is a protocol. Nginx and Apache are products. TCP/IP is a protocol. Every network device implements it. Git is a protocol (and a tool). GitHub, GitLab, and Bitbucket are products built on it. JSON is a format. Every API uses it.

Products compete. Protocols compound. A product captures value from its users. A protocol creates value for its entire ecosystem. The companies that build on open protocols tend to outlast the ones that build proprietary alternatives, because network effects favour open standards.

An operational integrity layer built on open protocols follows this pattern. Structured event envelopes for agent communication. Orchestration control layers for governance. Verifiable operation ledgers for tamper-evident audit trails. Content-free receipt chains for witness attestation.

These are not product features. They are protocol primitives. Any agent framework can implement them. Any enterprise can adopt them. The reference implementation captures value. The protocol creates the ecosystem.

The timing question

The AI stack is forming now. Most investment and attention goes to models and agents. That makes sense. Those layers deliver immediate capabilities.

But the layer that makes AI trustworthy in production may be the most valuable in the long term. Compute is commoditising. Models are converging. Context is becoming table stakes. The differentiation will shift from "what can AI do?" to "can you prove what it did?"

The operational integrity layer is still wide open. No dominant player. No established protocol. No standard approach. The companies and communities that define this layer will shape how AI is governed for the next decade.

That window will not stay open indefinitely. The first serious AI incident at a regulated enterprise will accelerate demand overnight. The infrastructure needs to exist before the crisis, not after.

More in this series: Verifiable AI Operations