Infrastructure and compute

Target state

In short: Every agent runs in its own isolated session, with its own identity, behind a controlled exit, on rented capacity by default.

Agents run on managed or platform-provided runtimesAgent platform. The software that runs agents: the runtime, the harness, the list of approved agents, and the rules for how they start, stop, and change., and each session is isolated from every other. Work that can act on the world (running code, browsing, using a computer) executes in a sandboxSandbox. An isolated environment where agent code runs with limited access to the network and the rest of the machine, so mistakes and attacks stay contained. of the micro virtual machine (microVM) class. Each sandbox sits behind an egress proxy, which enforces egress controlEgress control. Rules about what a sandbox or agent is allowed to send out to the internet or to other systems.: rules on what may leave. Every agent holds a first-class workload identityWorkload identity. Proof of who a piece of running software is, issued by the platform rather than typed in as a password. The machine equivalent of an ID badge., the machine equivalent of an ID badge. No agent shares a human's credentials. Sessions are event-logged, so they can resume after a crash, and they hibernate when idle. That matters because orchestrationOrchestration. Coordinating several agents or steps so they work together on one job. Useful for breadth; it does not make any single step smarter. work comes in bursts and is bound by input and output (I/O). It sits idle for 30 to 70 percent of elapsed time, waiting on modelModel. The AI engine that reads and writes text. On its own it has no access to your systems; everything built around it decides what it may do. and toolTool. A function an agent can call to do something real: look up a record, send an email, run a query. Tools are where agents touch your systems. calls. Model access is metered (paid per use) by default. It is routed to sovereignData residency and sovereignty. Rules about which country or region data may be stored and processed in, including the copies created for agents. or self-hosted serving only when the data classificationClassification (data). A label for how sensitive data is. Anything derived from data inherits the strictest label of its sources. requires it. Capacity and cost are managed per agent class, with idle time priced in. Three kinds of infrastructure carry all of this, and none existed in the pre-agent estate: execution sandboxes, durable session infrastructure, and egress-controlled networking.

layer · architecture

Execution infrastructure: sandboxed, egress-controlled, durable

Act-capable work runs in microVM-class sandboxes behind allowlisting egress proxies, on event-logged resumable sessions, under a first-class workload identity.

1. 01Agent

   Agent

   Agent runtime

   Managed or platform-provided, per-session isolation

2. 02Control

   Control

   microVM sandbox

   Code, browsing, computer use; session memory sanitized on termination

3. 03Control

   Control

   Egress proxy

   Domain allowlists; metadata endpoints and private ranges blocked

4. 04System

   System

   Durable session

   Event-logged, snapshot/restore, hibernate-and-wake

5. 05Evidence

   Evidence

   Workload identity

   Directory, WIMSE/SPIFFE, or XAA-based; ID2 floor

6. 06System

   System

   Model serving

   Metered by default; sovereign or self-hosted by classification

Gate on capability set (private data, untrusted content, external communication), re-evaluated on every tool grant.

Diagram description: Execution infrastructure with agent runtime, microVM sandbox, egress proxy, durable session log, workload identity issuance, and classification-routed model serving. The map contains Agent runtime: Managed or platform-provided, per-session isolation; microVM sandbox: Code, browsing, computer use; session memory sanitized on termination; Egress proxy: Domain allowlists; metadata endpoints and private ranges blocked; Durable session: Event-logged, snapshot/restore, hibernate-and-wake; Workload identity: Directory, WIMSE/SPIFFE, or XAA-based; ID2 floor; Model serving: Metered by default; sovereign or self-hosted by classification. Its connections are runtime to sandbox for act-capable work; sandbox to egress for all network egress; runtime to session for event log; identity to runtime for issued per agent; runtime to serving for classification-routed. Important boundary: Gate on capability set (private data, untrusted content, external communication), re-evaluated on every tool grant.

Reviewed 2026-08-20·Sources:r01-infrastructure findings

Mechanisms

Isolation classes and the capability-set gate

In short: Agents that can act need the strongest sandbox class, judged by what they can do, not by product name.

Agents that can act require isolation of the microVM, gVisor, or Hyper-V class. That is now industry consensus across all three hyperscalers (the largest cloud providers) and both major labs. Bedrock AgentCore runs per-session microVMs (generally available October 2025, sessions up to 8 hours, with virtual private cloud (VPC) and PrivateLink connectivity) [vendor]. Azure Container Apps dynamic sessions are Hyper-V isolated, with egress off by default [vendor]. OpenAI gives each session its own virtual computer (July 2025) [vendor]. Anthropic open-sourced its sandbox-runtime, which enforces domain allowlists at a proxy (October 2025; 84 percent fewer permission prompts, self-reported) [vendor]. The Cloud Native Computing Foundation (CNCF) Agent Sandbox project brings gVisor with Kata support to Kubernetes. It was announced in November 2025 and was not production-ready as of April 2026. The OWASP (Open Worldwide Application Security Project) Agentic Top 10 (December 2025) and Unit 42 guidance concur. Plain containers are not a sufficient boundary. Frontier models (the most capable current models) exploit container weaknesses when those weaknesses are present (SANDBOXESCAPEBENCH, arXiv 2603.02277). Start time separates the classes: containers start in under 90 milliseconds (Daytona [vendor]), while microVMs take seconds. The gating rule: decide by the agent's capability setCapability surface. Everything an agent can actually do: which systems it can read, write, send from, or appear in. Controls are chosen from this list, not from the product's name. (what it can actually do), not by product category. Apply the "lethal trifecta" or "Rule of Two" frame. It asks whether the agent has private data, reads untrusted content, and can communicate externally or change state. Re-evaluate it on every tool grant, because agents accumulate tools over time, and that erodes whatever category an agent was assigned at launch.

Egress control

In short: Nothing leaves a sandbox except through a proxy that allows only named destinations.

All outbound traffic from sandboxes and agents passes through allowlisting proxies. There is no default route out. Block cloud metadata endpoints and private address ranges from inside sandboxes (Unit 42 guidance). The metadata endpoint is how a sandboxed process turns into your cloud credentials. Shipped defaults confirm the posture. Azure dynamic sessions arrive with egress off [vendor]. Anthropic's sandbox-runtime enforces domain allowlists at a proxy [vendor]. AgentCore added VPC and PrivateLink connectivity in September 2025, and VPC egress for its gatewayGateway. The single door every agent request passes through on its way to a tool or model. Because everything goes through it, it is where rules are enforced and activity is recorded. and identity components in April 2026 [vendor]. Private connectivity to model providers (the PrivateLink class) exists on every hyperscaler, and it is opt-in, never the default. Egress is a first-order control, not hygiene. The headline exploit of 2025 (EchoLeak, below) was stopped by nothing in the compute stack. Egress and output restriction were the decisive controls.

Workload identity: three families

In short: An agent gets its own identity in one of three ways, and sharing a person's login is the failure to avoid.

First-class agent identityAgent identity. A registered account for an agent with a named owner, limited permissions, and credentials that expire, so every action can be traced to who asked. ships in three families. Directory-based: the Entra Agent ID class (preview May 2025) [vendor], where agents become principals (accounts) in the directory. Standards-track: the WIMSE architecture (Workload Identity in Multi System Environments) from the Internet Engineering Task Force (IETF). It has an AI-agent applicability draft (October 2025), and the architecture document went to the Internet Engineering Steering Group (IESG) in July 2026. SPIFFE (the Secure Production Identity Framework for Everyone) runs in production for agents at Block, which operates SPIFFE, WIMSE, and OAuth (the open authorisation standard) together. OAuth-extension: Okta's Cross App Access (XAA, June 2025). It was adopted as the Enterprise Managed Authorization extension of the Model Context Protocol (MCP)MCP (Model Context Protocol). An open standard for connecting agents to tools and data, like a universal plug. An MCP gateway is the governed version of that plug., the open standard for connecting agents to tools. More than 25 independent software vendors (ISVs) had committed to it by June 2026. The floor is ID2: one first-class identity and access management (IAM) principal per agent. Sharing human credentials is the ID1 anti-pattern, and it remains common. Only 23 percent of organisations have a formal agent identity strategy (Cloud Security Alliance, CSA 2025). This layerLayer (the fourteen layers). One of fourteen parts of the enterprise estate that agents touch, from infrastructure (R01) to agent data engineering (R14). issues the identity. The integration fabric enforces it on every tool call.

Durable sessions

In short: Agent sessions are built to survive crashes and pauses and to pick up where they left off.

Agents are long-running processes that get interrupted. Their state must survive crashes and pauses. The distinction that matters is checkpointing versus durable executionDurable execution. Running long jobs so they survive a crash or restart and pick up where they left off.. Checkpointing (LangGraph 1.0, October 2025) saves graph state at step boundaries and leaves recovery to the application. Durable-execution engines (Temporal, Restate, DBOS, Inngest) record every step in an event log and replay it, so the process itself survives failure. The durable-execution camp states that critique of checkpointing directly (Diagrid [vendor]). Event-sourced sessions are the same idea built into the runtime (Google Agent Development Kit, ADK). AgentCore holds sessions for up to 8 hours and charges no processor (CPU) time while a session waits on I/O [vendor]. Cloudflare Durable Objects hibernate idle agents at no cost [vendor]. Both matter for a workload that is idle 30 to 70 percent of elapsed time. Snapshot and restore is the undo button after a bad action. It is also what makes replaying evalsEval (evaluation suite). A set of test tasks with known right answers, written by people who know the work, used to check an agent before and after any change. (test runs with known right answers) cheap. Session memory is wiped when the session ends (the microVM pattern). The market treats durable execution as agent infrastructure. Temporal raised a $300M Series D at a $5B valuation (February 2026) and integrated with the OpenAI software development kit (SDK) in September 2025.

Serving economics and classification routing

In short: Pay-per-use access beats running your own models unless volume is high and steady; data sensitivity decides where a model runs.

Self-hosting beats metered frontier APIsAPI. A defined way for software to talk to other software. Agents reach systems through governed APIs rather than through screens. (application programming interfaces) only above roughly 100 to 250 million tokensToken. The unit AI usage is measured and billed in, roughly three-quarters of a word. per month, sustained, per workload. It also needs utilisation of 60 percent or more, and an honest operations multiplier of 1.3 to 2 times in the cost model. At 10 percent utilisation, cost per token rises roughly tenfold. These figures come from four mutually consistent practitioner analyses (2025 to 2026), none peer-reviewed, and utilisation is the pivotal assumption. Open-weight models are models whose trained weights are published for anyone to run. Against budget APIs serving them at roughly $0.14 per million input tokens, break-even recedes to billions of tokens per month. So the cost path is open-weight models through metered APIs, not self-hosting. Open-weight models lag the frontier by about 4 months (Epoch AI, May 2026). OpenRouter telemetryTelemetry. The stream of measurements a system emits about itself, collected centrally. (June 2026) shows US closed models falling from about 70 percent to about 30 percent of token share in twelve months. Chinese open-weight models stand at about 61 percent of tokens. Enterprises meanwhile still run 81 percent of large language model (LLM) workloads on closed models (Menlo, December 2025, a venture capital source [VC]). Cold starts (the delay when a model server starts from nothing) have stopped being the objection. Graphics processing unit (GPU) memory snapshots cut cold starts about tenfold (Modal, 118 seconds down to 12 [vendor]; Cerebrium, 71 percent average reduction [vendor]). Cloud Run GPUs (generally available June 2025) start a GPU instance in under 5 seconds, with about 19 seconds to first token on gemma3:4b [vendor]. Also, vLLM ships a sleep mode. Scale-to-zero (shutting capacity down entirely when idle) is viable for bursty agent serving.

Routing is by classification, not by preference. Metered managed serving is the default. A sovereign region is used where residency rules and operator control demand it (AWS European Sovereign Cloud, generally available 15 January 2026, with Bedrock [vendor]). Air-gapped managed AI (disconnected from any outside network) is used where classification demands it. Gemini became generally available on Google Distributed Cloud air-gapped in August 2025, authorised to US Secret and Top Secret [vendor]. The GPU market in mid-2026: reserved H100 capacity bottomed at $1.70 per GPU-hour in October 2025 and rebounded about 40 percent to $2.35 by March 2026 (SemiAnalysis). Spot prices run $2.19 to $3.50. Blackwell-class chips are scarce and reserved through 2026 to 2027, amid a squeeze on high-bandwidth memory (HBM). Roughly 20 percent of the supply of DRAM (dynamic random-access memory) now goes to AI. The assumption that compute always gets cheaper is not safe for the newest silicon.

The incident record

In short: Every control on this page answers a real incident, not a hypothetical.

Replit's agent deleted a production database during an explicit code freeze (July 2025). The answer: environment separation and snapshots. A compromised Amazon Q extension for the integrated development environment (IDE) carried a data-wiping promptPrompt. The instructions and information given to the model for a task. A prompt is advice to the model, not a control on it. to roughly 950,000 installs (July 2025). The answer: supply-chain review and egress control. SANDBOXESCAPEBENCH (arXiv 2603.02277) showed frontier models exploiting container weaknesses when present. The answer: microVM-class isolation. EchoLeak (June 2025) is vulnerability record CVE-2025-32711 (Common Vulnerabilities and Exposures), with a severity of 9.3 on the Common Vulnerability Scoring System (CVSS). It was a zero-click exfiltration (data theft without any user action) from a chat-and-retrieve agent (Microsoft 365 Copilot), carried out entirely in natural language. The compute sandbox was irrelevant; egress control and output sanitisation were decisive. Chat-and-retrieve agents are exempt from compute sandboxes, not from controls. NIST (the US National Institute of Standards and Technology) and the UK AI Security Institute (UK-AISI) ran joint agent-hijack evaluations. Attack success reached 81 percent once the attacks were optimised. The answer: capability-set gating, re-checked on every grant.

Design decisions

• Self-hosted GPU estates vs metered APIs (CD-6): metered by default. The one correction: the cost path is open-weight models through metered APIs, not self-hosting. Self-hosting earns its place for air-gap and classification requirements, rarely on cost alone. Below roughly 100 to 250 million sustained tokens per month per workload, or below 60 percent utilisation, the numbers do not close. The Airbnb counter-example marks the boundary. Its public commitment to Qwen for cost and speed (October 2025) shows that cost-driven open-weight adoption is real at hyperscale consumer volume. And it runs through APIs.
• Kubernetes everywhere vs managed agent runtimes (CD-7): split by layer. Model serving is where Kubernetes standardised, and that matters only if you self-host models at all. Dynamic Resource Allocation became generally available in v1.34 (September 2025). The llm-d project was donated to the CNCF in March 2026. CNCF AI Conformance v1.0 arrived in November 2025. The stack is vLLM and KServe. Agent orchestration fits managed runtimes. Per-session isolation is native there, billing matches the idle-heavy profile, and Kubernetes needed a new project just to match the isolation primitive. Lock-in is symmetric. Managed runtimes are proprietary control planes, and the portability camp sells its own ecosystem. Over half of surveyed Kubernetes shops admit to one-off, hand-maintained ("snowflake") clusters (Spectro Cloud 2025 [vendor]).

Cross-cutting concerns

Evidence and limits

Incidents carry CVE identifiers and CVSS scores where they exist. EchoLeak is CVE-2025-32711 at CVSS 9.3. The Replit and Amazon Q incidents are dated but carry no CVE. Isolation, billing, cold-start, and sovereign-serving claims are vendor-published [vendor] (AWS, Microsoft, OpenAI, Anthropic, Cloudflare, Modal, Cerebrium, Daytona, Google). Anthropic's figure of 84 percent fewer permission prompts is self-reported. The figure of 81 percent of workloads on closed models comes from a venture capital firm (Menlo). The snowflake-cluster survey is vendor-published (Spectro Cloud). One refusal: no peer-reviewed unit economics for self-hosted serving exist. The crossover figures here are ranges from four mutually consistent practitioner analyses, and this guide publishes them only as ranges, with the utilisation caveat. The strongest independent sources at this layer are Epoch AI, NIST and UK-AISI, OWASP, CNCF, IETF, SemiAnalysis, and SANDBOXESCAPEBENCH. Re-verify: CNCF Agent Sandbox production readiness (fourth quarter of 2026); WIMSE progress at the IESG (from July 2026); GPU pricing quarterly (SemiAnalysis); OpenRouter token-share drift.

The research behind this page

• Infrastructure and compute findings
• Research brief
• Vendor landscape
• Sources