Nearly a decade ago, we saw infrastructure responsibilities moving out of microservices and into Kubernetes, service meshes, and proxies. Application code stayed focused on business logic while Kubernetes, service meshes, and proxies took on more of the operational concerns.
Kubernetes supplied configuration and secrets and managed application rollouts through Deployments. Service meshes handled traffic management, certificate rotation, and traffic observability through metrics, access logs, and traces. Teams could configure these capabilities outside the application process.
I see the same separation emerging for AI agents. An agent calls models and tools, but increasingly that logic lives in a reusable harness. The harness can execute code in a sandbox, load skills and plugins from files, and use configuration to connect to MCP servers. Its working files and session state may need to survive between turns. These agents need infrastructure that runs and preserves their environments and controls the data flowing in and out.
Two areas where this shift is already emerging are lifecycle management and networking.
Agent lifecycle moves into infrastructure
An agent can spend a long time waiting for a model, a tool, or a person. If its environment stays running throughout that wait, it can keep memory and compute capacity allocated while doing no useful work. LangChain’s Deep Agents documentation leaves applications responsible for sandbox scope, reuse, and cleanup.
Agent Substrate handles sandbox creation and isolation, and provides suspension and restoration. When the harness or orchestrator decides an agent can safely pause, Substrate can snapshot its memory and filesystem, release its execution resources, and restore it onto an available worker when needed.
The harness or its orchestrator decides when suspension is safe. Framework session formats and conversation checkpoints remain useful; the runtime preserves the environment around them.
NVIDIA OpenShell and Kubernetes SIGs’ Agent Sandbox also put sandbox management outside the harness, with different priorities. Solo’s comparison of OpenShell and Agent Substrate examines isolation, resource efficiency, and state management.
Agent networking moves into gateways
Once a harness can run arbitrary code and call external tools, its access controls need to hold even when the model makes an unexpected choice. We need controls over the data and instructions entering the harness, the tools it can see and call, and the destinations it can send data to.
An agent’s LLM calls can go through the gateway too. A common API lets the harness use one client for supported providers while the gateway handles their request formats and injects credentials. That can reduce provider SDK maintenance inside each framework. The same gateway can route model requests, apply budgets and token limits, and observe LLM traffic and token usage.
A gateway can also authenticate callers, enforce access policies, and attach backend credentials outside the sandbox. With MCP virtualization, it can present tools from several servers through one endpoint. Configured guardrails can inspect prompts and, when explicitly enabled, tool inputs and outputs as traffic passes through.
Solo Enterprise for agentgateway supports Tool Search and Code Mode: agents can fetch tool definitions as needed and submit code that chains tool calls inside a gateway sandbox. Each harness can use that shared interface instead of implementing its own tool catalog and execution service.
The harness still chooses which tools to use and what work to perform. Task specific approvals and checks remain part of that work. Harness developers can then focus on getting the best out of the model and tools: planning the work, managing context, and verifying results against the goal.
I’ve used Agent Substrate and agentgateway as examples here, but other runtimes and gateways provide similar capabilities. The specific technologies matter less than the architectural shift: moving common runtime and networking responsibilities out of agent harnesses and into shared infrastructure.
👋 Found this useful? Like and restack it for other software engineers.