Should a Small Team Still Write Its Own Agent Framework from Scratch?

Take apart an agent application built by a small team and the genuinely differentiated part usually turns out to be a very thin layer. The basic capabilities—the task loop, tool calling, context assembly—now have multiple off-the-shelf sources: vendors’ Agent APIs, continuously iterated open-source harnesses, CLIs that can be embedded into your own product. How much of writing a framework from scratch is still worth doing depends on what each of these alternatives has taken over.

The Loop Is Easy; Long-Term Use Is Hard

A working agent loop is not complicated: hand the task to the model, parse which tools it wants to call, execute, stitch the results back into the context, and run another round. What really separates the players is the detail accumulated through long-term use. Agents look simple, and extended use reveals how hard they are—the organization of sessions and projects, cache hit rates, optimizations that work in concert with the model, all of it hidden behind “it runs.”

Codex is a specimen of this kind of accumulated detail: it provides dedicated tools that can pull out all of its own Projects and Sessions, show progress and status, and even locate a reconnection problem that appeared somewhere inside a long conversation. These capabilities come from keeping complete records for model training from the moment of architectural design, plus continuous iteration over a long period. Even if a small team writes a loop of exactly the same shape, it can hardly invest the same level of maintenance across all of these details.

And the complexity keeps growing. A self-built harness usually starts from the simplest loop, then keeps acquiring features, and once it grows complicated enough it needs to be encapsulated—which is precisely one of the vendors’ motives for turning it into a managed service.

The Shell Involves More Work Than You Think

Once execution is handed to a mature framework, what an application team has left is often summarized as “building a shell.” That summary understates the workload of the shell, and one case of actual development makes the point: CodePilot originally adapted to Claude Code alone; back then the Agent SDK did everything else well and the team only had to mind the UI. But as the various frameworks waxed and waned, users switched from one to another from day to day, and the shell was forced to adapt to every backend: building runtime routing to dispatch user input to different CLIs and frameworks; handling each vendor’s incompatible API formats, return data, and billing schemes; and on top of that the product’s own distinctive features and legacy debt—the adaptation work kept snowballing.

Deepening vendor lock-in makes the problem worse. Agents are binding ever more tightly to their own models, the various coding plans are mutually incompatible, and adaptation costs keep climbing. The “adapt to everything” shell grows more and more painful to maintain, yet nobody dares adapt to only one vendor—the one being mocked today may turn the tables a few days later on the strength of quotas or capability.

Three Options, Three Scopes of Responsibility

There are at least three possible routes, each with a different scope of responsibility.

The first is the managed service. An Agent API or an official harness packages the model, the tools, and the execution environment into a closed loop, and the team simply calls the service. The price is deep binding: the model and the agent are co-optimized as a matched pair, and people who support the official pairing often cite the cache-rate example—the same model gets better cache hits inside the official host, though the exact numbers vary depending on who is telling it. The advantage of the pairing also extends to integration coverage: in Chinese information retrieval, the plugin for DeepSeek’s official pairing can reach content that external tools cannot, such as the full text of WeChat official accounts.

The second is the SDK and the CLI. You embed the execution engine into your own product; the model and the base loop are maintained upstream, while the runtime shape, the interface, and the data remain under your control. This is the middle path, but following upstream is itself a burden: the DeepSeek Harness ships a dozen or more versions in a single day, and when the layer underneath a plugin moves, everything on top breaks; GUI adaptations die the moment the host interface shifts, which is why the community consensus has become to build against more stable exposure surfaces like MCP or the CLI; and if you want stability, you do what Linux users did in the early days—wait for someone else to package a distribution.

The third is building from scratch. It has not become pointless—when the execution environment is unusual, the security boundary is strict, or the product is itself a piece of framework research, building your own is still reasonable. But for most user-facing applications, the basic execution capabilities already have cheaper sources, and the criterion for building in-house has shifted from “no one else has it” to “does this layer really differentiate the product.”

After Handing It to Infrastructure

A further idea is to treat the entire execution layer as infrastructure: in the past, building an internet service meant renting a host from a cloud provider to deploy your program; in the future, what you rent might be an “agent host” or an agent environment that drives a large share of the business. The analogy is still a vision rather than reality, but the directional signals have appeared—agents are increasingly treated as a technology rather than a product, users no longer care which model is behind the scenes, and services expose only what is distinctive about themselves.

For application teams, what actually remains is, paradoxically, clearer than before: product definition, interface, vertical data and context, evaluation and delivery. Looking back, the investment many teams made over the past year in building their own harnesses resembles tuition paid—tuition that bought the ability to answer a question honestly: outsource the execution capabilities that can be outsourced, and concentrate resources on the layer that no one else can replace. As for which layer is irreplaceable for you, every team has to answer that with its own scenarios.