If the Model Eats the Harness, What Do Framework Startups Have Left?

Start with the source, because getting the attribution right is the whole point of what follows. The thesis "the model eats the harness" comes from Logan Kilpatrick, speaking on the Sequoia Capital podcast. The claim is a good one, and it is half right. It says that as frontier models get better at acting on their own, the external engineering people built to keep them on the rails — the scaffolding, the framework code — gets absorbed into the model and its API. Build a company on that scaffolding and you're building on ground the model is about to reclaim. The same source is where the 80/20 line comes from: roughly 80% of a harness is the same across use cases, and about 20% is use-case-specific tuning. Neither of those is a LensUp claim. What comes next — the two-layer split, the compound-failure math, the follow-the-tokens argument — is LensUp analysis extending Kilpatrick's thesis, and we'll flag it as ours where it starts.

Here's where we start. The trouble is the word "harness." It sounds like one thing. In our reading it is two things, they live in different parts of the codebase, and the thesis only touches one of them.

Layer A is capability-bound scaffolding. When a model calls a tool, it emits the arguments as text — usually JSON. The harness validates that JSON against a schema. When the model produces something malformed — a missing field, a trailing comma, a hallucinated parameter — the harness catches it and either re-prompts, repairs the output, or rejects it and tries again. This is the parse-and-repair loop, and it exists entirely because the model isn't reliable enough at structured output. So as the model climbs toward, say, 99% clean structured output, the repair loop fires less and less, until it's near-dead code. Native function-calling and structured-output features in the base API absorb it directly. This layer, the thesis has exactly right. The model eats it.

Now Layer B. This is reliability and autonomy scaffolding, and here — this is LensUp's read — the thesis inverts.

Take checkpoint and rollback. In a long run, the harness snapshots state between steps, deterministically, so that one bad action — the agent-equivalent of an rm -rf on the wrong directory — doesn't poison the other thirty-nine steps. Here's the part that matters: a better model doesn't remove this. A better model is trusted with a longer horizon. Longer horizon means more steps, more state, more places for state to corrupt, more need to roll back to a known-good point. The capability improvement doesn't dissolve the problem. It moves the problem here and makes it bigger.

Same shape everywhere in Layer B. Context compaction: a long run overflows the context window, so the harness summarizes and truncates tool outputs to survive. Budget-aware routing: send the trivial step to a cheap Flash-tier model, the hard step to the frontier model, so you don't burn twelve dollars on something a nickel would have covered. Permission gating and subagent orchestration: who can call what, and how the big task decomposes into sub-loops. None of this shrinks when the model improves. The tell sits in plain sight — Anthropic's own agent SDK still ships context compaction and subagents while Claude keeps getting better. That's not a crutch they forgot to remove. That's a load-bearing wall.

Here is the arithmetic that makes the whole thing turn — this is LensUp's, not the source's — and it's worth doing slowly.

Take a step that succeeds 90% of the time. Chain it twenty deep — a real agent run. End-to-end success is 0.9^20. That's about 12%. Now hand the agent a better model and push per-step reliability to 99%. Twenty steps deep, 0.99^20 is still only about 82%. The model improved enormously. The end-to-end run went from a coin-flip disaster to still-fails-one-in-five. Because the failure surface of a multi-step run grows combinatorially, model improvement raises the floor but multiplies the places a run can die. So the demand for tracing, for regression evals, for the tooling that tells you which step failed and whether your last change made it worse — that demand goes up as the model gets better, not down. The parsing layer is pro-cyclical: model improves, it gets eaten. The eval layer is counter-cyclical: model improves, you need it more. Two numbers, one conclusion. That's the hinge.

So why does a lab absorb some of this and not the rest? This is LensUp's extension of the thesis: follow the tokens. A lab absorbs, into its API, exactly the features that either pull more tokens onto its endpoint or deepen lock-in — native tool use, hosted code execution, server-side memory. It declines, deliberately, your last-mile business logic. Owning your business logic means owning your support burden, which it doesn't want. And cross-provider orchestration — the abstraction that lets you swap Gemini for Claude for GPT per step — is one thing a single lab has no incentive to build for you, because the whole point of it is to route tokens away from any one endpoint, including its own. Note who does build it: third-party tools like LiteLLM and OpenRouter sell exactly this cross-provider routing layer. So a lab's absorption map isn't drawn by what's technically hard. It's drawn by revenue design.

Which brings us back to Kilpatrick's 20% quote, read forward instead of backward. He said about 80% of a harness is the same across use cases; the remaining 20% is use-case-specific tuning. Everyone hears: 80% gets commoditized, frameworks are dead. LensUp reads it the other direction. The 80% that's shared was always the low-margin, easily-eaten part — the parsing, the generic loop. The 20% that's specific to coding, to a consumer agent, to your domain — that's not the ceiling on framework value. It's the floor. And it grows as tasks get more domain-specific. The eaten part was the cheap part. What's left is where switching cost lives.

So here's the thing the thesis actually gets wrong, in our view: it names the wrong killer.

It isn't the smarter model that ends the orchestration business. Watch what's already happened at real companies: teams hit debugging pain, a latency tax from unnecessary abstraction, and version-churn hell, and they rip out the framework and rebuild a couple hundred lines of loop plus direct API calls. For those teams, the model already ate the orchestration layer — not because it got smart, but because a competent team plus a decent API made the abstraction more expensive than it was worth. The 200-line in-house loop wins on debuggability. Abstractions are the first thing commoditized.

The real threat vector is different, and sharper: a model provider shipping a good-enough hosted agent runtime for free. An IDE, a web agent, a CLI, an SDK, and a managed agent through the base API, bundled at zero marginal price. That cannibalizes the orchestration layer for free — and it does something a smarter model can't. But notice the one thing even the free bundle can't touch: the cross-provider eval and observability layer. It can't help you swap its own model out for a competitor's, and it can't own your regression corpus without asking you to bet your whole stack on one vendor. The durable asset was never the DAG and never even the router — routers get commoditized too. It's the boring data flywheel: your trace history, your regression suite, your version-pinned prompts. Once your safety net lives in a tool, ripping the tool out means losing the safety net. That's stickier than swap-convenience and stickier than any single-capability trick — which, by LensUp's editorial estimate, tend to hold their edge for a year or two before absorption, no longer.

And the honest part, the part no framework deck admits: the survival lane exists, but it's narrow, and it's aimed at a shrinking population — teams too junior to build their own loop. As direct API use gets easier, that population contracts from the bottom. The company that survives isn't selling the abstraction. It's selling the eval corpus and the neutrality, to buyers who've outgrown the abstraction and still don't want to build their own data flywheel.

So if you're building or betting on one of these startups, do a two-layer audit tonight. Take your product's feature list and sort every line into Layer A or Layer B. One: Layer A is anything capability-bound — parsing, schema repair, generic tool-calling loop, "the model can't do X yet." Mark those as time-limited — our editorial estimate is a year or two before absorption, not a hard clock — and stop counting them as moat. Two: Layer B is anything that scales with autonomy — checkpoint/rollback, context compaction, and above all tracing, regression evals, human review queues. Move your revenue there, because that demand rises as models improve. Three: open your own deck and find where you claim neutrality or cross-provider swap — that's your entry ticket, but underline it in a different color than the eval layer, because neutrality gets you in the door and the eval corpus is the only thing that keeps you there.

One open question worth arguing about: if the free bundled runtime is the real killer of orchestration, is provider-neutral routing a durable moat — or just a thinner-margin thing that gets eaten next, the way LiteLLM-style routing already did the hard part for free? Say which layer your stack actually lives in.

Here's where LensUp lands. Kilpatrick's thesis is directionally right and points at the wrong casualty. The smarter model eats Layer A — the parsing, the generic loop, the cheap 80%. That was always going to go. What it does not eat is the counter-cyclical work: the tracing, the regression corpus, the neutrality that survives any single provider's roadmap. So don't defend the harness. Defend the data flywheel around it. The 0.9^20-versus-0.99^20 gap doesn't close as models improve; it moves, and it moves into exactly the layer the labs have no revenue reason to absorb. Build there, or you're building on ground the model is coming back for.