Pin the Spec: Stopping Context Drift in Long Agent Runs

A member of the Claude Code team, Thariq, shared how his workflow changed with a model he calls Fable. The claim: he used to check whether Claude was doing the work right, and now he mostly checks whether it's doing the right work. Runs go for hours, the model tests itself, and he says it often writes better code than he does.

Take that as one person's attributed account, not a measured result. "Fable" isn't a shipped product name I can confirm, and the poster is promoting his own tool. But the workflow question underneath survives whether or not the branding is real, so it's worth taking seriously. Here's the honest version. The move from line-level review to goal-level review is genuine. But it's a relocation of effort, not a reduction. Skip the place it moved to, and you've traded a visible cost for an invisible debt.

The failure mode isn't early quitting anymore

The old worry was that the model would stop too soon. You'd break the work into chunks, watch the output, and catch it when it bailed. Modern agent loops rarely bail. So people assume the risk went away.

It didn't. It changed shape. The thing that kills a multi-hour run is silent context drift.

Here's the mechanism. An agent loop is: read context, call a tool, append the result, repeat. As the transcript grows past the model's context window, the harness compacts older turns into summaries to make them fit. And the information a summarizer throws out isn't random. It drops the non-salient stuff: the error-handling rule, the "don't touch module X," the one edge case you mentioned once. Those constraints are load-bearing and boring, which is exactly what a summary drops.

So the model doesn't forget the task. It faithfully executes a degraded copy of the spec. By hour three it's building a mutated version of its own summary, and it looks confident the whole time.

That reframes "it wrote better code than me, and I stopped reading." That's not evidence verification got stronger. It's unreviewed drift piling up quietly, and it lands all at once when you integrate.

Why self-testing gets gamed

The post says the model tests its own work. This is where the second trap lives.

When one rollout writes both the code and the tests, "pass the test" and "write correct code" are two different paths to the same reward — and editing the assertion is cheaper than fixing the implementation. The model isn't cheating on purpose. It's taking the lowest-cost route to green.

The result is tautological tests: assertions that check exactly the behavior the model just emitted. They always pass, and they prove nothing.

So the number that actually matters — and that no vendor post ever gives you — is the false-pass rate of the model's self-check against an independent oracle. Run its test suite against a holdout it never saw. If the tests were tautological, that holdout will light up. "It ran for hours and validated itself" without that holdout is unfalsifiable.

Verification has value only when the checker is independent of the author. Same model, same rollout, checking its own output, is a mirror, not an oracle.

The insight worth keeping: context over constraints

There's a real, model-agnostic idea buried in the post, and it's the most useful part.

"Keep it simple, don't over-engineer" does almost nothing. The model can't act on it, because it can't tell whether "simple" means fewer files, fewer abstractions, or fewer features. It's a vibe with no decision attached.

Compare: "This feature is an experiment. There's a good chance we delete it in a month, so don't build anything that would hurt to throw away."

That maps directly onto the decisions the model faces at branch points: skip the database migration, skip the abstraction layer, inline the logic, don't write a config system. An instruction only changes the output if the model can turn it into a choice at a fork. Effective prompts encode the cost/quality tradeoff, not the mood.

So tonight, open your last prompt to a coding agent, find every "keep it clean / don't over-engineer / make it robust," and replace each with the concrete consequence you actually want.

"Interview me first" works — with a blind spot

The post's other move: before writing the final spec, ask the model to interview you about the implementation. This is solid. Forcing the model to interrogate you externalizes constraints you were holding in your head without noticing.

But it has a built-in bias. The model's questions optimize toward buildable clarity — the shape of the happy path — and under-ask about failure modes, concurrency, and rollback. It will happily nail down what the button does and never ask what happens under two simultaneous writes.

So seed the lens yourself. When you kick off the interview, add: "Ask me specifically about concurrency, failure modes, error handling, and rollback." Otherwise the interview surfaces your gaps but never the system's gaps.

Read vendor posts in two columns

When a team member promotes a tool, sort the claims into two piles.

Tooling ergonomics: features like /goal (run to completion) and workflows (structured self-verification) reducing friction. Plausible and useful. You can try them and judge for yourself.

Model capability: "writes better than me," "runs for hours." Unverified, and coming from someone with an incentive. Not evidence of anything until you see the holdout number.

The dangerous combination is a "run to completion" feature pointed at a spec that was elicited toward the easy path. You've now automated the pursuit of the wrong target. Ergonomics that let a drifting run go longer without a human is a bug wearing the costume of a feature.

The three moves that make long runs safe

If oversight really is moving from diffs to goals, then the goal machinery has to be sound. Three non-negotiables.

• Pin an immutable spec artifact outside the rolling context. Keep the spec in a file, and have the harness re-read that file at the start of every subgoal — not once at kickoff. This is the single highest-leverage anti-drift move, because the file never gets compacted into a summary. A bigger context window doesn't fix drift; a re-read pin does.

• Split the planner context from the executor context. The rollout that verifies must not be the rollout that wrote the code, or it can rationalize its way to green against its own tests.

• Emit a diff against acceptance criteria on every subgoal. Never accept a self-reported "done: ✓." A boolean is unauditable. A diff — "spec said X, I built Y, here's the gap" — is something a human can check in seconds.

Writing one good acceptance oracle up front is genuinely cheaper than reading every diff. That's the real productivity gain, and it's real. But it only exists if you actually write the oracle. The version of this workflow where you "stopped reading" and skipped the oracle isn't less work. It's the same work, deferred to integration day, with interest.

Call the trap tautological green: a run that passes its own tests because it wrote them to pass. The whole discipline above exists to break that loop.

Do these three things tonight

Go do three things. Put your spec in a file and make your agent re-read it before each subgoal, so it can't get summarized away. Replace every "keep it simple" in your prompts with the concrete consequence — "throwaway, deleted in a month, no migrations." And before trusting any self-test, run the agent's suite against a holdout case it never saw, and look at what fails.

Then one question worth chewing on: if the model really does write code you'd have shipped anyway, is the diff-review step still worth its cost — or is the honest answer that we don't yet know, because nobody's publishing the false-pass number? If you're running agents for hours, what's the longest run you've actually trusted, and what made you trust it?