The $100K Run-Rate That Became a "$100K Per Engineer" Headline

The claim someone actually made

Let's start with what was really said, because the headline mangled it.

On the podcast, Bavor pegged a top engineer's token run-rate at north of $100,000/year. The host reached for scale and compared it to a company spending millions of dollars a year on Anthropic. And here's the part that matters: this isn't the median engineer, and it isn't a per-seat rule — it's a run-rate for a top engineer, and a claim that this figure is a real and rising fraction of what you pay that person.

That's the real claim. Not "$100,000 per engineer, everyone." A frontier run-rate, and a comp-fraction trajectory. Everything interesting lives in the gap between "the average seat" and "the top of the distribution," so let's build up to what actually moves it.

The two numbers on the table — keep them straight

Before any math of our own, pin the two figures the transcript actually contains, because they're easy to blur together.

• The host framed current spend as roughly 3.8% of that engineer's comp — today's number, as stated.
• Bavor said he'd bet on something closer to 20% — his forward view of where it's heading.

So the debate on the podcast is 3.8% now versus ~20% later. Those are source-attributed numbers. Everything we compute below — the 35–40% anchoring, the 3–4% ordinary case — is our own back-of-envelope arithmetic against a hypothetical comp figure, not something anyone on the show said. Don't merge them. When we say "35–40%," that's us stress-testing the $100k against a $250–300k salary; it is not Bavor's 20% and not the host's 3.8%.

Why this is a capital-allocation question, not a usage stat

Here's the reframe that makes the whole thing click.

A year ago, tokens were a line item in a SaaS bill — someone in finance glanced at it and moved on. What Bavor's framing does is drag tokens into the same conversation as salary.

Because once token spend is a percentage of comp, a CFO stops asking "what's our API bill?" and starts asking "for this engineer, how much total capital do we deploy — headcount plus benefits plus the compute they command?" Suddenly OpEx, headcount, and tokens get allocated in one breath, per person. That's not an accounting curiosity. That's the difference between a token budget being a rounding error and a token budget being a line you defend in a comp review.

Hold that lens. The rest of this is really about which direction that percentage travels — 3.8% toward 20%, or not — and how much of the travel is real versus wasted.

The arithmetic nobody did

The hook promised arithmetic, so here it is — and again, this is our math, separate from the 3.8%/20% figures above.

Take a senior engineer in a top market — call total comp roughly $250–300k, which is a plausible anchor. A $100,000/year token run-rate against that comp is on the order of 35–40% of what you pay the person. That's an enormous share — higher than either the host's 3.8% or Bavor's 20% — which tells you the $100k number and the percentage figures are almost certainly describing different engineers against different comp bases. That mismatch is the point: the same headline gets used for very different people.

Now hold it next to a more ordinary case. If a normal-heavy user's run-rate is, say, $10k/year, that's roughly 3–4% of the same $250–300k comp. Same job title, same market — and a ~10x spread in token spend between the ordinary heavy user and the frontier one.

So the honest read: $100k isn't the median engineer. It's a frontier, top-of-the-distribution edge case — one person commanding an unusual amount of autonomous compute. The headline took that tail and pasted it onto every face in the org. Keep the distinction, because that's the whole game: are you pricing the average seat, or the frontier one?

The category error hiding in "per engineer"

There's a second reason $100k describes an edge case, not everyone.

A human caps out at maybe 50–100 agent turns a day. You physically cannot type your way to $8k/month. So high per-engineer numbers are never per seat — they're per engineer-sponsored compute: the fleet of autonomous agents one person points at problems while they sleep.

Consistent with that, Bavor noted Sierra isn't yet at token-budget discipline and its usage is comparatively modest — that's what he actually said, and it's a point about where Sierra is on that curve, not a general claim that nobody anywhere runs large agent fleets. The high numbers belong to whoever commands the most autonomous trajectories, not whoever is most "AI-native." That's the frontier tail, and averaging it across every engineer is how a $10k reality becomes a fake $100k rule.

Where the tokens actually go

So what does even a $10k run-rate buy, let alone a $100k one? Here's the mechanism most people miss.

In a tool-call loop, the model re-sends the entire working context — the repo files it's looking at, every prior turn, every tool output — on every single turn. Not once. Every turn.

A single agent session runs 20–100 turns and can pile up 2 to 5 million input tokens. The text you actually typed is a rounding error. This is why "write shorter prompts to save money" is noise — you're not paying for what the human writes, you're paying for the context re-transmitted twenty times over.

Tonight: open your provider's usage dashboard and look at the input-to-output token ratio on a coding session. If input is 20–50x output, that re-transmission engine is the thing worth attacking.

The master lever: prompt caching

Because context gets re-sent, providers let you cache a stable prefix.

Here are Anthropic's actual numbers as of mid-2025 (these prices change, so re-check before you build on them). A cache read costs 0.1x the base input-token price — 10% of full price. A cache write costs more than base up front: 1.25x for the 5-minute TTL, 2x for the 1-hour TTL. So you pay a small premium once to write the cache, then pay a tenth of the price on every read after that. That's Anthropic-specific — other providers price and expire caches differently, so check yours before you assume the ratio.

If your agent keeps the first chunk of context — system prompt plus repo snapshot — byte-for-byte identical across turns, you eat that 1.25–2x write once and pay 0.1x forever after. On a setup like that, it's a large effective cut on the biggest term in your bill.

The catch: caches bust the moment you reorder or edit early context. A timestamp near the top. A shuffled tool list. A file inserted in the middle instead of appended at the end. Any of those invalidates the cached prefix, and you're back to paying full write price again — the premium, not the discount.

Many orgs run a low cache hit rate and — this is the real problem — never measure it. If you're leaving reads uncached, you're paying full input price on the biggest term in your bill instead of 10% of it. That gap is a big, invisible slice of the total. It's not a governance failure. It's an unmeasured mechanical bug.

Tonight: find the cache-hit-rate metric in your proxy or provider logs. If it looks low, check whether your system prompt has anything dynamic — a date, a random ID, a reordered list — inside the cached prefix. Move it to the end.

Why commodity tokens fall but the frontier doesn't

Now the first force pushing the percentage down.

The cost of a fixed capability drops fast — GPT-4-class quality has fallen very roughly from tens of dollars per million tokens toward low-single-digits over something like a year and a half, call it a few-x per year. It happens through three concrete mechanisms:

• Continuous batching — many requests share one forward pass, so marginal cost per token drops toward raw compute instead of amortized model overhead.
• Mixture-of-experts routing — a "trillion-parameter" model activates only a fraction of its parameters per token (in many MoE designs, on the order of tens of billions). Quality tracks total params; cost tracks active params. That's how a model gets smarter and cheaper at once.
• Speculative decoding — a small draft model guesses the next tokens, the big model verifies them in one pass. When acceptance is high, you get meaningful throughput gains at roughly the same quality.

So what for your bill: a fixed capability — last year's frontier — commoditizes and collapses in price. The frontier stays expensive because it's a moving target you keep chasing. So "unbounded demand for intelligence" does not mean "unbounded spend." Your bill depends on how much of your workload genuinely needs the moving frontier versus how much can be routed down to a small model. Retrieval, routing, formatting, lint fixes — a large majority of an agent's sub-steps run fine on commodity models at a tiny fraction of frontier cost.

Tonight: pick one high-volume, low-stakes step in your agent flow (classification, formatting, a routing decision) and swap it to a small/cheap model. Diff the cost per run.

The other downward force: open weights and local inference

Falling API prices aren't the only thing pushing back on a giant frontier token budget — and this one you control directly.

Open-weight models (the Llama, Qwen, Mistral, and DeepSeek families) let you take spend off the metered API entirely for workloads that don't need the closed frontier. Three concrete levers:

• Distillation — train a small model on the outputs of a large one, so a 7–8B model handles your narrow, repetitive sub-steps (classification, extraction, routing) at a fraction of the tokens-per-dollar of a frontier call.
• Local hardware — run those distilled or open models on a workstation GPU or a shared inference box, where the marginal cost per token is your electricity and amortized hardware, not a per-token API line.
• On-prem / self-hosted inference — stand up an open model behind a vLLM or SGLang server your agents hit internally, moving high-volume, low-stakes traffic off the API meter.

The point isn't that open models replace the frontier — they don't, for the hard steps. It's that they redirect a large chunk of volume away from metered frontier tokens, so a rising comp fraction is fighting both falling API prices and the option to simply not pay for many tokens on an API at all.

Tonight: identify one repetitive sub-step you're currently sending to a frontier API and prototype it against an open model (self-hosted or via a cheap open-weight endpoint). Compare cost and pass-rate before you commit.

The real demand driver isn't intelligence — it's correctness

Here's the counterweight, and it's where the "percentage keeps climbing toward 20%" side has a real point.

Production doesn't want raw intelligence. It wants correctness, and correctness is bought with redundancy.

A task that costs a couple of cents to attempt once costs meaningfully more to make reliable — because to trust the output you run best-of-many, add a verifier pass, grade with an LLM-as-judge, take a majority vote. This is Jevons paradox in action: cheaper tokens don't get pocketed as savings, they fund more sampling. Falling unit prices don't shrink the bill; they let you buy reliability you couldn't previously afford. That's the engine that pushes the comp fraction up.

But — and this is the ceiling — it saturates. Pass@k flattens hard. Past roughly a handful of samples on most coding tasks, you pay meaningfully more for a tiny bump in pass-rate. So per-task redundancy has a hard cap. Sampling one task forever does not get you to six figures.

So where does the growth actually live? Trajectory count.

If per-task sampling saturates, where does the high burn come from? Eval harnesses.

Every time you change a prompt or swap a model, a proper CI-style eval re-runs thousands of agent trajectories to check nothing regressed. Not many samples of one task — thousands of tasks times several samples, on every merge, running around the clock. That's the plausible path to the top of the range. It's not a human commanding one agent; it's a continuous integration pipeline for agent behavior, grinding tokens while everyone sleeps.

Which reframes Bavor's number: per-engineer spend mostly reflects how many autonomous trajectories that engineer commands and how disciplined the org is about caching, routing, and gating evals — not how advanced anyone is. That's the difference between landing near the host's 3.8% and drifting toward Bavor's 20%.

Why a big, invisible fraction is untraceable — by construction

Here's the uncomfortable part.

Without per-request tagging — team, feature, environment — you cannot distinguish a legitimate best-of-many reliability run from an agent looping on its own error with no kill-switch. In an untagged ledger they look identical.

The usual suspects, all invisible without tags:

• Eval suites re-running full trajectory sets on trivial diffs because nobody gated on env=staging.
• Agents looping on their own errors with no turn limit.
• Verbose system prompts re-sent uncached every call (the cache bug from earlier, wearing a different hat).
• Abandoned and background runs nobody owns.

If your org climbs toward the high end of that comp fraction, expect a large, often-invisible share of it to be avoidable — by construction, not by accident. The fix isn't spending less. It's making spend legible.

Tonight: add three tags to every LLM request through your proxy — team, feature, env — and set a hard turn cap on agent loops. You can't cut what you can't see.

The honest bottom line

Don't swing to the cynical opposite either — "so spend actually falls." It doesn't cleanly.

The truthful answer: Bavor's $100k+ run-rate for a top engineer is real, his bet that the comp fraction climbs toward ~20% (from the host's stated 3.8%) is plausible, and the "$100k per engineer" headline is a frontier edge case — top of the distribution, per engineer-sponsored compute, not every seat. Whether your engineers drift up that curve is set by agent count and org discipline, not destiny.

And the downward forces are real and stackable: commodity prices fall a few-x a year, and open weights, distillation, and on-prem inference let you take whole categories of volume off the metered frontier entirely. The unbounded thing was never intelligence — it's correctness, and correctness has both a Jevons engine pushing the percentage up and a saturation curve capping it. Where you land inside that range is a set of dials, not a fate.

Do these three things tonight

• Measure your cache hit rate. If it looks low, hunt for anything dynamic inside your cached prefix and move it to the end — remember you eat a 1.25–2x write once (Anthropic pricing, mid-2025) and then pay 0.1x on every read, so a stable prefix is the largest lever on your single biggest cost.
• Move one cheap sub-step off the frontier. Pick a formatting or routing call and either swap in a small API model or prototype it on an open-weight model you self-host — then diff the cost per run.
• Tag every request with team/feature/env and cap agent loop turns — so you can finally tell reliability spend from a runaway loop.

And one question worth arguing about in your next budget review — this one's for founders, CFOs, and engineering leaders: if a frontier engineer's token line really is running from 3.8% toward a fifth of comp, are you prepared to allocate it like headcount — a defended number per person, reviewed alongside salary? Or is it still buried in an API bill nobody owns? Whoever at your company can actually answer that has the most interesting number in the building.