Why SemiAnalysis Puts Engineers and Quants in One Room

The short answer

A team of engineers and a team of quants each produce confident, wrong analysis in a predictable way. Engineers over-index on the technically superior thing and treat spec sheets as reality. Quants anchor on TAM, sell-side margins, and smooth capex curves, and treat a FLOP as a FLOP.

Put them in the same room and force them to reconcile one number, and something specific happens: their errors point in opposite directions. The disagreement surfaces the assumption that would otherwise ship unexamined. That's the edge. Not the volume of the fight. The orthogonality of the blind spots.

But there's a catch, and it's the whole game. Friction alone regresses to seniority and loudness. The thing that turns the fight into signal isn't the fight. It's grading it. More on that below.

Why one discipline alone always ships a hidden assumption

Start with a naive question. Why can't a good quant just build the cost model correctly on their own?

Because a cost model is a chain of assumptions, and some links are invisible from inside finance. A quant builds cost-per-token off nameplate FLOPS and a clean depreciation curve. That model is fiction the moment it touches a physical site — but the quant has no way to know which link is fake, because the fake link lives in a domain they don't inhabit.

Same for the engineer. The engineer knows the chip is technically dominant. What they systematically miss is that "dominant" dies in the DCF if the ROIC doesn't clear financing at realized utilization.

Neither is being dumb. Each is missing a specific thing that is only visible from the other seat. That's why hiring two quants who disagree gives you noise, and pairing a quant with an engineer gives you signal. Two people wrong the same way argue about volume. Two disciplines wrong differently argue about assumptions.

The three-stage derating chain

The public description of how these teams work is short: technologists and former hedge-fund people fight it out organically. Dylan Patel of SemiAnalysis has described the setup in roughly those terms in a Sequoia interview. Fair enough. But look closely at what a good version of that fight has to do, and it's a chain of three gates — and no single discipline can run all three.

Gate 1 — Is the comparison even valid? (architect) Before you compute cost-per-token, prove the workloads are comparable. LLM inference is typically memory-bandwidth-bound, not FLOP-bound — the binding constraint is HBM bandwidth and capacity, not compute. So a "cheaper $/FLOP" chip that can't hold the KV-cache or feed the tensor cores is irrelevant to the workload it's benchmarked against. This gate kills apples-to-oranges cost math before it starts.

Gate 2 — Does the paper spec survive deployment? (physics) A 100MW campus can't just absorb next-gen racks. Legacy air-cooled racks were typically designed for 5–15 kW per rack, with 20–30 kW marking the high-density threshold where standard air cooling starts to get impractical. A GB200 NVL72 rack is a different animal, discussed in the industry as a liquid-cooled, high-density system in the six-figure-watt range — well past what any air-cooled hall was designed to remove. You need CDUs, direct-to-chip loops, or rear-door heat exchangers — a retrofit, not a plug-in. And on large-scale public training runs, realized MFU (Model FLOPs Utilization) is usually reported in the 35–45% range, not the ~90% a spreadsheet assumes. (That range is a cluster-scale training figure, not a universal constant across inference or every architecture.) This gate derates nameplate to deliverable.

Gate 3 — Does the derated number clear ROIC? (finance) Whether you depreciate on a 3-year or 5-year schedule can move cost/token materially — and once you layer in financing cost and realized utilization, it can be the difference between a call that clears its cost of capital and one that doesn't. Architectural superiority is necessary, not sufficient. This gate kills "cool but uneconomic."

Run any AI-infra claim through all three and watch how many die at a gate the originating discipline couldn't see.

The step everyone skips

Now the uncomfortable part. Everything above describes a productive argument. But a productive-feeling argument has a known failure mode: it converges on whoever is senior and confident, not whoever is right.

"They fight it out organically" is what a desk says right before it discovers its best calls came from three loud people — and its worst came from the same three.

The fix isn't more debate. It's instrumentation. Make each side state a falsifiable, priced prediction — not "this matters," but "this chip does X tokens/sec at Y cost under this workload, and here's when." Then grade the calls against reality and re-weight who actually gets it right.

Follow that one step further than is comfortable, and you get the real thesis: the fight surfaces the assumptions; the grading is what makes them converge instead of just get loud. An adversarial team without a hit-rate ledger isn't research. It's a debate club that feels productive. (To be clear, the grading step is the argument here, not a description of any one firm's internal process.)

The number to demand at each gate

You've got the three gates. Here's the part that's easy to skip and shouldn't be: which specific number to demand at each one, so you can check a claim without trusting anyone's authority.

At Gate 1, ask for the denominator. Reject $/FLOP; demand $/effective-token under real utilization. KV-cache footprint, MoE routing, and FP8/FP4 quantization all move this, and a claim that hides them behind raw FLOPS has skipped Gate 1 entirely.

At Gate 2, ask what it took to power and cool. "It fits in the site" usually means a facility retrofit — moving from an air-cooled hall built for 5–15 kW per rack to liquid cooling for racks many times denser — that nobody priced. And grid interconnection queues are frequently cited as averaging 4–7+ years now (five-year averages show up in PJM and ERCOT data), while large transformers routinely carry lead times of 18–48+ months. So model capacity as discrete gated availability, never a smooth spend line. This is where a finance-only model is most confidently wrong: it treats time as money and money as instant.

At Gate 3, ask for the depreciation schedule. Any cost/token quoted without its 3-yr vs. 5-yr assumption can't be checked — it's unfalsifiable by construction. Ask for it every time.

And one that sits underneath all three: recent estimates put HBM at something like 40–50% of GPU BOM, but the binding constraint upstream is usually TSMC's CoWoS advanced-packaging capacity, not wafer starts. If you're tracking supply, track CoWoS allocation. That's the gate the fab-wafer headline misses.

Where the smart people still disagree

Two live disagreements are worth watching, because they don't resolve cleanly.

One: are today's physical constraints — rack density, cooling, grid — durable planning anchors, or do they reset every architecture generation? FP4, MoE sparsity, and better packaging change work-per-watt enough that "this is the density limit" may not survive one product cycle.

Two: is "workload non-equivalence" real signal, or a way for an engineer to dodge a losing cost number? Sometimes the cost math is just worse and the architecture argument is cope.

An honest desk leaves these standing rather than pretending consensus. If you follow AI infra, which one do you actually believe — is physics the durable anchor, or does architecture keep resetting the board?

FAQ

Does mixing disciplines slow decisions down? Yes, and that's a real cost. The friction that produces signal also produces meetings. It only pays off if you grade calls; otherwise you've bought slowness with no accuracy.

Can't AI tools do the cross-discipline reconciliation now? They can surface the assumptions faster — pulling the physical constraint a finance model skipped, or the cost constraint an architecture pitch ignored. What they don't do on their own is commit to a dated, priced number and then live with the score. The grading loop still has to be built and owned by someone, and grading is the part that matters.

Is this only useful for semis? No. Any domain where physics and economics both bind — energy, biotech, hardware — has the same orthogonal-blind-spot structure.

Do these three tonight

Pick one AI-infra forecast you believe and run it through the gates:

• Find its hidden assumption. Check whether it uses nameplate FLOPS or derates to a realistic large-run MFU in the 35–45% range. Write down which.
• Demand the depreciation schedule. If the cost/token has no stated 3-yr vs. 5-yr assumption, treat the number as unfalsifiable and set it aside.
• Turn it into a priced prediction. Rewrite the claim as a specific, dated, checkable number — then set a reminder to grade it against reality in six months. That last step is the one that separates a team from a debate club.