The honest one-sentence version

Let's start with the version of the claim that survives contact with a real codebase: AI didn't make implementation cheap. It made code generation cheap, and it left verification, integration, and provenance exactly as expensive as before — arguably more expensive, because now there's more stuff to verify and reconcile.

So the real reframe isn't "implementation is cheap." It's this: the cost migrated from authoring to selection-and-integration. Taste becomes the bottleneck only after you've built and paid for the machine that gets you from many candidates down to a few. Skip that machine and the bottleneck isn't taste — it's a human reading a stack of diffs, which is just labor wearing a nicer word.

If you're deciding whether to blow up your PRD process, or trying to make parallel-prototyping actually pay off, this is the mechanism you need. Let's build it one rung at a time.

A note on sourcing before we go further. The framing that kicks this off — implementation is cheap, taste is what's left, and an AI-driven process can spin up something like 90 uncoordinated attempts to sift for what's worth keeping — comes from Andrew Ambrosino on Lenny's Podcast, describing how OpenAI (and AI-driven product development more broadly) works. The rest of the machinery in this piece — the staged selection funnel, golden traces, the pairwise LLM judge, Elo/Bradley-Terry ranking, and prototype isolation — is LensUp editorial synthesis: a proposed workflow for making that framing usable, not a claim about what any lab actually does.

Implementation isn't one cost — it's four

When someone says "implementation is expensive," they've quietly bundled four different jobs into one word:

  • Generation — writing the code.
  • Verification — proving it does the right thing (builds, tests, matches expected behavior).
  • Integration — making it live alongside everything else without breaking the seams.
  • Operation / provenance — running it, and being able to figure out why it does what it does when it breaks at 3am.

AI collapsed generation toward zero. It touched none of the other three. If anything it made them worse, because when generation gets cheap you produce more candidates — and every one still has to be verified, integrated, and attributed.

That's the whole trick behind the "taste is the bottleneck" line. It's true about the one cost center that got cheap, and completely silent about the three that didn't.

Tonight: take your last shipped feature and write down where the hours actually went — writing code vs. reviewing it vs. reconciling merge conflicts vs. debugging in prod. If "writing" wasn't the biggest bucket even before AI, the "implementation is cheap now" story never applied to you in the first place.

The funnel is the real machine

Say generation really is free and you now have a pile of candidate implementations of the same feature. The scarce resource is no longer typing — it's throughput of judgment. And a human judging raw candidates is brutally expensive per unit.

So the thing that makes cheap implementation actually pay off is a selection funnel that stages judgment from cheap to expensive:

  • Deterministic gates (run on everything). Does it build? Lint pass? Do unit and property tests pass? Does it reproduce a golden trace — a recorded reference execution the candidate must match? Many candidates should die here before a human looks at anything.
  • LLM-as-judge, pairwise. Survivors get compared two at a time — pairwise is more reliable than asking a model for an absolute score — and the results aggregate into an Elo / Bradley-Terry ranking. This narrows the field to a shortlist.
  • Human final cut (run on the top few). The human ranks only the shortlist and picks.

Notice what happened. "Taste is the bottleneck" is only true at step 3. Steps 1 and 2 are what make step 3 affordable. The funnel is the new expensive artifact. Nobody who says "taste is the new scarcity" mentions that they already built the machine that makes taste the only thing left.

Tonight: pick one feature and write a single golden trace for it — the exact input, the exact expected output or behavior, the thing a candidate must reproduce to survive. That one assertion is the first gate of your funnel.

Where the funnel works — and where it collapses

Parallel fan-out pays off under exactly one condition: the spec is a verifiable oracle. You can mechanically say "candidate X beats candidate Y." That holds for backend logic with property tests and traces.

It collapses for ambiguous UX. You cannot cheaply, mechanically rank subjective interaction quality. So the automatable middle of the funnel — step 2 — has nothing to grade on, and every candidate gets dumped straight onto a human, which reintroduces the exact labor cost the fan-out was supposed to remove.

This is the real, unromantic reason AI is "bad at design": design lacks a cheap grading function. No cheap oracle means no funnel means the human stays in the loop for every single candidate. Keep that failure mode in mind — it's also where the hidden costs below come from.

The "90 prototypes" number is a frontier-lab artifact

Ambrosino's illustrative context is a process running something like 90 different explorations — 90 uncoordinated attempts, which get reframed as "90 prototypes" you then sift down to what's worth keeping. It sounds like rigor. Before you copy the number, notice what makes it possible there: tokens are effectively free at that scale, and there's no shared production constraint forcing the attempts to serialize. Both are luxuries most teams don't have.

There's also a diminishing-returns question worth sitting with. What kills a wrong direction is divergence, not sample count — a handful of genuinely different approaches falsifies more than dozens of variations on the same idea. So for most resource-constrained teams, 3–5 deliberately divergent prototypes may be a better starting point than copying a frontier-lab-scale number. Aim to cheaply kill a wrong direction, not to nudge around one you've already committed to.

The point isn't that 90 is wrong. It's that 90 is a description of a specific, well-funded context, not a method that generalizes. Copy the number without the unlimited budget and you're copying someone's constraint and calling it a methodology.

The hidden tax: shared schema, flaky tests, lost provenance

Here's the cost that "cheap implementation" pretends away. Your candidates aren't actually comparable, because they diverge on the seams. Each agent invents a slightly different API contract, writes its own migration, adds tests that pass in isolation.

Even the candidates that clear every gate still can't all merge, because:

  • Shared schema is a serialization point no parallelism removes. If two prototypes both touch the same database schema, you're reconciling by hand no matter how parallel the generation was.
  • Flaky-test amplification: tests that pass in isolation interact combinatorially once merged.
  • Provenance loss: you pick a winner authored by nobody. Bisecting an agent-written regression when no human holds the original intent is a new, genuinely nasty class of pain.

The rule: cheap parallel prototyping only stays cheap if each prototype is architecturally isolated — separate branch, separate ephemeral DB, no shared migration. Skip the isolation and "cheap prototyping" quietly converts into expensive integration debt.

Tonight: for your next parallel experiment, give each prototype its own ephemeral database and forbid shared-schema changes. If they can't be isolated, run 3, not 30.

What "taste" actually decodes to

"Taste" sounds like an innate, mystical trait. It's mostly a legible checklist that AI output fails structurally — not superficially.

Agent output clusters to the median of its training data. It looks done. It's generic. And it silently omits the states nobody prompted for. The reject-in-five-seconds heuristics:

  • Does it handle zero / one / many?
  • Does the primary action survive when the happy-path data is gone?
  • Is the empty state honest about what the user should do next — or is it a pretty blank box?
  • Error, loading, overflow, novice-first-run — are they there at all?

Agents nail the populated happy path and structurally miss the missing 10%. So "taste" decodes to a concrete question: who has the strongest mental model of the user's actual context? And that's exactly what parallel generation can't supply — you can't sample your way to a prior you don't have.

The PRD didn't die — it inverted jobs

The PRD isn't dead. Its job flipped.

The old PRD de-risked implementation — a 20-page spec plus full Figma, written because building the wrong thing was expensive. The new artifact de-risks selection — because now generating the wrong thing is cheap, but choosing among many is where you bleed.

So the artifact shrinks and changes shape: from "20-page spec + full Figma" to "1-page problem framing + 15 testable assertions + 3 reference interactions." And here's the elegant part — that assertion set is the golden trace your funnel gates on, and it's the rubric your human curates against. Same document. Different job.

Tonight: rewrite one PRD into that shape — one paragraph of problem, 15 assertions a candidate must satisfy, 3 reference interactions. Watch how much of it becomes machine-checkable.

The one fight that isn't settled

There's a real disagreement worth sitting with. Every legible taste heuristic — "empty state exists" — can be encoded as a screenshot-diff or a stateful trace test and moved left in the funnel, automated, killed deterministically, so it never costs a human's attention.

But you can automate "empty state exists." You can't cheaply automate "empty state is honest and useful" — that still needs a human holding the user's context. Same with the problem definition itself. There's an irreducible residual — problem framing plus user-context mental model — that no rubric captures.

So the practical answer is: taste is progressively automatable at the edges and irreducible at the core. Every legible heuristic you write down shrinks the human's job to the part that actually needed a human. That's not a threat to product judgment. It's a promotion.

Which raises the honest question for your own team: how much of what you currently call "taste" is really a checklist you just never wrote down — and how much is the irreducible part? The answer decides how much of your funnel you can automate and how much stays human-bound.

Do these three things tonight

  • Write one golden trace for a real feature — exact input, exact expected behavior — and make it the first gate any candidate must pass.
  • Rewrite one PRD into: 1-page problem framing + 15 testable assertions + 3 reference interactions. That's your rubric and your grader.
  • Isolate your prototypes — separate branch, ephemeral DB, no shared migration — and run 3–5 deliberately divergent ones, not 90 variations. If you can't isolate them, that's your real bottleneck — and it isn't taste.