The Real Reason AI Is Bad at Design (It's Not Creativity)

Ask any frontier model to write a working function and it nails it. Ask it to design a screen and it gives you something that looks fine and is quietly wrong. Wrong hierarchy, wrong thing on the screen, a nice-looking answer to a question nobody asked.

People explain this by saying design needs taste, and taste is human, and that's still out of reach. That's directionally right and mislabeled. It makes the gap sound like a mystery about the human soul. It isn't a mystery. It's one variable.

Here's the variable: does the domain have a cheap automated verifier?

Start with how these models actually get good at anything. After the base model, you improve it by sampling outputs, scoring each one with a reward, and pushing the model toward the high-scoring ones. Run that loop millions of times. Whether it scales depends entirely on the reward — how dense it is, how fast, how cheap, how accurate.

Now look at code. The reward is a program. Unit tests pass or fail. The type checker throws or it doesn't. The compiler exits 0 or nonzero. The trace shows where it broke. That gives you three things for free.

One, density. A failing test tells you which line was wrong, so the model gets signal at nearly every step instead of one score at the end. Two, throughput. You can run millions of test executions an hour at basically zero marginal cost, and RL needs enormous volume. Three, fidelity. The oracle is nearly deterministic. When the test says wrong, it's wrong, and the gradient points exactly where it should.

Now the judgment part of design. There's no program that runs your layout and returns "good." So you fall back to a learned reward model — usually a Bradley-Terry model trained on pairwise human comparisons. Show a person A and B, ask which is better, fit a model to that. This inverts all three properties.

It's sparse: one score for the whole screen, no signal about which of your thirty decisions made it good. It's slow and expensive: a human does a few hundred comparisons an hour and each one costs money, versus millions of free test runs. And it's noisy: ask several people whether a design is good and they barely agree. Agreement on "is this good design" is notoriously noisy — the kind of noisy where the real signal can be smaller than the disagreement. When your labels contradict each other that much, RL either learns nothing useful or learns the noise.

Ambrosino has made roughly this point — that the human aspect of taste is part of the feedback mechanism, and that it feels out of reach with current technology. That's what people are gesturing at when they call design a taste problem. It's true. But it's true for a mechanical reason, not a mystical one.

Here's the part most people stop short of. Design isn't one task, and the plateau sits at a precise, findable boundary inside it.

Split design in two. The downstream layer is assembling a screen from an existing design system: pick the right component variant, use the system's color and spacing tokens, snap to the 8pt grid, hit WCAG contrast ratios. Every one of those is checkable. Contrast is a hard pass/fail — 4.5:1 for normal text. Token compliance is "is this color from the palette or a random hex." Grid snap is arithmetic. That layer is lintable, which means it's basically code, which means models are already good at it.

The upstream layer is the thirty decisions before that: what belongs on this screen at all, what's the hierarchy, does this metaphor reduce the user's effort or add to it. None of those has a local checkable property. Whether the hierarchy was right only shows up in aggregate user behavior, weeks later. No verifier, no dense reward, no scaling.

The model plateaus exactly at the seam between the two. That's the whole phenomenon. And you can prove it: hand a model a tight enough design system and the taste gap shrinks dramatically, because you've converted judgment calls into lintable constraints. If taste were metaphysically special, constraining the system wouldn't help. It helps a lot. That tells you taste is just the name we give the region where no cheap verifier exists.

When you do force a reward onto the upstream part, you get two specific failures, not vague ones.

First, saturation. A preference model can tell bad from decent, but not good from great. Once every output clears the annotator's threshold, the reward flattens and the model stops improving on the axis you actually care about. It gets stuck at competent.

Second, reward hacking. The model doesn't learn quality, it learns the rater's proxy. Fast or underpaid annotators reward confident, polished, Dribbble-looking UI, so the model produces more of that — and it looks more impressive while getting worse as product design. Optimizing a flawed reward doesn't just fail, it fails in the direction of things that fool the grader.

Now the objection people reach for immediately: just grade design by engagement. A/B test it, measure clicks, let the numbers pick the winner.

That breaks at the mechanism. To be an RL reward, a signal has to be fast, causally tied to the specific artifact, and on-policy. Engagement is delayed by weeks, tangled up with traffic and novelty and seasonality, and measured on last month's model, not the one you're training. But the deeper problem is the fixed point. Maximize engagement and you converge on retention tricks — infinite scroll, manipulative defaults, the slot-machine feed. That's not a noisy version of good taste. It's a different objective that actively rewards worse design. You'd be optimizing hard toward the thing you were trying to avoid.

So where does this leave AI and design?

The lintable half is already collapsing into the solved, code-like regime and will keep getting better fast. The judgment half — what goes on the screen for this user chasing this goal — stays hard, because its quality only exists relative to a specific person's context, and that context has no local property any automated verifier can check. The wall erodes for surface aesthetics. It stays firm for judgment.

Which means "AI can't design" is the wrong sentence. The right one is: AI can execute a design system and can't yet frame a design problem, and the line between those is exactly the line between what's cheaply gradable and what isn't.

Here's the question I don't think anyone's settled. Is that upstream region permanently ungradable, or is it just waiting for a good enough learned verifier — a critic trained on real product outcomes instead of pretty screenshots? If you've watched a model do a design task well, I'd bet it was one where the constraints were tight enough to grade. Tell me where you've seen it hold up, and where it fell apart.

Three things you can do with this.

One, if you want AI to help with design, don't ask it to design. Hand it a real design system with named tokens and component variants, and ask it to assemble screens inside those constraints. You're moving the work into the gradable half where it's actually good.

Two, before you trust any "AI is good at X now" claim, ask the one question: is there a cheap automated verifier for X? If yes, believe it. If no, expect polished output that's quietly wrong, and check it yourself.

Three, whenever you see a metric proposed as a proxy for quality — engagement, dwell time, a preference score — ask what its fixed point is. Picture the world where you've maximized it all the way. If that world is worse, the metric isn't a grader. It's a trap.