Building AI Agents: Where Ops Ends and Engineering Begins

The people building the best enterprise AI agents mostly can't write code. That's the claim, and it's half right in a way that matters. Here's the honest version, and it's the answer if you're deciding who should own agents on your team.

In a mature customer-experience agent setup, operations and CX staff author the agent's behavior through no-code natural-language "journeys." Engineers don't write those. But engineers own the capability surface — the actual actions the agent can take — and the blast radius when something breaks. So "no engineers needed" and "ops builds the agent" are two different sentences that got smashed together. Nobody replaces anybody. The dependency inverts. Ops moves to the front. Engineering moves under the floor.

Where this comes from

The claims and mechanics below come from LangChain's Max Agency podcast — a conversation between Harrison Chase and Sierra's Zack Reneau-Wedeen — not from independent LensUp reporting. In that conversation, the workflow described is a build/analyze/release web app built with a tool called Ghostwriter, a Codex/Claude-Code-style agent builder. In it, "journeys" are described as natural-language standard operating procedures, and the people who iterate on them day to day are described as primarily operations people and customer experience managers, not engineers.

I'm attributing this deliberately. Everything specific here — journeys as SOPs, ops as the iterators, the voice-agent architecture, the pricing example — is how that source describes its own system. Treat it as one credible account of how a production agent platform is built, not as a universal law of the field. The reasoning about tradeoffs is where you should push back yourself.

An agent isn't a chatbot. It's a pipeline.

Start with the wrong picture most people carry: an agent is one big model you talk to. It isn't.

A single conversation turn can invoke multiple specialized models — small classifiers that decide what the user wants, generators that decide what to say, routers that decide where to go next. According to the source conversation, for voice agents the hard part was running thinking, listening, and talking in parallel, with a dedicated silence-detection model whose verdict gets trusted or overridden by another model to decide when the agent should speak. So "the agent" is described as an orchestration of specialized parts, not a personality.

This is why ops can build it. The behavior doesn't live in the model weights. It lives in the routing and policy config on top. You can change what the agent does without retraining anything. That's the crack no-code slips through.

A "journey," as the source describes it, is an SOP compiled into a decision graph.

The word "journey" makes it sound soft. In the source's account, it isn't. A journey reads like plain English — "process a return" — but functions as a natural-language SOP that resolves into a decision graph. Each branch is a node. Each node needs a signal to decide which way to go.

And here's where the naive version dies. "Refund if reasonable" is a null instruction to a model. A human reading that SOP silently fills in "reasonable" with years of context. The model has none. So it invents a policy and states it with total confidence to your customer.

That failure has a name worth carrying: implicit branching. Every unstated judgment call becomes a random node. The real authoring work is dragging those judgments into the light. For each ambiguous case, you decide whether the agent should ask — get a signal from the customer — infer — pull it from a tool — or escalate — hand it to a human. A journey with no explicit branch for "customer already got a refund" will just make something up.

So the person who's worked thousands of tickets is the strongest author. They know the minority of return cases where the customer already got refunded, has a warranty claim tangled in, or is lying about the ship date. An engineer doesn't know those edges exist. That domain knowledge is the product. It's exactly why the source names ops and CX managers, not engineers, as the iterators.

But every action bottoms out in a tool someone hardened.

Now follow the logic one step past the comfortable stop.

"Lying about the ship date" — where does the real ship date come from? A tool. The agent can only see reality an engineer surfaced into the journey's context. Your policy is only as good as the observable signals available to grade against.

And anything the agent does — not says, does — bottoms out in a typed function an engineer wrote and hardened:

• issueRefund(order_id, amount, idempotency_key). The idempotency key exists so a network retry doesn't refund twice. Non-negotiable for anything that moves money.
• Scoped OAuth. The agent can only touch accounts and actions explicitly granted. A prompt injection can't widen that grant, because the grant is enforced outside the model, not asked of it politely.
• Schema-constrained arguments. A hallucinated argument gets rejected at the tool boundary instead of executed.

So "no-code" is true, but only inside a sandbox engineering defined. Ops has freedom to rearrange behavior right up to the walls of that sandbox. The moment ops needs a new action the platform doesn't have, they're blocked on engineering again. That's the real bottleneck, and no natural-language UI removes it.

This is the balanced picture the source supports: ops may own day-to-day behavior because they know the workflow and the customer conversations, but engineers stay essential for integrations, tools and packages, observability, security, testing, escalation, extending the platform, and guardrails.

The commission-pricing example the source uses to mark the boundary.

The source conversation mentions outcome-based pricing — cases where the vendor earns a commission when the agent closes a sale. Sit with that for a second.

You can't let a probabilistic model self-attest "I closed the deal," because the money depends on it. The attestation has to be an engineered verification against the order-management system of record. No journey node can do that.

The same logic is described for payment data. Where LLM providers aren't in scope for card-data handling, the safe move described is to keep card information off the model entirely — payments run on isolated infrastructure the model never touches. I'll hedge here on purpose: compliance scoping and certification depend on your provider, your contracts, and your auditor, so treat "isolate payment data from the model" as a sound design instinct, not a blanket claim that any given setup is certified. The regulated, money-moving surface is engineered and quarantined. Ops authors everything around it.

That's the boundary in one sentence: ops owns policy and evals; engineering owns capability and blast radius.

Stop optimizing accuracy. It's the wrong number.

If you take one metric away, take this one. On a customer-service agent, "accuracy" is close to meaningless. The number that matters is containment rate at a fixed CSAT floor — how many conversations the agent fully handles without tanking satisfaction.

Containment that wrecks CSAT is worse than a warm transfer to a human. And a single accuracy score hides the tradeoff that actually costs money: over-refund rate, where you leak cash, versus over-escalate rate, where you defeat the whole point and pay humans anyway. These fail in opposite directions. No accuracy number tells you which way you're bleeding.

So the artifact a good ops author actually maintains isn't prose. It's a regression suite of adversarial conversations and grading rubrics, plus a per-journey confusion matrix showing over-refund against over-escalate. That's engineering discipline applied to language — another reason engineers don't disappear, they just move under the floor.

The part nobody budgets for: ops now owns the 2am incident.

Here's the step most write-ups skip. The moment ops owns the journey, ops owns the production incident — and most ops teams have never carried one.

Ops can edit a journey and ship it to 100% of traffic instantly. That's not empowerment. That's an unguarded gun. No on-call rotation, no staging environment, no rollback muscle. A confusion matrix means nothing if there's no pager when the over-refund branch fires overnight.

So the tooling has to impose the discipline ops never had: versioned journeys, staged rollout by traffic percentage — ship to 5% first, watch, then widen — and a hard approval gate on anything that touches money or PII. Not policy-discouraged. Structurally enforced. If ops can quote a made-up return policy to 100% of customers with one save, your platform is broken, not your people.

The team shape that actually works.

Put it together and the structure falls out on its own. It's forward-deployed: a small platform team backing many more ops authors — a handful of engineers behind a whole floor of them.

Engineers front-load the constraint surface: tools, permissions, guardrails, the eval harness. Ops changes behavior freely inside that surface, day to day, without an engineer in the loop. Get the ratio and the RACI wrong and you land in one of two failure modes: bottleneck everything on engineering, which defeats the point of no-code, or let ops fire the unguarded gun, which defeats the point of production.

There's a real tension still living here, and it's worth arguing about. Ops people say the platform's value is that domain experts iterate without engineers. Infra people say that's only safe because engineers built the walls first — and the instant ops needs a new action, they're blocked again. Both are right. Where the true bottleneck sits — edge-case knowledge or capability surface — probably flips depending on how mature your tool registry is. If you run agents in production, which one bites you more?

Three things to do tonight.

• Pull one journey and find its implicit branches. Read it as if you have zero domain context. Every place you'd have to "just know" something, mark it. For each mark, decide: ask, infer, or escalate. That's your list of null instructions to fix.

• Replace your accuracy dashboard with a two-number one. Add over-refund rate and over-escalate rate, tracked against a CSAT floor. If you can't compute those yet, that's tomorrow's ticket for engineering — you need the tool signals to measure them.

• Check whether ops can ship to 100% of traffic in one click. If yes, that's your top risk, not a feature. Ask your platform team for staged rollout by traffic percentage and an approval gate on any money- or PII-touching action.

Ops authors behavior. Engineering owns capability. The best agents come from getting that boundary exactly right — not from pretending either side can build alone.