The demo always works

The demo always works. That's what makes it dangerous. A vendor shows you an agent answering a hard question, or drafting a policy, or triaging a queue, and it lands. You approve the pilot. Six months later, the pilot is a tab nobody opens, the number on the spreadsheet hasn't moved, and the board wants to know what happened to the AI budget.

Nothing exotic happened. The pilot hit the same two walls most enterprise AI projects hit. Neither wall is the model. The first is the harness. The second is your own team.

Start with the harness. A model is a gifted hire on day zero: no tools, no rules, no memory, no manager. On its own it can reason, and almost nothing else. Everything that turns it into something you can depend on sits around it: the tools it can call, the gates that stop a bad output before it ships, the evals that say whether this week's version beat last week's, the memory that keeps it from starting every task from a blank slate, the traces that let you audit why it did what it did. That scaffolding is the harness, and it is roughly ninety percent of the work. It is also the ninety percent the demo skips.

The industry has started to name this. The framing making the rounds puts a spectrum between "vibe coding," where you prompt an agent and accept whatever comes back, and "agentic engineering," where formal specs, automated evals, and deterministic gates surround every model call. The honest finding underneath the framing is the one worth keeping: examined closely, most agent failures are not model failures. They are configuration failures. A missing tool. A vague rule. An absent guardrail. A context window stuffed with noise.

We agree with that, because we see it in every stalled pilot we're brought in to diagnose. The model was fine. What was missing sat around it. Where we part company with the discourse is who it's written for. The spectrum is a developer's map. It treats the problem as one of engineering rigor and stops there. Inside a real company, engineering rigor is necessary and not sufficient, because the second wall has nothing to do with code.

The second wall is adoption. This is the one nobody pitches, and it is the one we hear about most. The tooling gets built. It works. And the team doesn't use it. The support org keeps answering tickets the old way. The analysts keep pulling the report by hand. The thing you paid for runs in a corner while the work routes around it. Building the capability turned out to be the easy half. Getting a hundred people to change how they work on a Tuesday is the half that decides whether any of it was worth doing.

The reason we see this coming is that we came up in games, where you cannot ship a system your players won't touch and call it a success. Adoption is the whole scoreboard. That instinct transfers. When we build for an enterprise, adoption is scoped like a feature, not hoped for after launch: named internal champions, training built into the rollout, usage targets you can actually see, and runbooks so the capability survives after we leave. A system your people don't use is not a smaller win. It's a loss that takes longer to show up on the invoice.

So the work is two problems, not one, and we build for both.

On the harness side, the standard is concrete. Every number a system produces carries a tag saying where it came from: sourced, estimated, modeled, or derived. A surface that can't say where its numbers come from isn't finished. Every output clears a deterministic gate before it ships or spends. Agents earn their place by beating real history, not by looking impressive in a demo. One of our live systems, the live-ops brain of a top-20 mobile games studio, simulates every event against real player segments, checks it against hard economy rules, and backtests it against events that actually shipped, under 490 automated tests. Another runs a company's organic and paid marketing on one platform where nothing publishes and no budget moves without passing a rubric-graded gate first.

On the adoption side, the standard is just as concrete, and it starts before the build. We map who does the work today, what would make them trust a system over their own hands, and what "used" looks like in numbers we agree on up front. Then we hold ourselves to it, because the measurement was there from the start.

The pitch is always the model. The next model will be smarter, the story goes, and then this will work. It won't, for the same reason a faster engine doesn't finish a car. The teams and firms getting real value out of AI are not the ones with privileged access to a better model. They are the ones who built the harness once and got their people to actually use what it produces.

If you've run the pilots and watched them flatten, the problem is almost certainly one of these two walls, and usually both. That is a diagnosis we can make on a thirty-minute call, and it's the one we'd start with.