AI Builds Exactly What You Ask For — That's the Problem

A Product Manager (PM) at a mid-size SaaS company specs an AI summary feature for Flowdesk, their ticket management tool. Support engineers have been complaining for months — too many tickets, too much context-switching, too much time spent reading thread histories before they can respond. The solution seems obvious: surface an AI-generated summary at the top of every ticket. Let the agent read the thread. Let the engineer start from the summary.

Four acceptance criteria. Summary generated on ticket open. Accuracy threshold defined. A thumbs up / thumbs down feedback toggle. The sprint review passes without comment.

The agent builds it in two days. Summaries generate correctly. Accuracy scores green in testing. The PM ships it.

Week one adoption is strong. Engineers open tickets, glance at the summary, move faster.

The summary was not wrong in the way a hallucination is wrong. It was confidently incomplete. It captured the majority of the thread accurately and omitted the one exchange that changed everything.

After that postmortem, the word spread quietly across the support floor. Nobody sent a memo. Nobody filed a ticket. Engineers simply stopped trusting the summaries. They kept opening tickets, reading the AI output, and then reading the full thread anyway — double the work, none of the speed.

The spec had an accuracy metric. It had a feedback toggle.

The bug was not in the model. The bug was born in the spec — two weeks before the agent wrote a single line.

The problem AI-assisted development didn't create — but made more expensive

This is not an unusual story. Every Product Manager reading this has either written that spec or worked alongside someone who did.
The acceptance criteria were reasonable. The instinct was right. The feature shipped on time.

What was missing was not skill. It was a question nobody thought to ask before the build started.

This is where a new category of problem enters the age of AI-assisted development.

Hallucination lives in the document, not just the model

Hallucination is the AI's most discussed flaw. But there is a quieter version of it — one that lives not in the AI's output, but in the document it builds against.

The spec is the product manager's artifact. And product management is not a universal discipline. It is shaped by company culture, by what the organization measures, by the specific product being built, and by the individual PM's experience and blind spots. Two PMs writing a spec for the same feature will produce different documents — not because one is wrong, but because each brings a different set of lenses and leaves different gaps.

This is where spec-level hallucination lives - unknown assumptions not missing facts, unasked questions not wrong answers.

The Sparring library is built to surface them.

But before we go to the Sparring library, a key question needs to be answered

Why competent PMs miss the obvious question(s)

Let me starts by clarifying that it is not negligence. It is how working memory actually works.

In 1988, cognitive psychologist John Sweller described what he called cognitive load — the limits of how much a working mind can hold and process simultaneously. The finding is straightforward: human working memory is narrow. Experts compensate not by having better memory but by chunking — collapsing familiar patterns into single cognitive units so they can be recognised and retrieved quickly without occupying working memory.

A PM who has shipped AI features into B2B products before has a chunk for accountability. "Who owns a decision made on AI output?" surfaces automatically — before the spec is written, not after the post-mortem. The question feels obvious in retrospect.

A PM who has mostly built deterministic features — where the output is predictable and verifiable — may not have built that chunk yet. They spec the accuracy threshold and add a feedback toggle. Not carelessly. That is just how working memory works when a pattern is not yet stored.

Nobody in the sprint review asked it either. When everyone in the room is operating from the same pattern library, they share the same blind spots.

Why agents make this worse

AI-assisted development sharpens this problem. The speed at which software is expected to be developed with smaller teams has created the second order impact of reducing the opportunities of scrutinizing the spec.

In a non-AI world, an architect building the high level design based on the spec catches might catch the unsaid assumptions. An expert reviewing the spec to build the Low level design flags might flag the missing Non functional asks. A human developer who misses a gap in the spec will sometimes catch it mid-implementation, the act of writing code makes the missing piece visible. They flag it and spec if fixed.

An AI agent does not do this. It builds what the spec says.

Tools like Superpower have brought real discipline to the coding phase — the agent now plans, executes, and reviews in structured sequence, no longer racing ahead on vague instructions. The spec it builds against is still entirely on you.

The gap that a human might have caught in the act of building now ships cleanly and invisibly.

This is an exposure gap, not a talent gap

The Tier-1 tech organizations have build Product practices and processes to catch spec hallucination even before AI took over the software development world. The thinking models at Tier-1 companies were built over decades of shipping at scale, getting things wrong, and encoding those lessons into how teams work.

Most PMs reading this will not have worked at Amazon or Google. They will not have sat in a room where a press release is written before a spec, or where a counter-metric is as mandatory as a success metric.

That gap is not a talent gap. It is an exposure gap.

A PM at a mid-tier company does not lack the intelligence to ask those questions. They lack the structure that makes asking them automatic.

What thirty minutes of spec interrogation using Sparring looks like

Go back to that Flowdesk AI summary spec.

The Amazon PM persona reads it before the sprint starts. First question: who is the specific customer? Not "support engineers" — which engineer, in which moment, making which kind of decision? The spec has not named them. The persona blocks.

The PM thinks about it properly for the first time. The primary user is not the engineer skimming low-stakes tickets. It is the engineer making a decision on a complex escalation — the highest-stakes moment in the workflow. The spec was optimised for the easy case. It had no answer for the hard one.

The Amazon PM persona asks a second question: what does failure look like — and is it defined? The accuracy threshold in the spec measures whether the summary is technically correct. It does not measure whether the engineer can tell when the summary is incomplete. Those are different things. A summary that is 90% accurate and omits the critical 10% is not a failed summary by the spec's definition. It is a passing summary that produces a wrong decision.

The spec narrows: the primary user is the engineer making a consequential decision, not a routine one. The feature needs a completeness signal — not a confidence score the AI cannot honestly produce, but structural heuristics based on what can actually be verified: thread length, time span, whether a status change was detected mid-thread. These are facts about the input, not claims about the AI's self-awareness.

The Google PM persona reads the revised spec. No accountability model. No definition of what happens when an engineer acts on a summary that turns out to be wrong. If the engineer bears the consequence of a decision made on AI output, the spec needs to say so explicitly. Silence on this question does not make the question disappear. It makes it a post-mortem.

The Google PM persona also flags: no counter-metric. Adoption is the obvious success metric. But adoption can be high while trust is eroding — the week-one data would have looked like success right up until the escalation. The spec needs a metric that measures whether engineers are making better decisions, not just whether they are opening the feature.

Seven findings. Two personas. Thirty minutes of spec review.

The agent then builds a summary feature with structural risk indicators based on verifiable thread signals, a source-jump link when those signals fire, an audit trail logging which summary version the engineer viewed before taking action, and a measurement plan that tracks decision quality — not just adoption.

Each persona also documents where it over-indexes. The Amazon PM will demand a named customer narrative even when the customer is already understood. The Google PM will ask for instrumentation depth a small team cannot sustain. The bias section exists so the PM remains the judge, not the persona. Every finding is evaluated — accepted, overridden, or deferred — with a documented reason.

Add your product specific context

And because your company is not Amazon or Google — because your product has its own context, its own users, its own constraints — every persona has a configuration block to respect your compliance requirements, your team conventions, your tech stack.

The persona brings the cognitive model. Your PM brings the domain knowledge. What comes out of that combination is stronger than either produces alone.

The Flowdesk AI summary spec would have taken thirty minutes to interrogate. The feature would have shipped with a trust architecture the team had actually thought through, a metric that could detect silent erosion before it reached a postmortem, and an accountability model that protected both the engineer and the product. Instead it shipped with an accuracy threshold, a thumbs toggle, and a silent assumption that correctness and trustworthiness are the same thing.

Try Sparring on your next spec

The Sparring library is open source. The personas are plain markdown — no tooling dependency, no installation, no configuration beyond your own project context. You paste the persona, you run the interrogation, you decide what to accept. The PM stays the judge.

The worked example in the repo shows the full Flowdesk AI summary pipeline: the input spec, each persona's findings, and the delta between what was written and what needed to exist before a single line of code was written. It takes less time to read than the sprint review that passed the original spec without comment.

Image by Rosy / Bad Homburg / Germany from Pixabay