TL;DR: I wrote an acceptance criterion for a contractor I hadn’t hired yet, and it was specific, binary, and testable. It was also impossible, because one of the three possible causes of the bug I wanted fixed is operating-system behavior nobody can change. Specific and testable is two-thirds of the job; the missing third is controllable. Run every criterion through three questions before it touches a contract, and require the root cause to be named with evidence before you fix the scope or the price.
On Sunday evening, I was in my mother’s kitchen in Miami, three thousand miles away from the jars I was writing about, when I typed a sentence I felt proud of.
AC-1.3. A physical tag tapped by a phone with no app installed reaches the destination page with no interstitial of any kind.
Take another look at it. It names the actor, the action, the artifact, and the outcome. Even someone who never joined a meeting with me could read it and make a decision. It’s binary, with no room for partial credit, no “substantially complete,” and no vague adjectives for lawyers to debate. By every rule I’ve learned about writing acceptance criteria, that sentence is solid work.
But it could have ended up being the most expensive sentence I wrote all year.
About ten minutes later, I realized I should have started with a different question: not how to describe the outcome I wanted, but who could actually deliver it. The truth was, I didn’t know, and one possible answer was nobody. Not this contractor, not a better one, not even Apple unless they released a new OS. I would have written a promise into the product spec that nobody could keep, and then paid someone to fail. I’ve faced something like this before, when I said you should stop tuning the prompt and give your AI agent a contract. I still think that’s true. But I missed that an impossible contract is worse than having no contract at all. Here is what happened.
The bug, and why it mattered so much
I’m building a spice jar that lets you tap it with your phone to see where the contents came from, the farm, the grind date, and the person who grew it. The entire product depends on this one physical interaction for a customer who’s never heard of us, standing in their kitchen with a phone. If the tap doesn’t work, there’s no product. Everything else is just packaging.
A thousand printed tags arrived in late July. While I was in Florida for my mother’s ninety-fifth birthday, a candidate for the build role told me, without being asked, that the tap didn’t go straight through. Instead, it hit a banner first. You tap, something pops up, you tap again, and only then do you reach the destination. Out of thirteen candidates, nobody else mentioned it. A few days later, another candidate tested a live tag on her own device and sent me a video showing the same banner.
So the defect was confirmed, reproduced on two devices, and documented. That’s good. But confirming a defect isn’t the same as diagnosing it, and I only had confirmation.
Three causes, three completely different worlds
The main thing to understand is that there are three possible reasons for that banner, and the differences between them really matter.
Cause A: a redirect service is inserting a pass-through page. If the URL burned into the tag points at a short-link or tag-management service rather than at our own domain, that service can put its own branded page in front of the destination. Confirm it by reading the raw data off a tag and checking whether the address on it is ours. Fix: re-encode the tags with our address written in directly. This takes an afternoon of work.
Cause B: the destination is behind a wall. Our hosting project may have deployment protection switched on, or the domain isn’t fully attached, so the page serves an auth or preview screen first. Confirm it by opening the same address in a desktop browser with no session; if the screen still appears, the tag has nothing to do with it. Fix: turn a setting off, finish a domain attach. An hour, maybe two of work.
Cause C: it’s the phone. iOS shows a system notification when it reads an NFC tag, and the person has to tap that notification to open the link. That behavior belongs to the operating system, and any encoding, any hosting change, any framework, or any amount of money can’t remove it. Confirm it by tapping the same tag on an iPhone and an Android side by side; if Android is clean and iPhone isn’t, you have your answer. Fix: there isn’t one.
A and B can be fixed in a few hours. C, on the other hand, is like a law of physics for me and the product spec. The criterion I wrote no interstitial of any kind applies to all three cases. It’s specific and testable in each one. But in one case, it’s a trap.
Why smart people write these
I want to be fair here; AC-1.3 wasn’t careless. In fact, it was the opposite.
When you write acceptance criteria, your goal is to clearly describe the experience you want the customer to have. I want someone to tap a jar and see a farmer, not a banner. Being clear about that is the job, and all the advice you get from your first agile training to your last contract review tells you to be specific and testable, leaving nothing open to interpretation. That advice isn’t wrong, but it’s incomplete. Its limits only show up when you’re paying someone to deliver.
The problem isn’t wanting something. The problem is turning that wish into a pass/fail condition before anyone has figured out why it isn’t already true. Contract best practices make this clear: criteria must be possible for the responsible party to pass or fail, and a good test is whether an operations lead who never joined the original meetings could apply them (Sprintlaw, a UK legal practice writing on acceptance testing for software and SaaS agreements). I used to think this was just about avoiding vagueness, but it’s also about making sure the goal is actually reachable. A criterion can be perfectly clear and still be impossible for everyone who signed it.
So if you’ve written a criterion like this—and I bet most PMs have you weren’t being careless. You were being clear, just like everyone told you to be.
The Controllability Test
Here’s the check I now use for every acceptance criterion before it goes into a contract, statement of work, or milestone payment. There are three questions, in this order. If a criterion can’t answer all three, it’s not ready, yet it’s just an open diagnosis that got promoted too soon.
1. Who can make this true?
Name a person, not a team, not “engineering,” not “the vendor.” It should be one person with both the access and authority to make the change. If you can name them, write their name in the margin. If you can’t, you’ve found an uncontrollable criterion, and you should stop and start diagnosing instead.
This test is intentionally simple, because vagueness can hide here. Saying “the contractor will ensure the tap is clean” sounds like you’ve named someone, but it doesn’t count if the contractor can’t actually access the layer where the problem is.
2. What evidence proves the cause?
Focus on the cause, not just the symptom. A video of a bug shows the bug exists, but it doesn’t explain why. My candidate’s video proved the banner was real and reproducible, but it didn’t help tell A, B, and C apart. Root-cause discipline is a skill I’ve written about before, like when I traced why most kitchen spices taste bland back through five layers of the supply chain.
So, write down what would answer the question. For me, that meant reading the raw data from a tag, opening the destination in a clean desktop browser, and tapping the same tag on both an iPhone and an Android device. These three simple tests together reveal the cause. Then, make producing that evidence its own paid deliverable before you price the rest of the work.
3. What happens to the price when the cause is named?
This is the question that turns the test from theory into real money. If it’s cause A, the fix takes an afternoon. If it’s cause B, it’s an hour. If it’s cause C, there’s no fix, and what I really need is a design decision to make the extra tap feel intentional instead of broken.
Three causes mean three different prices, but only one criterion. Set the price first, then figure out the cause, and one person takes on the difference. I’ve seen this from the other side too, when four co-packers stalled because of a decision I hadn’t made, and I spent nine days blaming vendors for a bottleneck that was actually mine.
The PM lesson
Specific, testable, controllable. Everyone teaches the first two, but almost no one talks about the third, and that’s where the money is.
An acceptance criterion isn’t just a description of the outcome you want. It’s a promise about who controls what, written while both sides are still on good terms. That’s why a criterion that’s specific but uncontrollable is riskier than one that’s just vague. Vagueness gets renegotiated because everyone knows it’s flexible. Precision gets enforced. A precise but impossible clause leads straight to a dispute with a deadline.
There’s another effect too. If you write a criterion that nobody can meet, you don’t just risk a contract fight later;r you get bad information right away. Good candidates will read it, decide you don’t understand your own product, and either add risk to their price or walk away. The criterion sends a message about you before it ever tests them.
What this looks like in practice
Here is the actual rewrite, from the document I approved this morning.
Before, version 1:
AC-1.3. A physical tag tapped by a phone with no app installed reaches the destination with no interstitial of any kind.
After, version 2, split into three:
AC-1.1, root cause named. The contractor identifies the interstitial’s cause as A, B, or C, and provides the evidence used to reach it: a raw tag read, a clean desktop browser check, or a cross-platform comparison. A written cause with no supporting evidence does not pass.
AC-1.3, clean tap on Android. A physical tag tapped by an Android device with no app installed reaches the destination with no interstitial of any kind. Demonstrated on video.
AC-1.4, iOS tap resolves. A physical tag tapped by an iPhone reaches the destination. If the operating-system notification is confirmed as the cause, that single banner is accepted and is not a defect. No interstitial beyond it is acceptable. Demonstrated on video.
Then, in a section called Explicitly out of scope, I added one line: the iOS system NFC notification banner, if confirmed. Nobody can deliver this. Naming what you’re not buying is just as important as naming what you are. This is a habit I first developed when writing the Subtraction Audit for features. It works the same way for obligations.
Notice what this split achieves. Diagnosis becomes a paid, evidence-based deliverable instead of a favor I hope someone does. The controllable part, Android, remains binary and strict, because there I can truly demand perfection. The uncontrollable part gets named, limited, and removed from the discussion before it can cause arguments months later.
The whole reframe took about forty minutes and left me with one unanswered question, which I’ve now sent: which device and OS version was your video recorded on? That one message narrows three possible causes down to one or two, and forty minutes is a lot better than dealing with a change order or a strained relationship.
What to do this week
Pick the nearest contract, statement of work, or milestone definition you have open. Then:
- Read each acceptance criterion out loud and write the name of a single person next to it in the margin. Not a team, a name.
- Circle every line where the margin is blank. Those aren’t real criteria yet.
- For each circled line, write down the simplest test that would identify the cause. If you can’t name a test, the line isn’t ready, and no amount of rewording will fix that.
- Make diagnosis its own milestone, with its own evidence requirements and payment, before you move on to fixing anything.
- Add an “explicitly out of scope” section and list the uncontrollable items by name. Do it now, while everyone is still on good terms.
- Send the clarifying question you’ve been avoiding because it seemed too minor. Mine was about a device and OS version. Yours is probably just as small.
If you only do one thing, do the margin pass. It takes twenty minutes and will catch the sentence that could have cost you.
What to steal
Paste this into Claude, or run it by hand, against any acceptance criteria you have open.
You are reviewing acceptance criteria before they attach to a contract.
For EACH criterion below, output a table row with:
1. CRITERION (verbatim)
2. CONTROLLING PARTY: the single named party with both the access
and the authority to make it true. If none exists, write NONE.
3. UNKNOWN CAUSE? YES if the underlying behavior has not been
root-caused with evidence. NO if it has.
4. CHEAPEST DIAGNOSTIC: the single least expensive test that would
name the cause. Leave blank only if row 3 is NO.
5. PRICE SENSITIVITY: HIGH if plausible root causes imply materially
different levels of effort, LOW if they don't.
6. VERDICT: READY / SPLIT / OUT OF SCOPE.
SPLIT = separate the diagnosis into its own paid milestone.
OUT OF SCOPE = no party can deliver it; name it as excluded.
Then list every row with a VERDICT other than READY, and for each,
write the replacement text: a diagnosis criterion with an evidence
bar, plus the narrowed outcome criterion that survives.
Do not soften any criterion to make it passable. Either a named
party controls it, or it comes out.
CRITERIA:
[paste yours here]
The one thing to change before you run it: give it the real technical context of your product, especially the platforms and third parties sitting in the path. Without that, it can’t tell a cause A from a cause C, and telling those apart is the entire job.
What I’m doing this week
I’m sendinI’m sending the device-and-OS question and then hiring based on version 2. Out of thirteen candidates, two finalists remain, and both stood out by testing the physical hardware on their own—that’s a story for another time. The criteria they’ll be measured against now reflect how the product actually works, not how I wished it worked on Sunday night, and that’s what I’m happiest about. For years, I’ve had a habit of circling decisions instead of closing them, as I wrote about when we stopped changing the lid. This time, I closed it in a weekend because I stopped trying to write a better sentence and started focusing on naming the cause. Not sure yet which of the three causes I have. That’s fine. Version 2 is written so that not knowing costs me forty minutes instead of forty thousand dollars.
Related reading from PMJ
- [Stop Tuning the Prompt. Your AI Agent Needs a Contract: the argument that delegation runs on explicit contracts rather than better instructions. This post is the failure mode that appears when the contract itself is unsatisfiable.
- The Five Whys of a Flat-Tasting Kitchen: root-cause discipline applied to a spice rack. Same muscle as question two of the Controllability Test, in a much less technical room.
- [The Bottleneck Wasn’t the Vendor. It Was a Decision I Parked: nine days of blaming four co-packers for a stall that traced back to one decision of mine. What happens when you price and chase before you diagnose.
- The Subtraction Audit: A Framework for Removing Features: naming what you’re deliberately not building. The “explicitly out of scope” section is the same move applied to obligations.
- The Night We Stopped Changing the Lid: closing a decision instead of circling it, and the six weeks of speed that came out of it.
FAQ
What is an acceptance criterion, exactly?
It’s the binary condition that decides whether a deliverable is done and payable. Not a description of quality, not a goal, not a wish. Either it’s demonstrated on the real product, or it isn’t, and someone who never attended a project meeting should be able to make that call.
How is the Controllability Test different from just writing better acceptance criteria?
Standard advice optimizes for specificity and testability. The Controllability Test adds a third filter most guidance skips: whether a named party has both the access and the authority to make the criterion true. A criterion can pass the first two and fail the third, and when it does, precision makes the problem worse rather than better.
Isn’t requiring root cause before pricing just a delay tactic from the vendor’s side?
It can be, so bound it. Make diagnosis a small, fixed-price milestone with a named evidence bar and a short deadline, rather than an open-ended discovery phase. In my case, the diagnosis is three cheap tests a competent developer runs in under a day. If a vendor wants three weeks to root-cause a redirect, that answer is itself useful information.
What if I can’t tell whether a cause is controllable or not?
Then that uncertainty is exactly what belongs in the contract. Write the criterion conditionally: name the possible causes, state which ones are in scope, and say plainly what happens to scope and price under each. Ambiguity you’ve written down and priced is manageable; ambiguity you’ve papered over with a confident sentence is not.
Does this apply outside vendor contracts, on an internal team?
Yes, and the failure is quieter there because nobody sues. An internal story with an uncontrollable acceptance criterion doesn’t produce a dispute; it produces a ticket that reopens every sprint and an engineer who slowly concludes the PM isn’t listening. Run the margin pass on your own backlog, and you’ll find a few.
What’s the one thing to do this week?
The margin pass. Take your nearest set of acceptance criteria, write one named party beside each line, and circle every blank margin. Twenty minutes, and it finds the sentence that was going to cost you.
If you want the frameworks I actually use rather than the ones I talk about, grab the free Startup PM Toolkit. Five of them, all in use. And if you’ve ever written an acceptance criterion that turned out to be impossible, hit reply and tell me what it was. I read every one.
Dan