Most pilots die of ambiguity rather than failure — nobody agreed in advance what success looked like, so the result was whatever the loudest person said it was. Here is the structure that avoids that.
Pilots rarely fail on the technology. They fail because at the end nobody can agree on whether they worked, so the decision gets made on the basis of who was most persuasive in the room. That outcome is avoidable, and avoiding it is mostly a matter of writing things down before you start.
Before anything is connected
Four documents, none longer than a page.
- The problem, in one measurable sentence. 'Deals in the mid-market segment stall between demo and proposal, and we find out late.' Not 'improve pipeline visibility'.
- The success criteria, as thresholds with numbers and a date. Whoever will make the expand-or-stop decision signs this before the pilot starts.
- The kill criteria. What would make you stop early? Writing this is uncomfortable and it is the single highest-value item on the list.
- The baseline. What the metric reads today, measured the same way it will be measured at the end.
The baseline is the one most often skipped, and skipping it guarantees the readout becomes an argument about whether things were already improving.
Scope: one problem, one segment, one owner
The failure pattern is a pilot that connects five systems, addresses four problems, spans three teams and has no single owner. It produces a large amount of activity and no clear answer, and it takes a quarter to do it.
A pilot that addresses one problem for one segment with one accountable owner produces an answer in weeks. If it works, the second problem is much easier to sponsor — you are no longer selling a concept.
A pilot that cannot fail cannot prove anything either.
The reason kill criteria matter
Six to eight weeks
Shorter than six and you are measuring novelty. Longer than eight and it stops being a pilot: attention drifts, the sponsor's priorities move, and the readout lands in a different context than the one it was commissioned in.
A workable shape: week one to connect and validate, week two to establish the baseline and tune thresholds, weeks three to seven running live with weekly checkpoints, week eight for the readout.
Instrument the counterfactual
The hardest question at any readout is 'would that have happened anyway?' You cannot answer it retrospectively, so build the answer in from the start.
- A holdout — a comparable segment or team that does not get the system, if the organisation will accept one.
- Prior-period comparison on the same cohort, same weeks, previous year, if seasonality allows.
- Action-level tracking — which specific recommendations were acted on, and what happened to those records versus the ones where the recommendation was dismissed.
The third is the most practical and the most persuasive, because it is measured inside the pilot and does not require anyone to give up a control group.
Weekly checkpoints, thirty minutes
Three questions, every week, with the same people: what did the system surface, what did we do about it, and what was wrong. The third question is the one that improves the pilot, and it needs someone senior enough in the room to make it safe to answer honestly.
The readout
Against the criteria signed at the start, in the same order, with the baseline next to the result. Then three sections: what worked, what did not, and what we now know that we did not know eight weeks ago.
A pilot that misses its criteria but produces a clear explanation of why is a good pilot. A pilot that hits its criteria and cannot explain why is a coin flip that landed well, and expanding it is a bet rather than a decision.
What happens after a good pilot
Expansion should follow the same discipline at a larger scale: next segment, criteria written first, kill criteria intact. The temptation after a successful pilot is to remove the structure because it feels like ceremony.
That structure is the reason the pilot produced an answer. It is worth keeping until the system is boring — which, for infrastructure a revenue team plans quarters around, is exactly the right destination.