Decide how an AI pilot ends before it starts

Ask a team how their AI pilot went and the common answer is not “it worked” or “it failed.” It is “it’s still going.” The results were promising, a few people use it, and someone is going to look at the numbers once things calm down.

That pilot is not a success, and it is not a cheap option either. It is a production system with no owner, no budget line, and a small group of users who now depend on it. The fix is one page written before the work starts.

Why pilots do not end on their own

Nothing about a pilot forces a conclusion. Stopping feels like admitting a mistake, and shipping means asking for real money, so extending it is the one choice that needs no meeting. A partial result makes this worse. Good on most cases, bad on some is exactly what “a few more weeks of tuning” sounds like an answer to.

The team also learns things along the way, which is the point of a pilot, and those lessons quietly move the target. By the end, nobody can say what the pilot set out to prove, because the goal has been rewritten a little every week to match what the system does well.

Write the ending down first

Our two-week assessment sets the accuracy bar a candidate has to clear. A bar tells you what the result was. It does not tell you what happens next, so we add a page that does:

• The end date. A real one on the calendar, with the decision meeting already booked. The pilot stops on that date whether or not everyone feels ready.
• The person who decides. One name, with the budget to act on the answer. A committee can always ask for more data, and it usually will.
• What each outcome triggers. Clear the bar and the run cost gets funded, an owner is named, and the launch goes on the schedule. Miss it and the system comes down on a known date, and users are told what replaces it.
• What would stop it early. The failure that ends the pilot on the day it happens, without waiting for the meeting. If the pilot touches customers or anything regulated, this line matters most.
• Who pays afterward. If nobody will name the budget for year two now, a good result will not produce one later. It is better to learn that before the build than after it.

All of this goes in front of the sponsor before anyone writes code. A sponsor who will not sign it has told you something useful about how much the result matters to them.

Three outcomes, and they are all fine

A pilot should end in one of three ways. Ship it, and the work moves to production with an owner and a run cost. Stop it, and the team writes down what it learned. Change the scope, and that counts as a new pilot, with a new page and a new date, not an extension of the old one.

The third outcome is where discipline slips, so make it cost something. Having to write the new page and defend it to the sponsor is usually enough to separate a real discovery from a reluctance to stop.

A stopped pilot is also a good result, and it should be reported as one. A team that finds out cheaply that an idea does not clear the bar has done its job. If the only acceptable outcome is shipping, you were not running a pilot. You were running a slow launch with no launch plan, and people will shade their results to keep it alive.

Build it so the ending is real

We build pilots in production shape, one thing end to end with results you can measure, because a demo can only ever answer a demo’s questions. The risk of that approach is that a pilot in production shape is easy to leave running. The written ending is what keeps it honest. If the answer is ship, the keep-or-rebuild question still has to be answered on purpose. If the answer is stop, someone actually turns it off.

If you have a pilot running now with no end date, you can still do this. Put a date on it four weeks out, name the person who decides, and write down what each answer means. Then hold the meeting. That is the middle step of how we run assess, pilot, ship, and it is the first thing we would ask about any pilot that has been promising for a while.