Pilot-to-Production Checklist

Did the pilot test what was uncertain, or only demonstrate what was already known?

• The pilot's question was written down before it startedIt was a question that could have been answered no.
• The pilot was run against the hardest case availableFor example, the oldest machine, the poorest data, or the shift that works differently.
• The answer is recorded, including what did not workIf a pilot has no negative findings, it has not tested the real risks.

What this gate prevents: The demonstration pilot. It runs on the newest asset with clean data, and everyone is pleased, but it tests none of the risks. When a project scales up a pilot like this, it finds its real problems at ten times the cost.

What happens to the cases the system cannot handle, and who deals with them?

• The exception rate has been measured from real recordsThe figure comes from the pilot's own records, over a representative period.
• Exceptions follow a defined route, with an ownerThe route has a named person and a named queue. "It falls back to the old process" is not specific enough.
• Someone has the capacity to handle the exception workloadThat person and their manager have confirmed it, in hours per week.

What this gate prevents: The saving that never arrives. Exceptions turn out to be a quarter of the volume, and the same people as before have to handle them. The business case stops being true, even though everyone considers the build a success.

Has the system been tested against failure as well as success?

• Tested with a source unavailableFor example, the network is down, a machine is off, or an upstream system is in maintenance.
• Tested with bad input as well as missing inputA sensor that sends wrong values is more dangerous than one that sends nothing.
• Recovery is automatic, and someone has watched it recoverA document that says recovery is automatic is not enough.
• Someone is alerted when it stops, and that alert has been testedThis includes out-of-hours alerts, sent to a person who is awake and responsible for responding.

What this gate prevents: Silent failure. The system stops updating on a Friday, and the dashboard keeps showing the last known values. Nobody notices until Tuesday, because an old number looks exactly like a current one.

Who runs this system on the Monday after handover?

• A named individual owns it, and their manager agreesThis is agreed in writing, with time allocated, before go-live.
• A second person could keep it runningIf only one person understands the system, that knowledge will be lost when they leave.
• An operating budget exists, separate from the build budgetIt covers licences, support and the owner's hours, in a budget line that continues after the project closes.
• The owner has run it without help before acceptanceThis includes the routine interventions the system will need.

What this gate prevents: The most common failure on this list, and the least technical. The system works and the project closes. Eighteen months later, staff have stopped using it because nobody was funded to maintain it.

When the process changes next year, what will it take to update the system?

• You hold the source code, the configuration and the credentialsYour company holds them. If only one individual or the vendor holds them, that is a gap.
• There is a test environment, or a defined way to test safelyIf changes to a live integration cannot be tested, people stop making them, and the system can no longer adapt.
• The change process is documented, and someone has followed itThe owner has followed it at least once before acceptance.
• Another integrator could take over the systemAnswer honestly. If the answer is no, you are tied to one vendor without having decided to be.

What this gate prevents: The frozen system. It works, but nobody dares to change it, so the process grows around it. Within two years, the workarounds cost more than the system saves.

How will you know after six months whether this was worth doing?

• A baseline was captured before go-liveWithout a baseline, you cannot prove the improvement, and the next business case is harder to make.
• The measure is defined, and somebody produces it on a scheduleEach measure has a named metric, a named owner and a set interval.
• There is a review point with a decision attachedThe review decides whether to continue, extend or stop. A review that cannot change anything is only a status update.

What this gate prevents: The benefit nobody can prove. Everyone believes the system helped, but nobody can show it. The next project must then be justified from the beginning, to a finance team that has learnt to discount such claims.