Executive summary
4
areas that assurance must cover before go-live
L4
the Accountability layer in the framework we publish
Key conclusions
- A one-time sign-off shows that a system worked on the day it was tested. Assurance needs a standing control that shows the system still works after handover.
- Adding assurance later means reconstructing decisions from logs that were never designed for that purpose. By month nine, the evidence usually no longer exists.
- ISO/IEC 42001 governs the management system around an AI system. It gives no guarantee about any model's output, and a proposal that treats the two as the same overstates its credentials.
AI is going into production faster than most organisations can build governance for it. The risk sits in the gap between deployment speed and governance maturity, and closing that gap is engineering work.
The key difference is between a one-time sign-off and a standing control. Conventional software is tested once against a specification. If the code does not change, the software behaves the same way in month thirteen as it did on day one. A model offers no such guarantee. It behaves probabilistically, and its behaviour drifts in production even when the code is unchanged.
Assurance must cover four areas before the system goes live
None of these four can be added convincingly once the system is in live use:
- What the system may decide on its own, and the point at which it must refer a case to a person.
- What is logged, in how much detail, and whether the log is enough to reconstruct a decision months later.
- Who reviews its decisions, how often, and what threshold triggers an intervention.
- How it is withdrawn: the rollback path, who authorises a rollback and what runs in the meantime.
Design assurance in before commissioning, because the evidence cannot be recreated later
Adding assurance after go-live means reconstructing decisions from logs that were never designed for reconstruction. The evidence needed to prove that the system behaved correctly in month three usually no longer exists in month nine, because nobody specified that it should be kept.
This is the L4 Accountability layer of the framework we publish. It belongs in the preliminary engineering design, before commissioning. Our practice is aligned to ISO/IEC 42001:2023, and we use the Singapore IMDA framework and the NIST AI Risk Management Framework as reference points. ISO/IEC 42001 covers the management system around the AI: governance, roles and continual improvement. It says nothing about the output of any particular model. A proposal that treats the two as the same is overstating what it offers.
