Enthusiastic engineers are cranking out quick AI/ML prototypes at great speed. This check list tries to help VPs shepherd this energy into valuable solutions for the business.

Business Value Promise: Traditional project gating and prioritizing methods work here. The only upgrade needed for traditional methods is in the computation of life time costs of a ML system (intelligence monitoring)

Project Governance Structure: Traditional project teams need to be augmented with AI/ML product management capability to define and drive the features needed to build and monitor a reliable AI/ML system.

Business Validation of Problem: Data Scientists need to defend their choices of data sources (provenance, pedigree and vintage), feature engineering choices and business validation methodology for results.

A/B Testing: Do inputs and outputs of the proposed system make business sense, how do we measure the quality of output and compare the quality to alternatives.

Reproducibility and Versioning: Has the AI/ML and data science team adopted mature practices for versioning and reproducing their work. More explicitly, are their data pipe line choices, feature engineering choices, data enrichment choices, model choices, hyper parameter tuning choices, model validation choices, training data set and validation data set choices tagged and version controlled.

Performance, Scalability and Reliability design: Have the APIs, and Mlops steps been validated for the expected PSR requirements.

Failure Plan: Is there a documented back up plan in case of failure due to intelligence drift or otherwise.

Intelligence validation and Intelligence Monitoring: Have automated test cases and intelligence validation metrics been developed for A/B testing and intelligence monitoring infrastructure.