Phase 1: Validation of the training data and the ML model
Training DataTraining data is analyzed, looking for undesired issues regarding the training process, and collecting statistics to be used during monitoring.
ModelModel is analyzed for limitations, characteristics, and determining the borders of confidence regions.
Phase 2: Ongoing testing and monitoring of the production data and the ML model
Data SourcesImproved observability of the ML system are obtained by connecting to the data in it’s raw format, across all of the relevant data sources.
Input DataMonitoring of the input data in production, before and after various phases of the preprocessing. These are constantly compared to the historic data as well as to the corresponding data in the original training set.
ModelResults stored during the pre-launch analysis of the model are used to determine the severity of different issues that are detected.