Monitoring your AI systems

+ To configure monitoring for this platform, see Configuring monitoring for the model serving platform.

Check the status of the trustyai-service-operator pod:

Use the Open Data Hub dashboard or the OpenShift CLI (oc) to install an instance of the TrustyAI service.

When you send inferences to your KServe Raw model, TrustyAI acknowledges the data capture in the output logs.

When running curl commands, pass the token to the TrustyAI service using the Authorization header. For example:

Follow these steps to view cluster metrics and verify that TrustyAI is receiving data.

A "Feature and output name mapping successfully applied" message is displayed.

For information about the specific bias metrics, see Using bias metrics.

For the complete list of TrustyAI metrics, see TrustyAI service API.

You can create a bias metric for a model by using the Open Data Hub dashboard or by using the OpenShift CLI (oc).

You can use the following bias metrics in Open Data Hub:

Ensuring that your machine learning models are fair and unbiased is essential for building trust with your users. Although you can assess fairness during model training, it is only in deployment that your models use real-world data. Even if your models are unbiased on training data, they can exhibit serious biases in real-world scenarios. Therefore, it is crucial to monitor your models for fairness during their real-world deployment.

In this tutorial, you learn how to monitor models for bias. You will use two example models to complete the following tasks:

To set up your environment for this tutorial, complete the following tasks:

To deploy the models for this tutorial, run the following commands from the directory containing the downloaded tutorial files (odh-trustyai-demos-main).

Pass the training data through the models.

You can apply name mappings to your inputs and outputs for more meaningful field names by sending a POST request to the /info/names endpoint.

For this tutorial, run the following command from the directory containing the downloaded tutorial files (odh-trustyai-demos-main):

For general steps, see Labeling data fields.

To understand the payload structure, see the odh-trustyai-demos-main/2-BiasMonitoring/scripts/apply_name_mapping.sh file in your downloaded tutorial files.

Compute the model’s cumulative fairness up to this point.

The payload structure is as follows:

Confirm that you see outputs similar to the following examples:

The specificDefinition field is important in understanding the real-world interpretation of these metric values; you can see that both model Alpha and Beta are fair over the Is Male-Identifying field, with the two groups' rates of positive outcomes only differing by -0.3% for model Alpha and 2.8% for model Beta.

After you confirm that the models are fair over the training data, you want to ensure that they remain fair over real-world inference data. To monitor their fairness, you can schedule a metric request to compute at recurring intervals throughout deployment by passing the same payloads to the /metrics/group/fairness/spd/request endpoint.

These commands return the IDs of the created requests. Later, you can use these IDs to delete the scheduled requests.

You can monitor the average values of various data fields over time to see the average ratio of loan-payback to loan-default predictions and the average ratio of male-identifying to non-male-identifying applicants. To monitor the average values, you create an identity metric request by sending a POST request to the /metrics/identity/request endpoint.

The payload structure is as follows:

Now that you have scheduled your fairness and identify metric requests, you can simulate sending some "real world" data through your models to see if they remain fair.

For information about the specific data drift metrics, see Using drift metrics.

For the complete list of TrustyAI metrics, see TrustyAI service API.

Use GET /metrics/{{metric}}/requests to list all currently active requests for the metric and verify the metric that you deleted is not shown. For example:

You can use the following data drift metrics in Open Data Hub:

The XGBoost model was created for the purpose of this demonstration and predicts credit card approval based on the following features: age, credit score, years of education, and years in employment.

When the model is deployed and the data that you upload is formatted, use the mean shift metric to monitor for data drift. This metric is useful for ensuring that a model remains accurate and reliable in a production environment.

Mean shift compares a numeric test dataset against a numeric training dataset. It produces a p-value that measures the probability the test data has originated from the same numeric distribution as the training data. A p-value less than 0.05 indicates a statistically significant drift between the two datasets. A p-value equal to or greater than 0.05 indicates no statistically significant evidence of drift.

For more information, see Using explainers.

For more information, see Using explainers.

You can use the following explainers in Open Data Hub:

LIME

Local Interpretable Model-agnostic Explanations (LIME) ^[1] is a saliency explanation method. LIME aims to explain a prediction 𝑝 = (𝑥, 𝑦) (an input-output pair) generated by a black-box model 𝑓 : ℝ^𝑑 → ℝ. The explanations come in the form of a "saliency" 𝑤_𝑖 attached to each feature 𝑥_𝑖 in the prediction 𝑥. LIME generates a local explanation ξ(𝑥) according to the following model:

In the original paper, 𝐺 is the class of linear models and 𝜋_𝑥 is an exponential kernel on a distance function 𝐷 (for example, cosine distance). LIME converts samples 𝑥_𝑖 from the original domain into interpretable samples as binary vectors 𝑥′_𝑖 ∈ 0,1. An encoded data set 𝐸 is built by taking nonzero elements of 𝑥′_𝑖 , recovering the original representation 𝑧 ∈ ℝ^𝑑 and then computing 𝑓(𝑧). A weighted linear model 𝑔 (with weights provided via 𝜋_𝑥) is then trained on the generated sparse data set 𝐸 and the model weights 𝑤 are used as feature weights for the final explanation ξ(𝑥).

SHAP

SHapley Additive exPlanations (SHAP), ^[2] seeks to unify several common explanation methods, notably LIME ^[1] and DeepLIFT, ^[3] under a common umbrella of additive feature attributions. These methods explain how an input 𝑥 = [𝑥₁, 𝑥₂, …​, 𝑥_𝑀] affects the output of some model 𝑓 by transforming 𝑥 ∈ ℝ^𝑀 into simplified inputs 𝑧′ ∈ 0, 1^𝑀 , such that 𝑧′_𝑖 indicates the inclusion or exclusion of feature 𝑖. The simplified inputs are then passed to an explanatory model 𝑔 that takes the following form:

In that form, each value 𝛷_𝑖 marks the contribution that feature 𝑖 had on the output model (called the attribution), 𝛷₀ marks the null output of the model; the model output when every feature is excluded. Therefore, this presents an easily interpretable explanation of the importance of each feature and a framework to permute the various input features to establish their collection contributions.

The final result of the algorithm are the Shapley values of each feature, which give an itemized "receipt" of all the contributing factors to the decision. For example, a SHAP explanation of a loan application might be as follows:

From this, the applicant can see that the biggest contributor to their denial was their home ownership status, which reduced their acceptance probability by 30 percentage points. Meanwhile, their number of children was of particular benefit, increasing their probability by 22 percentage points.