Open Data Hub logo
DOCS
BLOG
COMMUNITY
GITHUB
GET STARTED

Info alert:Important Notice

Please note that more information about the previous v2 releases can be found here. You can use "Find a release" search bar to search for a particular release.

Upgrading Open Data Hub

Overview of upgrading Open Data Hub

As a cluster administrator, you can configure either automatic or manual upgrades for the Open Data Hub Operator.

  • If you configure automatic upgrades, when a new version of the Open Data Hub Operator is available, Operator Lifecycle Manager (OLM) automatically upgrades the running instance of your Operator without human intervention.

  • If you configure manual upgrades, when a new version of the Open Data Hub Operator is available, OLM creates an update request.

    A cluster administrator must manually approve the update request to update the Operator to the new version. See Manually approving a pending Operator upgrade for more information about approving a pending Operator upgrade.

  • By default, the Open Data Hub Operator follows a sequential update process. This means that if there are several minor versions between the current version and the version that you plan to upgrade to, Operator Lifecycle Manager (OLM) upgrades the Operator to each of the minor versions before it upgrades it to the final, target version. If you configure automatic upgrades, OLM automatically upgrades the Operator to the latest available version, without human intervention. If you configure manual upgrades, a cluster administrator must manually approve each sequential update between the current version and the final, target version.

  • When you upgrade Open Data Hub, the upgrade process automatically uses the values of the previous version’s DataScienceCluster object. After the upgrade, you should inspect the default DataScienceCluster object to check and optionally update the managementState status of the components.

    Note

    New components are not automatically added to the DataScienceCluster object during upgrade. If you want to use a new component, you must manually edit the DataScienceCluster object to add the component entry.

  • For any components that you update, Open Data Hub initiates a rollout that affects all pods to use the updated image.

  • Notebook images are integrated into the image stream during the upgrade and subsequently appear in the Open Data Hub dashboard.

    Note
    		 Notebook images are constructed externally; they are prebuilt images that undergo quarterly changes and they do not change with every Open Data Hub upgrade.
Important

After upgrading from Open Data Hub 2.19 to 2.20, existing instances of the model registry component are not updated, which causes the instance pods to use older images than the ones referenced by the operator pod.

To resolve this issue, create a new instance of each existing model registry, using the same database configuration, and delete the old model registry instance. The new model registry instance contains all existing registered models and their metadata.

Upgrading Open Data Hub version 2.0 to version 2.2

You can upgrade the Open Data Hub Operator from version 2.0 or 2.1 to version 2.2 or later by using the OpenShift console.

For information about upgrading from version 1 to version 2, see Upgrading Open Data Hub version 1 to version 2.

Upgrading Open Data Hub involves the following tasks:

  1. Reviewing and understanding the requirements for upgrading Open Data Hub version 2.

  2. Installing version 2.2 or later of Open Data Hub.

Requirements for upgrading Open Data Hub version 2

When upgrading Open Data Hub version 2.0 or 2.1 to version 2.2 or later, you must complete the following tasks.

Check the components in the DataScienceCluster object

When you upgrade to version 2, the upgrade process automatically uses the values from the DataScienceCluster object in the previous version.

After the upgrade, you should inspect the 2 DataScienceCluster object and optionally update the status of any components as described in Installing Open Data Hub components.

Note

New components are not automatically added to the DataScienceCluster object during upgrade. If you want to use a new component, you must manually edit the DataScienceCluster object to add the component entry.

Note that Open Data Hub Operator versions 2.2 and later use an upgraded API version for a DataScienceCluster instance, resulting in the following differences.

Table 1. DataScienceCluster instance differences
ODH 2.1 and earlier ODH 2.2 and later

API version

v1alpha1

v1

Enable component

.spec.components.[component_name].enabled: true

.spec.components.[component_name].managementState: Managed

Disable component

.spec.components.[component_name].enabled: false

.spec.components.[component_name].managementState: Removed

Migrate data science pipelines

Previously, data science pipelines in Open Data Hub were based on KubeFlow Pipelines v1. Starting with Open Data Hub 2.10.0, data science pipelines are based on KubeFlow Pipelines v2, which uses a different workflow engine. Data science pipelines 2.0 is enabled and deployed by default in Open Data Hub.

Starting with Open Data Hub 2.16, data science pipelines 1.0 resources are no longer supported or managed by Open Data Hub. It is no longer possible to deploy, view, or edit the details of pipelines that are based on data science pipelines 1.0 from either the dashboard or the KFP API server.

Open Data Hub does not automatically migrate existing data science pipelines 1.0 instances to 2.0. Before upgrading to Open Data Hub 2.16 or later, you must manually migrate your existing data science pipelines 1.0 instances. For more information, see Migrating to data science pipelines 2.0.

Important

Data science pipelines 2.0 contains an installation of Argo Workflows. Red Hat does not support direct usage of this installation of Argo Workflows.

If you upgrade to Open Data Hub 2.10.0 or later with data science pipelines enabled and an Argo Workflows installation that is not installed by data science pipelines exists on your cluster, Open Data Hub components will not be upgraded. To complete the component upgrade, disable data science pipelines or remove the separate installation of Argo Workflows. The component upgrade will complete automatically.

Recreate model registries

When you upgrade from Open Data Hub 2.19 or earlier to Open Data Hub 2.20 or later versions, existing instances of the model registry component are not updated, which causes the instance pods to use older images than the ones referenced by the operator pod.

To resolve this issue, after upgrading, create a new instance of each existing model registry, using the same database configuration, and delete the old model registry instance. The new model registry instance contains all existing registered models and their metadata.

Update workflows interacting with OdhDashboardConfig resource

Previously, cluster administrators used the groupsConfig option in the OdhDashboardConfig resource to manage the OpenShift Container Platform groups (both administrators and non-administrators) that can access the Open Data Hub dashboard. Starting with Open Data Hub 2.17, this functionality has moved to the Auth resource. If you have workflows (such as GitOps workflows) that interact with OdhDashboardConfig, you must update them to reference the Auth resource instead.

Table 2. User management resource update
ODH 2.16 and earlier ODH 2.17 and later

apiVersion

opendatahub.io/v1alpha

services.platform.opendatahub.io/v1alpha1

kind

OdhDashboardConfig

Auth

name

odh-dashboard-config

auth

Admin groups

spec.groupsConfig.adminGroups

spec.adminGroups

User groups

spec.groupsConfig.allowedGroups

spec.allowedGroups

Update Kueue

In Open Data Hub, cluster administrators use Kueue to configure quota management for distributed workloads.

When upgrading from Open Data Hub v2.23.1 or earlier, the version of the MultiKueue Custom Resource Definitions (CRDs) changes from v1alpha1 to v1beta1.

However, if the kueue component is set to Managed, the Open Data Hub Operator does not automatically remove the v1alpha1 MultiKueue CRDs during the upgrade. The deployment of the Kueue component then becomes blocked, as indicated in the default-dsc DataScienceCluster custom resource, where the value of the kueueReady condition remains set to False.

You can resolve this problem as follows:

Note

If you created any resources based on the MultiKueue CRDs, those resources will be deleted when you delete the CRDs. If you do not want to lose your data, create a backup before deleting the CRDs.

  1. Log in to the OpenShift Console.

  2. In the Administrator perspective, click Administration → CustomResourceDefinitions.

  3. In the search field, enter multik.

  4. Update the MultiKueueCluster CRD as follows:

    1. Click the CRD name, and click the YAML tab.

    2. Ensure that the metadata:labels section includes the following entry:

      app.opendatahub.io/kueue: 'true'

    3. Click Save.

  5. Repeat the above steps to update the MultiKueueConfig CRD.

  6. Remove the MultiKueueCluster and MultiKueueConfig CRDs, by completing the following steps for each CRD:

    1. Click the Actions menu.

    2. Click Delete CustomResourceDefinition.

    3. Click Delete to confirm the deletion.

The Open Data Hub Operator starts the Kueue Controller, and Kueue then automatically creates the v1beta1 MultiKueue CRDs. In the default-dsc DataScienceCluster custom resource, the kueueReady condition changes to True. For information about how to check that the kueue-controller-manager-<pod-id> pod is Running, see Installing the distributed workloads components.

Check the status of certificate management

You can use self-signed certificates in Open Data Hub.

After you upgrade, check the management status for Certificate Authority (CA) bundles as described in Understanding how Open Data Hub handles certificates.

Installing Open Data Hub version 2

You can install Open Data Hub version 2 on OpenShift Container Platform from the OpenShift web console. For information about upgrading the Open Data Hub Operator, see Upgrading Open Data Hub.

Installing Open Data Hub involves the following tasks:

  1. Optional: Configuring custom namespaces.

  2. Installing the Open Data Hub Operator.

  3. Installing Open Data Hub components.

  4. Accessing the Open Data Hub dashboard.

Configuring custom namespaces

By default, Open Data Hub uses predefined namespaces, but you can define a custom namespace for the operator and DSCI.applicationNamespace as needed. Namespaces created by Open Data Hub typically include openshift or redhat in their name. Do not rename these system namespaces because they are required for Open Data Hub to function properly.

Prerequisites

  • You have access to an Open Data Hub cluster with cluster administrator privileges.

  • You have downloaded and installed the OpenShift command-line interface (CLI). See Installing the OpenShift CLI.

Procedure

  1. In a terminal window, if you are not already logged in to your OpenShift cluster as a cluster administrator, log in to the OpenShift CLI as shown in the following example:

    oc login <openshift_cluster_url> -u <admin_username> -p <password>

  2. Enter the following command to create the custom namespace:

    oc create namespace <custom_namespace>

  3. If you are creating a namespace for a DSCI.applicationNamespace, enter the following command to add the correct label:

    oc label namespace <application_namespace> opendatahub.io/application-namespace=true

Installing the Open Data Hub Operator version 2

Prerequisites

  • You are using OpenShift Container Platform 4.14 or later.

  • Your OpenShift cluster has a minimum of 16 CPUs and 32GB of memory across all OpenShift worker nodes.

  • You have cluster administrator privileges for your OpenShift Container Platform cluster.

  • If you are using custom namespaces, you have created and labeled them as required.

  • If you are installing Open Data Hub 2.10.0 or later with data science pipelines, ensure your cluster does not have a separate installation of Argo Workflows that was not installed by Open Data Hub.

    Important

    Data science pipelines 2.0 includes an installation of Argo Workflows. Red Hat does not support direct customer usage of this installation of Argo Workflows.

    If there is an existing installation of Argo Workflows that is not installed by data science pipelines on your cluster, data science pipelines will be disabled after you install Open Data Hub.

    To enable data science pipelines, remove the separate installation of Argo Workflows from your cluster. Data science pipelines will be enabled automatically.

    Argo Workflows resources that are created by Open Data Hub have the following labels in the OpenShift Console under Administration > CustomResourceDefinitions, in the argoproj.io group:

     labels:
    app.kubernetes.io/part-of: data-science-pipelines-operator
    app.opendatahub.io/data-science-pipelines-operator: 'true'
Procedure

  1. Log in to your OpenShift Container Platform as a user with cluster-admin privileges. If you are performing a developer installation on try.openshift.com, you can log in as the kubeadmin user.

  2. Select Operators → OperatorHub.

  3. On the OperatorHub page, in the Filter by keyword field, enter Open Data Hub Operator.

  4. Click the Open Data Hub Operator tile.

  5. If the Show community Operator window opens, read the information and then click Continue.

  6. Read the information about the Operator and then click Install.

  7. On the Install Operator page, follow these steps:

    1. For Update channel, select fast.

      Note

      Version 2 of the Open Data Hub Operator represents an alpha release, accessible only on the fast channel. Later releases will change to the rolling channel when the Operator is more stable.

    2. For Version, select the version of the Operator that you want to install.

    3. For Installation mode, leave All namespaces on the cluster (default) selected.

    4. For Installed Namespace, select the openshift-operators namespace.

    5. For Update approval, select automatic or manual updates.

      • Automatic: When a new version of the Operator is available, Operator Lifecycle Manager (OLM) automatically upgrades the running instance of your Operator.

      • Manual: When a new version of the Operator is available, OLM notifies you with an update request that you must manually approve to upgrade the running instance of your Operator.

  8. Click Install. The installation might take a few minutes.

Verification

  • Select Operators → Installed Operators to verify that the Open Data Hub Operator is listed with Succeeded status.

Next Step

  • Install Open Data Hub components.

Additional resources

Installing Open Data Hub components

You can use the OpenShift web console to install specific components of Open Data Hub on your cluster when version 2 of the Open Data Hub Operator is already installed on the cluster.

Prerequisites

  • You have installed version 2 of the Open Data Hub Operator.

  • You can log in as a user with cluster-admin privileges.

  • If you want to use the trustyai component, you must enable user workload monitoring as described in Configuring monitoring for the multi-model serving platform.

  • If you want to use the kserve, modelmesh, or modelregistry components, you must have already installed the following Operator or Operators for the component. For information about installing an Operator, see Adding Operators to a cluster.

  • If you want to use kserve, you have selected a deployment mode. For more information, see About KServe deployment modes.

Table 3. Required Operators for components
Component Required Operators Catalog

kserve

Red Hat OpenShift Serverless Operator, Red Hat OpenShift Service Mesh Operator, Red Hat Authorino Operator

Red Hat

modelmesh

Prometheus Operator

Community

modelregistry

Red Hat Authorino Operator, Red Hat OpenShift Serverless Operator, Red Hat OpenShift Service Mesh Operator

NOTE: To use the model registry feature, you must install the required Operators in a specific order. For more information, see Configuring the model registry component.

Red Hat
Procedure

  1. Log in to your OpenShift Container Platform as a user with cluster-admin privileges. If you are performing a developer installation on try.openshift.com, you can log in as the kubeadmin user.

  2. Select Operators → Installed Operators, and then click the Open Data Hub Operator.

  3. On the Operator details page, click the DSC Initialization tab, and then click Create DSCInitialization.

  4. On the Create DSCInitialization page, configure by using Form view or YAML view. For general information about the supported components, see Tiered Components.

    • Configure by using Form view:

      1. In the Name field, enter a value.

      2. In the Components section, expand each component and set the managementState to Managed or Removed.

    • Configure by using YAML view:

      1. In the spec.components section, for each component shown, set the value of the managementState field to either Managed or Removed.

  5. Click Create.

  6. Wait until the status of the DSCInitialization is Ready.

  7. Click the Data Science Cluster tab, and then click Create DataScienceCluster.

  8. On the Create DataScienceCluster page, configure the DataScienceCluster by using Form view or YAML view. For general information about the supported components, see Tiered Components.

    • Configure by using Form view:

      1. In the Name field, enter a value.

      2. In the Components section, expand each component and set the managementState to Managed or Removed.

    • Configure by using YAML view:

      1. In the spec.components section, for each component shown, set the value of the managementState field to either Managed or Removed.

  9. Click Create.

Verification

  1. Select Home → Projects, and then select the opendatahub project.

  2. On the Project details page, click the Workloads tab and confirm that the Open Data Hub core components are running. For more information, see Tiered Components.

Note: In the Open Data Hub dashboard, users can view the list of the installed Open Data Hub components, their corresponding source (upstream) components, and the versions of the installed components, as described in Viewing installed Open Data Hub components.

Next Step
Additional resources

Upgrading Open Data Hub version 1 to version 2

You can upgrade the Open Data Hub Operator from version 1 to version 2 by using the OpenShift console.

For information about upgrading from version 2.0, see Upgrading Open Data Hub version 2.0 to version 2.2.

Upgrading Open Data Hub involves the following tasks:

  1. Reviewing and understanding the requirements for upgrading Open Data Hub version 1.

  2. Upgrading the Open Data Hub Operator version 1.

  3. Installing Open Data Hub components.

Requirements for upgrading Open Data Hub version 1

This section describes the tasks that you should complete when upgrading Open Data Hub version 1 to version 2.

Note

After you install Open Data Hub 2, pipelines created with data science pipelines 1.0 continue to run, but are inaccessible from the Open Data Hub dashboard. If you are a current data science pipelines user, do not install Open Data Hub with data science pipelines 2.0 until you are ready to migrate to the new pipelines solution.

Check the components in the DataScienceCluster object

When you upgrade to version 2, the upgrade process automatically uses the values from the DataScienceCluster object in the previous version.

After the upgrade, you should inspect the 2 DataScienceCluster object and optionally update the status of any components as described in Installing Open Data Hub components.

Note

New components are not automatically added to the DataScienceCluster object during upgrade. If you want to use a new component, you must manually edit the DataScienceCluster object to add the component entry.

Recreate existing pipeline runs

When you upgrade to Open Data Hub 2, any existing pipeline runs that you created in version 1 continue to refer to the previous version’s image (as expected).

After upgrading, you must delete the pipeline runs (not the pipelines) and create new pipeline runs. The pipeline runs that you create in version 2 correctly refer to the version 2 image.

For more information about pipeline runs, see Managing pipeline runs.

Address KServe requirements

For KServe (single-model serving platform), you must meet these requirements:

  • Install dependent Operators, including the Red Hat OpenShift Serverless and Red Hat OpenShift Service Mesh Operators. For more information, see Serving large models.

  • After upgrading, you must inspect the default DataScienceCluster object and verify that the value of the managementState field for the kserve component is Managed.

  • In Open Data Hub version 1, the KServe component is a Limited Availability feature. If you enabled the kserve component and created models in version 1, then after you upgrade to version 2, you must update some Open Data Hub resources as follows:

    1. Log in to the OpenShift Container Platform console as a cluster administrator:

      $ oc login

    2. Update the DSC Initialization resource:

      $ oc patch $(oc get dsci -A -oname) --type='json' -p='[{"op": "replace", "path": "/spec/serviceMesh/managementState", "value":"Unmanaged"}]'

    3. Update the Data Science Cluster resource:

      $ oc patch $(oc get dsc -A -oname) --type='json' -p='[{"op": "replace", "path": "/spec/components/kserve/serving/managementState", "value":"Unmanaged"}]'

    4. Update the InferenceServices CRD:

      $ oc patch crd inferenceservices.serving.kserve.io --type=json -p='[{"op": "remove", "path": "/spec/conversion"}]'

    5. Optionally, restart the Operator pod.

  • If you deployed a model by using KServe in Open Data Hub version 1, when you upgrade to version 2 the model does not automatically appear in the Open Data Hub dashboard. To update the dashboard view, redeploy the model by using the Open Data Hub dashboard.

Upgrading the Open Data Hub Operator version 1

Prerequisites

  • You have installed version 1 of the Open Data Hub Operator.

  • You are using OpenShift Container Platform 4.14 or later.

  • Your OpenShift cluster has a minimum of 16 CPUs and 32GB of memory across all OpenShift worker nodes.

  • You can log in as a user with cluster-admin privileges.

Procedure

  1. Log in to the OpenShift Container Platform web console as a user with cluster-admin privileges.

  2. Select Operators → Installed Operators, and then click the 1.x version of the Open Data Hub Operator.

  3. Click the Subscription tab.

  4. Under Update channel, click the pencil icon.

  5. In the Change Subscription update channel dialog, select fast, and then click Save.

    If you configured the Open Data Hub Operator with automatic update approval, the upgrade begins. If you configured the Operator with manual update approval, perform the actions in the next step.

  6. To approve a manual update, perform these actions:

    1. Next to Upgrade status, click 1 requires approval.

    2. Click Preview InstallPlan.

    3. Review the manual install plan, and then click Approve.

Verification

  • Select Operators → Installed Operators to verify that the Open Data Hub Operator is listed with the 2.x version number and Succeeded status.

Next Steps

  • Install Open Data Hub components.

  • Access the Open Data Hub dashboard.

Additional resources

Installing Open Data Hub components

You can use the OpenShift web console to install specific components of Open Data Hub on your cluster when version 2 of the Open Data Hub Operator is already installed on the cluster.

Prerequisites

  • You have installed version 2 of the Open Data Hub Operator.

  • You can log in as a user with cluster-admin privileges.

  • If you want to use the trustyai component, you must enable user workload monitoring as described in Configuring monitoring for the multi-model serving platform.

  • If you want to use the kserve, modelmesh, or modelregistry components, you must have already installed the following Operator or Operators for the component. For information about installing an Operator, see Adding Operators to a cluster.

  • If you want to use kserve, you have selected a deployment mode. For more information, see About KServe deployment modes.

Table 4. Required Operators for components
Component Required Operators Catalog

kserve

Red Hat OpenShift Serverless Operator, Red Hat OpenShift Service Mesh Operator, Red Hat Authorino Operator

Red Hat

modelmesh

Prometheus Operator

Community

modelregistry

Red Hat Authorino Operator, Red Hat OpenShift Serverless Operator, Red Hat OpenShift Service Mesh Operator

NOTE: To use the model registry feature, you must install the required Operators in a specific order. For more information, see Configuring the model registry component.

Red Hat
Procedure

  1. Log in to your OpenShift Container Platform as a user with cluster-admin privileges. If you are performing a developer installation on try.openshift.com, you can log in as the kubeadmin user.

  2. Select Operators → Installed Operators, and then click the Open Data Hub Operator.

  3. On the Operator details page, click the DSC Initialization tab, and then click Create DSCInitialization.

  4. On the Create DSCInitialization page, configure by using Form view or YAML view. For general information about the supported components, see Tiered Components.

    • Configure by using Form view:

      1. In the Name field, enter a value.

      2. In the Components section, expand each component and set the managementState to Managed or Removed.

    • Configure by using YAML view:

      1. In the spec.components section, for each component shown, set the value of the managementState field to either Managed or Removed.

  5. Click Create.

  6. Wait until the status of the DSCInitialization is Ready.

  7. Click the Data Science Cluster tab, and then click Create DataScienceCluster.

  8. On the Create DataScienceCluster page, configure the DataScienceCluster by using Form view or YAML view. For general information about the supported components, see Tiered Components.

    • Configure by using Form view:

      1. In the Name field, enter a value.

      2. In the Components section, expand each component and set the managementState to Managed or Removed.

    • Configure by using YAML view:

      1. In the spec.components section, for each component shown, set the value of the managementState field to either Managed or Removed.

  9. Click Create.

Verification

  1. Select Home → Projects, and then select the opendatahub project.

  2. On the Project details page, click the Workloads tab and confirm that the Open Data Hub core components are running. For more information, see Tiered Components.

Note: In the Open Data Hub dashboard, users can view the list of the installed Open Data Hub components, their corresponding source (upstream) components, and the versions of the installed components, as described in Viewing installed Open Data Hub components.

Next Step
Additional resources