Open Data Hub logo
DOCS
BLOG
COMMUNITY
GITHUB
GET STARTED

Info alert:Important Notice

Please note that more information about the previous v2 releases can be found here. You can use "Find a release" search bar to search for a particular release.

Customizing models with LAB-tuning

Enabling LAB-tuning

Overview of enabling LAB-tuning

Data scientists can use LAB-tuning in Open Data Hub to run an end-to-end workflow for customizing large language models (LLMs). The LAB (Large-scale Alignment for chatBots) method provides a more efficient alternative to traditional fine-tuning by using taxonomy-guided synthetic data generation (SDG) combined with a multi-phase training process. LAB-tuning workflows can be launched directly from the Open Data Hub dashboard using the preconfigured InstructLab pipeline, simplifying the tuning process.

LAB-tuning depends on several Open Data Hub components working together to support model customization. It uses data science pipelines to run the tuning workflow, KServe to deploy and serve the teacher and judge models, and the Training Operator to run distributed model training across GPU-enabled nodes. LAB-tuning also relies on the model registry to manage model versions, storage connections (such as S3 or OCI) to store pipeline artifacts and model outputs, and GPU hardware profiles to schedule training workloads.

To enable LAB-tuning, an OpenShift Container Platform cluster administrator must configure the required infrastructure and platform components by completing the following tasks:

  • Install the required Operators

  • Install the required components

  • Configure a storage class that supports dynamic provisioning

A cluster administrator or an Open Data Hub administrator must perform additional setup within the Open Data Hub dashboard:

  • Make LAB-tuning features visible in the dashboard

  • Create a model registry

  • Create a GPU hardware profile

Requirements for LAB-tuning

  • You have an OpenShift Container Platform cluster with cluster administrator access.

  • Your OpenShift Container Platform cluster has at least one node with 1 NVIDIA GPUs (for example, NVIDIA L40S 48 GB) for the LAB-tuning process. Multiple GPUs on the same or different nodes are required to run distributed LAB-tuning workloads.

  • To deploy the teacher model (rhelai1/modelcar-mixtral-8x7b-instruct-v0-1:1.4), the OpenShift Container Platform cluster has a worker node with one or multiple GPUs capable of running the model (for example, a2-ultragpu-4g, which contains 4 x NVIDIA A100 with 80GB vRAM) and has at least 100 GiB of available disk storage to store the model.

  • To deploy the judge model (rhelai1/modelcar-prometheus-8x7b-v2-0:1.4), the OpenShift Container Platform cluster has a worker node with one or multiple GPUs capable of running this model (for example, a2-ultragpu-2g, which contains 2 x NVIDIA A100 with 80GB vRAM) and has at least at least 100 GiB of available disk storage to store the model.

  • Your environment meets the prerequisites for installing the required Operators and using the required components, storage, model registry, and GPU hardware profiles.

Installing the required Operators for LAB-tuning

To enable your data scientists to customize models using LAB-tuning in Open Data Hub, a cluster administrator must install the following Operators in OpenShift Container Platform:

  • Open Data Hub Operator

  • Red Hat Authorino Operator, version 1.2.1 or later

  • Red Hat OpenShift Container Platform Serverless Operator

  • Red Hat OpenShift Container Platform Service Mesh 2 Operator

  • NVIDIA GPU Operator, version 24.6

    Important

    LAB-tuning requires NVIDIA GPU Operator version 24.6. If a different version is currently installed, uninstall it before proceeding. Install the NVIDIA GPU Operator version 24.6 from OperatorHub in the recommended namespace: nvidia-gpu-operator. After installation, create a ClusterPolicy resource using the default values.

  • Node Feature Discovery Operator

    Important

    Install the Node Feature Discovery Operator from OperatorHub in the recommended namespace: openshift-nfd. After installation, create a NodeFeatureDiscovery resource using the default values.
Prerequisites

  • You have logged in to OpenShift Container Platform with the cluster-admin role.

Procedure
Verification

  • In the OpenShift Container Platform web console, click Operators → Installed Operators and confirm that the Operators show the Succeeded status.

Installing the required components for LAB-tuning

To use the LAB-tuning in Open Data Hub, you must install several components.

Prerequisites

  • You have logged in to OpenShift Container Platform with the cluster-admin role.

  • You have installed the required Operators for LAB-tuning.

Procedure

  1. In the OpenShift Container Platform console, click Operators → Installed Operators.

  2. Search for the Open Data Hub Operator, and then click the Operator name to open the Operator details page.

  3. Click the Data Science Cluster tab.

  4. Click the default instance name (for example, default-dsc) to open the instance details page.

  5. Click the YAML tab to show the instance specifications.

  6. In the spec.components section, set the managementState field to Managed for the following components:

    • datasciencepipelines

    • kserve

    • kueue

    • trainingoperator

  7. In the spec.components section, include the following modelregistry component entry with the managementState field set to Managed and the registriesNamespace field set to odh-model-registries:

     modelregistry:
    managementState: Managed
    registriesNamespace: odh-model-registries

  8. Click Save.

Additional resources

Configuring a storage class for LAB-tuning

The InstructLab pipeline requires a storage class that supports dynamic provisioning with the ReadWriteMany access mode. This ensures that PersistentVolumes can be created automatically and shared across multiple pods.

Prerequisites

  • You have installed the required Operators and components for LAB-tuning.

  • You are logged in to Open Data Hub as a user with administrator privileges.

Procedure

  1. To configure a storage class, follow the steps described in Configuring persistent storage in the OpenShift Container Platform documentation.

    Tip
    		 To quickly configure a compatible storage class for a non-production environment, see Set up NFS StorageClass. The image provided in this example is for test purposes only. For production environments, you must use a production-ready storage class that supports ReadWriteMany access mode.

  2. Follow the steps described in Configuring storage class settings to ensure that the new storage class is available for use in Open Data Hub.

Making LAB-tuning and hardware profile features visible

By default, hardware profiles and LAB-tuning features are hidden from the Open Data Hub dashboard navigation menu and user interface. You must manually enable these features in your current session to access the Model catalog, Model customization, and Hardware profiles pages.

Prerequisites

  • You have installed the required Operators and components for LAB-tuning.

  • You are logged in to Open Data Hub.

Procedure

  1. In the browser tab where the Open Data Hub dashboard is open, add ?devFeatureFlags to the end of the URL. For example: https://<your-dashboard-url>?devFeatureFlags

    The following banner appears at the top of the Open Data Hub dashboard:

    Feature flags are overridden in the current session. Click here to reset back to defaults.

  2. Click the overridden link in the banner to open the Feature flags modal.

  3. In the Feature flags modal, clear the following checkboxes:

    • disableModelCatalog: Clear the checkbox to enable the Models → Model catalog page in the dashboard.

    • disableFineTuning: Clear the checkbox to enable the Models → Model customization page and the LAB-tune button on the model detail page in the model registry.

    • disableHardwareProfiles: Clear the checkbox to enable the Settings → Hardware profiles page and related UI components.

  4. Close the Feature flags modal.

Verification

The following pages appear in the Open Data Hub dashboard navigation menu:

  • Models → Model catalog

  • Models → Model customization

  • Settings → Hardware profiles

Creating a model registry for LAB-tuning

Configure a model registry in Open Data Hub so that users can register base models, launch a LAB-tuning run, and manage tuned model versions from the dashboard.

A model registry is required to register base models and manage LAB-tuned models in Open Data Hub. To start a LAB-tuning run, users must first register a base model from the model registry. The LAB-tune workflow is then launched directly from the model’s detail page. In addition, after a LAB-tuning run completes, the resulting fine-tuned model can be automatically added to the registry where users can track versions, view metadata, and deploy the model.

Prerequisites

  • You are logged in to Open Data Hub as a user with administrator privileges.

  • The model registry component is enabled for your environment.

Procedure

Creating a hardware profile for LAB-tuning

Configure a GPU hardware profile in Open Data Hub that users can select when launching a LAB-tuning run.

A GPU hardware profile is required to run LAB-tuning workloads in Open Data Hub. LAB-tuning uses distributed training that must be scheduled on nodes with GPU resources. A GPU hardware profile allows users to target specific GPU-enabled worker nodes when launching pipelines, ensuring that training workloads run on compatible hardware.

Prerequisites

  • You are logged in to Open Data Hub as a user with administrator privileges.

  • The relevant hardware is installed and you have confirmed that it is detected in your environment.

Procedure

  1. Follow the steps described in Creating a hardware profile to create a LAB-tuning hardware profile, adapting the following configurations to your specific cluster setup:

    Setting Value

    CPU

    Default: 4 Cores; Minimum allowed: 2 Cores; Maximum allowed: 4 Cores

    Memory

    Maximum allowed: Greater than 100 GiB

    Resource label

    nvidia.com/gpu

    Resource identifier

    nvidia.com/gpu

    Resource type

    Accelerator

    Node selector key (optional)

    node.kubernetes.io/instance-type

    Node selector value

    a2-ultragpu-2g

    Toleration operator (optional)

    Exists

    Toleration key

    nvidia.com/gpu

    Toleration effect

    NoSchedule

  2. Ensure that the new hardware profile is available for use with a checkmark in the Enable column.

Overview of LAB-tuning

You can use LAB-tuning in Open Data Hub to run an end-to-end workflow for customizing large language models (LLMs). The LAB (Large-scale Alignment for chatBots) method offers a more efficient alternative to traditional fine-tuning by combining taxonomy-guided synthetic data generation (SDG) with a multi-phase training process. You can run LAB-tuning workflows directly from the Open Data Hub dashboard using the preconfigured InstructLab pipeline, which simplifies the tuning process.

LAB-tuning provides the following benefits:

  • Create LLMs that reflect your domain-specific knowledge.

  • Automatically generate high-quality training data from a structured taxonomy.

  • Fine-tune models faster with a streamlined, multi-phase training process.

  • Improve performance and scalability by running training as distributed workloads across the cluster.

  • Run the entire workflow securely within Open Data Hub.

LAB-tuning workflow

LAB-tuning simplifies model customization through the preconfigured InstructLab pipeline, which follows this workflow:

  1. Your structured taxonomy defines the knowledge and skills the model should learn.

  2. A teacher model uses the taxonomy to generate synthetic training data.

  3. A judge model reviews and scores the synthetic data to ensure quality.

  4. The synthetic data is used to fine-tune a base model through a multi-phase training pipeline.

The result is a fine-tuned model that you can version, review, and deploy from the model registry.

Model customization page

You can use the Models → Model customization page in Open Data Hub to guide you through the LAB-tuning process. To view the Model customization page, see Making LAB-tuning features visible.

To use LAB-tuning, complete the following tasks:

  • Create a taxonomy in a Git repository.

  • Prepare an OCI storage location for the LAB-tuned model.

  • Create a data science project and configure its pipeline server.

  • Deploy the teacher and judge models.

Then, use LAB-tuning to customize a model by completing following tasks:

  • Register a base model from the model catalog.

  • Start a LAB-tuning run from the registered model.

  • Monitor the run’s progress.

  • Review and deploy the tuned model from the registry.

Making LAB-tuning features visible

By default, LAB-tuning features are hidden from the Open Data Hub dashboard navigation menu and user interface. You must manually enable these features in your current session to access the Model catalog and Model customization pages and use the InstructLab pipeline.

Prerequisites

  • Your cluster administrator has configured LAB-tuning in your cluster, as described in Enabling LAB-tuning.

  • You have logged in to Open Data Hub.

Procedure

  1. In the browser tab where the Open Data Hub dashboard is open, add ?devFeatureFlags to the end of the URL. For example: https://<your-dashboard-url>?devFeatureFlags

    The following banner appears at the top of the Open Data Hub dashboard:

    Feature flags are overridden in the current session. Click here to reset back to defaults.

  2. Click the overridden link in the banner to open the Feature flags modal.

  3. In the Feature flags modal, clear the following checkboxes:

    • disableModelCatalog: Clear the checkbox to enable the Models → Model catalog page in the dashboard.

    • disableFineTuning: Clear the checkbox to enable the Models → Model customization page and the LAB-tune button on the model detail page in the model registry.

  4. Close the Feature flags modal.

Verification

The following pages appear in the Open Data Hub dashboard navigation menu:

  • Models → Model catalog

  • Models → Model customization

Preparing LAB-tuning resources

Before you can customize models with LAB-tuning in Open Data Hub, complete the following tasks:

  • Create a taxonomy in a Git repository.

  • Prepare an OCI storage location for the LAB-tuned model.

  • Create a data science project and configure its pipeline server.

  • Deploy the teacher and judge models.

Creating a taxonomy

To use LAB-tuning in Open Data Hub, you need a taxonomy stored in a Git repository. Later, when you create your LAB-tuning run in Open Data Hub, you will provide the Git repository URL and any necessary authentication details.

A taxonomy is a structured set of training data that defines the skills and knowledge your model should learn. For more information, see Creating skills and knowledge YAML files in the Red Hat Enterprise Linux AI documentation.

A taxonomy tree for LAB-tuning organizes training data using a cascading directory structure. Each branch ends in a leaf node, and each leaf node is a directory with a qna.yaml file focused on a specific knowledge area or skill. Your taxonomy tree for Open Data Hub must include a root directory, a knowledge directory, and a compositional_skills directory, as shown in the following example:

taxonomy_root/
  knowledge/
    topic_1/
      qna.yaml
    topic_2/
      qna.yaml
  compositional_skills/
    skill_1/
      qna.yaml
    skill_2/
      qna.yaml

For more detailed examples, see the RHEL AI sample taxonomy and the InstructLab taxonomy.

Prerequisites

  • You have an account on a Git hosting platform, such as GitHub or GitLab.

Procedure

Create a taxonomy by using one of the following methods:

Note

Open Data Hub must be able to access and clone the Git repository during the LAB-tuning run. Have the Git repository URL and authentication details ready for when you configure your LAB-tuning run.

Using Red Hat Enterprise Linux AI

  1. Use RHEL AI to create your taxonomy. See Creating skills and knowledge YAML files in the Red Hat Enterprise Linux AI documentation.

Using the sample taxonomy repository

  1. Create a Git repository on GitHub, GitLab, or another Git hosting platform.

  2. Locally clone the RHEL AI sample taxonomy at https://github.com/RedHatOfficial/rhelai-sample-taxonomy.

  3. Customize the taxonomy by creating or editing folders and their corresponding qna.yaml files.

    For more information, see Adding knowledge to your taxonomy tree and Adding skills to your taxonomy tree in the Red Hat Enterprise Linux AI documentation.

  4. Push the updated taxonomy to your Git repository.

Manually creating a taxonomy

  1. Create a Git repository on GitHub, GitLab, or another Git hosting platform.

  2. Locally clone your new repository.

  3. Set up the folder structure. For example:

    mkdir -p taxonomy_root/knowledge/topic_1
mkdir -p taxonomy_root/compositional_skills/skill_1

  4. Add qna.yaml files to each folder, containing the questions and answers you want the model to learn.

    For more information, see Adding knowledge to your taxonomy tree and Adding skills to your taxonomy tree in the Red Hat Enterprise Linux AI documentation.

Additional resources

Preparing a storage location for the LAB-tuned model

The LAB-tuning workflow pushes the final tuned model to an OCI-compliant registry so that it can be stored, versioned, and deployed. Before starting a LAB-tuning run, create or choose a repository in an OCI-compliant registry, configure credentials with push access, and make sure the location has enough space for the model output.

When configuring the LAB-tuning run, enter the full URI for where the LAB-tuned model will be stored (for example, oci://quay.io/my-org/fine-tuned-model-name:version) and provide the credentials needed to access that location by using a new or existing connection.

Creating a project for LAB-tuning

To run LAB-tuning, create a data science project in Open Data Hub and set up its pipeline server. The data science project keeps your resources organized, and the pipeline server runs and manages each step of the LAB-tuning workflow.

Prerequisites

  • You have logged in to Open Data Hub.

  • You have the appropriate roles and permissions to create projects.

  • You have an existing S3-compatible object storage bucket and you have configured write access to your S3 bucket on your storage account.

  • If you are configuring a pipeline server for production pipeline workloads, you have an existing external MySQL or MariaDB database.

    • For an external MySQL database, your database must use at least MySQL version 5.x. However, MySQL version 8.x is recommended.

    • For an external MariaDB database, your database must use MariaDB version 10.3 or later. However, MariaDB version 10.5 is recommended.

  • Your cluster administrator has configured LAB-tuning in your cluster, as described in Enabling LAB-tuning.

Procedure

  1. From the Open Data Hub dashboard, click Data science projects.

    The Data science projects page opens.

  2. Create a data science project:

    1. Click Create project.

      The Create project form opens.

    2. For Name, enter a unique display name for your project.

    3. Optional: If you want to change the default resource name for your project, click Edit resource name.

      The resource name is how your resource is labeled in OpenShift Container Platform. Valid characters include lowercase letters, numbers, and hyphens (-). The resource name cannot exceed 30 characters, and it must start with a letter and end with a letter or number.

      Note: You cannot change the resource name after the project is created. You can edit only the display name and the description.

    4. Optional: In the Description field, provide a project description.

    5. Click Create.

      The project details page opens.

  3. Create a connection that links an object storage bucket to your data science project for saving pipeline artifacts:

    1. Click the Connections tab, and then click Create connection.

      The Create connection form opens.

    2. For Connection type select S3 compatible object storage - v1.

  4. Complete the Connection details. For more information, see Adding a connection to your data science project.

    1. Click Create.

      The new connection is displayed on the Connections tab for the project.

  5. Configure a pipeline server to run and track the InstructLab pipeline:

    1. Click the Pipelines tab, and then click Configure pipeline server.

      The Configure pipeline server form opens.

    2. In the Object storage connection section, in the Access key field next to the key icon, click the drop-down menu, and then select the connection you just created.

      The form populates with credentials for the connection.

    3. In the Database section, select one of the following options:

      • Use default database stored on your cluster: For development and testing purposes only.

      • Connect to external MySQL database: For production pipeline workloads. For more information, see Configuring a pipeline server.

    4. In the Install preconfigured pipelines section, select the InstructLab pipeline checkbox.

      This installs the InstructLab pipeline on your project, allowing you to customize your model with LAB-tuning.

      Important

      Open Data Hub automatically updates the InstructLab pipeline. To disable automatic updates, go to the Pipelines tab of your data science project, click the drop-down arrow next to Import pipeline, select Manage preconfigured pipelines, clear the InstructLab pipeline checkbox, and click Apply.

    5. Click Configure pipeline server.

Verification

  • The InstructLab pipeline appears on the Pipelines tab for your data science project.

Additional resources

Deploying teacher and judge models

LAB-tuning in Open Data Hub uses a teacher model to generate synthetic data and a judge model to evaluate the performance of the LAB-tuned model.

You can deploy the models in the same cluster as the LAB-tuning process or in a different cluster. When you configure your LAB-tuning run, you provide the URL endpoints and authentication details for the teacher and judge models.

Prerequisites

  • You have logged in to Open Data Hub.

  • Your cluster administrator has configured LAB-tuning in your cluster, as described in Enabling LAB-tuning.

  • You have enabled the LAB-tuning features in your current session, as described in Making LAB-tuning features visible.

  • You have created a data science project and configured its pipeline server.

  • For the teacher model (rhelai1/modelcar-mixtral-8x7b-instruct-v0-1:1.4), the OpenShift Container Platform cluster has a worker node with 1 or multiple GPUs capable of running the model (for example, a2-ultragpu-4g, which contains 4 x NVIDIA A100 with 80GB vRAM) and has at least 100 GiB of available disk storage to store the model.

  • For the judge model (rhelai1/modelcar-prometheus-8x7b-v2-0:1.4), the OpenShift Container Platform cluster has a worker node with 1 or multiple GPUs capable of running this model (for example, a2-ultragpu-2g, which contains 2 x NVIDIA A100 with 80GB vRAM) and has at least at least 100 GiB of available disk storage to store the model.

Procedure

  1. From the Open Data Hub dashboard, click Models → Model catalog.

    The Model catalog page opens, which shows the pre-built, curated models that are available to your organization.

  2. Under Red Hat models, select a model with the LAB teacher label.

  3. On the model details page, click Deploy model to deploy a model server that hosts the selected model and makes it available to your project.

    The Deploy model form appears.

  4. For Project, select your data science project.

  5. For Model deployment name, enter the name of the inference service to be created when the model is deployed.

  6. For Serving runtime, select vLLM NVIDIA GPU ServingRuntime for KServe.

    The vLLm Model framework is automatically selected.

  7. For Deployment mode, select one of the following options:

    • Standard: Recommended for catalog models. Uses default settings for a quick, guided deployment.

    • Advanced: Use for custom models or when you need to change the model URI, runtime settings, or storage connection.

  8. Enter the Number of model server replicas to deploy. Set this number based on expected traffic and availability needs, such as support for failover.

  9. Select a Hardware profile with the Compatible label.

  10. For Model route, select the Make deployed models available through an external route checkbox.

  11. For Token authentication, select whether to Require token authentication. Enabling this option is recommended to reduce the risk of security vulnerabilities.

  12. For Source model location, select Current URI.

  13. Optional: Under Configuration parameters, add Additional serving runtime arguments or Environment variables, if needed. The values that you configure here apply only to this model deployment.

  14. Click Deploy.

Verification

  • The Model deployments page opens and shows your deployed model in the list.

  • The In Progress icon is displayed in the Status column. When the deployment completes successfully, the icon changes to a green checkmark.

Using LAB-tuning

To customize a model with LAB-tuning in Open Data Hub, complete the following tasks:

  • Register a base model from the model catalog.

  • Start a LAB-tuning run from the registered model.

  • Monitor the run’s progress.

  • Review and deploy the fine-tuned model from the registry.

Registering a base model

To start LAB-tuning, you must first register a base model, or LAB starter model, that will be customized using the synthetic data generated from your taxonomy.

Prerequisites

  • You have logged in to Open Data Hub.

  • Your cluster administrator has configured LAB-tuning in your cluster, as described in Enabling LAB-tuning.

  • You have enabled the LAB-tuning features in your current session, as described in Making LAB-tuning features visible.

  • You have access to an available model registry in your deployment.

  • You have created your taxonomy, prepared an OCI storage location for the LAB-tuned model, created a data science project, and deployed teacher and judge models, as described in Preparing LAB-tuning resources.

Procedure

  1. From the Open Data Hub dashboard, click Models → Model catalog.

    The Model catalog page opens.

  2. Select a model with the LAB starter label.

    The details page for the model opens.

  3. Click Register model.

    The Register model page opens.

  4. In the Model details section, configure details to apply to all versions of the model:

    1. In the Model name field, enter a name for the model.

    2. Optional: In the Model description field, enter a description for the model.

  5. In the Version details section, enter details to apply to the first version of the model:

    1. In the Version name field, enter a name for the model version.

    2. Optional: In the Version description field, enter a description for the first version of the model.

    3. Optional: In the Source model format field, enter the name of the model format.

    4. Optional: In the Source model format version field, enter the version of the model format.

  6. Use the default Model location.

  7. Click Register model.

Verification

The details page for the registered base model opens.

Starting a LAB-tuning run from the registered model

After registering a base model, you can start a LAB-tuning run to create your customized large language model (LLM). When you configure the run, you will provide the following details:

  • The Git repository URL for your taxonomy.

  • The URLs where the teacher and judge models are deployed and the credentials needed to access them.

  • The hardware profile to use for data generation, training, and evaluation.

  • Any hyperparameters to apply to the tuning process.

  • The location to store the LAB-tuned model and the credentials needed to access the storage location.

Prerequisites

  • You have logged in to Open Data Hub.

  • Your cluster administrator has configured LAB-tuning in your cluster, as described in Enabling LAB-tuning.

  • You have enabled the LAB-tuning features in your current session, as described in Making LAB-tuning features visible.

  • You have access to an available model registry in your deployment.

  • You have created your taxonomy, prepared an OCI storage location for the LAB-tuned model, created a data science project, and deployed teacher and judge models, as described in Preparing LAB-tuning resources.

  • You have registered a base model with the LAB starter label from the model catalog, as described in Registering a base model.

Procedure

  1. From the Open Data Hub dashboard, click Models → Model registry.

    The Model registry page opens.

  2. Click the name of the model you registered.

    The details page for the registered model opens.

  3. On the Versions tab of the model details page, click the name of the version you want to LAB-tune.

    The details page for the version opens.

  4. Review the version details.

  5. Click LAB-tune.

    The Start a LAB-tuning run dialog opens.

  6. Select your project from the Data science project drop-down list.

  7. Click Continue to run details.

  8. The Start a LAB-tuning run page opens.

  9. Review the Project details, Pipeline details, and Base model sections for accuracy.

  10. In the Taxonomy details section, enter your Taxonomy GIT URL.

  11. If your Git repository requires authentication, select either the SSH key or Username and token method and enter the appropriate credentials.

  12. In the LAB teacher model and LAB judge model sections, configure the following settings:

    1. Select Authenticated endpoint if your model requires token authentication, otherwise select Unauthenticated endpoint.

    2. Enter the Endpoint ending with /v1. For example: https://mixtral-my-project.apps.my-cluster.com/v1

      Tip

      To find authentication details for your judge and teacher models, go to the Models tab of your data science project. For Endpoint and Model name, click Internal and external endpoint details for your model. For Token, expand the section for your model and find the Token authentication section.

    3. Enter the Model name.

    4. If authenticated, enter the Token.

  13. In the Training hardware section, configure the following settings:

    1. For Hardware profile, select a hardware profile to match the hardware requirements of your workload to available node resources.

    2. For Training nodes, enter the total number of nodes to use for the run. One node is used for the evaluation run phase.

    3. For Storage class, select a storage class that is compatible with LAB-tuning and distributed training.

  14. Optional: In the Hyperparameters section, configure advanced settings for the run.

  15. In the Fine-tuned model details section, configure settings for the fine-tuned version of the base model:

    1. For Model output storage location, select an Existing connection location to store the fine-tuned model output, or select Create connection to create a new connection.

    2. For OCI storage location field, enter the full Model URI where the LAB-tuned model will be stored. For example: oci://quay.io/my-org/fine-tuned-model-name:version

      The value of the URI is different from the connection. The connection provides access, while the URI defines the specific location.

  16. Select the Add model to registry checkbox so that you can store, share, version, and deploy the LAB-tuned model in the model registry.

    1. For Model version name, enter a name for the new LAB-tuned model version.

  17. Click Start run.

Verification

  • The Runs page opens for the pipeline version.

Monitoring your LAB-tuning run

To monitor the status of your LAB-tuning run, follow these steps:

  1. From the Open Data Hub dashboard, click Data science pipelines → Runs.

  2. Select your project from the Project list.

  3. Check the status for the run. For more information, see Viewing active pipeline runs.

  4. When the status is Succeeded, click the name of the run to view the pipeline graph and details. For more information, see Viewing the details of a pipeline version.

Reviewing and deploying your LAB-tuned model

When the pipeline run is finished, your LAB-tuned model is available in the storage location that you specified during the LAB-tune run configuration.

If you selected Add model to registry when you configured the run, the LAB-tuned model is in the model registry as a new version of the registered base model.

To view and deploy your LAB-tuned model from the model registry, follow these steps:

  1. From the Open Data Hub dashboard, click Models > Model registry.

    The Model registry page opens.

  2. Click the name of the base model you registered.

    The details page for the registered model opens.

  3. On the Versions tab of the model details page, click the name of the new version.

    The details page for the version opens.

  4. To deploy the LAB-tuned model, click Deploy.