Uploaded image for project: 'Red Hat OpenShift AI Engineering'
  1. Red Hat OpenShift AI Engineering
  2. RHOAIENG-556

ServingRuntime for Kserve model is created regardless of error

XMLWordPrintable

    • 3
    • False
    • Hide

      None

      Show
      None
    • False
    • Release Notes
    • RHOAISTRAT-93 - Model serving 1H24 enhancements
    • Hide
      ServingRuntime for KServe model is created regardless of error

      Previously, when you tried to deploy a KServe model and an error occurred, the `InferenceService` custom resource (CR) was still created and the model was shown in the *Data Science Project* page, but the status would always remain unknown. The KServe deploy process has been updated so that the ServingRuntime is not created if an error occurs.
      Show
      ServingRuntime for KServe model is created regardless of error Previously, when you tried to deploy a KServe model and an error occurred, the `InferenceService` custom resource (CR) was still created and the model was shown in the *Data Science Project* page, but the status would always remain unknown. The KServe deploy process has been updated so that the ServingRuntime is not created if an error occurs.
    • Bug Fix
    • Done
    • Dashboard - General 2.7, Dashboard - General-1
    • Moderate

      Is there an existing issue for this?

      •  I have searched the existing issues

      Deploy type

      Downstream version (eg. OpenShift AI 2.4)

      Version

      2.5 RC1

      Current Behavior

      When trying to deploy a Kserve model, some error might be shown to the user after clicking on 'Deploy'. In this specific case, the error was about a missing webhook (which I believe was triggered by trying to enabled modelmesh in the same cluster, FWIW).

      Even though it appears as if the error prevented the successful deployment of the model, you can see from the first screenshot that the headers for the deployed models table are already loaded into the DS Project page. After a few seconds, the actual InferenceService CR is created in the Project namespace, which is then shown as a row in the table - although with unknown status.

      Given the fact that there was an error during the model's deployment, the status will never be updated and keeps showing as unknown in the table - which I believe is already covered by #2172 for this use case and others.

      Expected Behavior

      No InferenceService is created if an error happened, the model is not shown in the table if it has no chance of actually becoming available

      Steps To Reproduce

      Install RHODS 2.5 RC1
      Create DS Project
      Create Data Connection
      Try to deploy a kserve/caikit model while triggering an error (e.g. enable/disable modelmesh from the DSC CR before the attempt)
      See the error

            dipgupta@redhat.com Dipanshu Gupta
            dgutride@redhat.com Dana Gutride
            Noam Manos
            Savitha T Jose Savitha T Jose
            RHOAI Dashboard
            Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

              Created:
              Updated:
              Resolved: