Uploaded image for project: 'OpenShift Service Mesh'
  1. OpenShift Service Mesh
  2. OSSM-12585

Multiple InferencePools on same Gateway - ext_proc lost for all but first

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Critical Critical
    • None
    • OSSM 3.1.0
    • RHOAI
    • None
    • False
    • Hide

      None

      Show
      None
    • False
    • Customer Facing

      We are seeing an issue where smart load-balancing doesn't work for OpenShift AI and llm-d when multiple models (1 HTTPRoute each) share a Gateway on OCP 4.20 (and maybe affects OCP 4.21 too).

      This is caused by a bug in InferencePool support in Istio.
      This is already fixed upstream in Istio 1.29: commit.

      Can we expedite a backport of this commit to previous OSSM and OCP versions?

              aknutsen@redhat.com Aslak Knutsen
              pdipilat@redhat.com Pierangelo Di Pilato
              Ofer Aharon Blaut, Rob Cernich
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Created:
                Updated: