Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-10601

[OCP 4.13] Telco HW-Offload performance degradation

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Critical Critical
    • None
    • 4.13
    • Node Tuning Operator
    • None

      Description of problem:

      Our telco-tuned dut pods are not starting in 4.13 with these definitions:
      irq-load-balancing.crio.io: disable
      cpu-load-balancing.crio.io: disable
      
      
      31s         Normal    Scheduled        pod/hw-offload-testpmd   Successfully assigned default/hw-offload-testpmd to ostest-zh6lq-worker-0
      31s         Normal    AddedInterface   pod/hw-offload-testpmd   Add eth0 [10.131.0.44/23] from ovn-kubernetes
      21s         Normal    Pulled           pod/hw-offload-testpmd   Container image "registry.redhat.io/openshift4/dpdk-base-rhel8:v4.10.0-5" already present on machine
      21s         Normal    Created          pod/hw-offload-testpmd   Created container hw-offload-testpmd
      11s         Warning   Failed           pod/hw-offload-testpmd   Error: failed to run pre-start hook for container "hw-offload-testpmd": set CPU load balancing: timed out waiting for the condition
      
      Removal of these components will result in a pod starting, but the performance will be significantly degraded.

      Actual results:

      Actual results:
      Per-port with packet loss - 0.011 
      Total Mpps - 0.022

      Expected results:

      Expected results:
      Per-port with 0 packet loss - 13.70
      Total Mpps - 27.4

      Additional info:

      (shiftstack) [cloud-user@installer-host ~]$ oc get clusterversion
      NAME      VERSION                              AVAILABLE   PROGRESSING   SINCE   STATUS
      version   4.13.0-0.nightly-2023-03-14-053612   True        False         5h19m   Cluster version is 4.13.0-0.nightly-2023-03-14-053612
      
      core@ostest-zh6lq-worker-1 ~]$ cat /etc/os-release
      NAME="CentOS Stream CoreOS"
      ID="rhcos"
      ID_LIKE="rhel fedora"
      VERSION="413.92.202303061740-0"
      VERSION_ID="4.13"
      VARIANT="CoreOS"
      VARIANT_ID=coreos
      PLATFORM_ID="platform:el9"
      PRETTY_NAME="CentOS Stream CoreOS 413.92.202303061740-0 (Plow)"
      ANSI_COLOR="0;31"
      CPE_NAME="cpe:/o:centos:centos:9coreos"
      HOME_URL="https://centos.org/"
      DOCUMENTATION_URL="https://docs.openshift.com/container-platform/4.13/"
      BUG_REPORT_URL="https://bugzilla.redhat.com/"
      REDHAT_BUGZILLA_PRODUCT="OpenShift Container Platform"
      REDHAT_BUGZILLA_PRODUCT_VERSION="4.13"
      REDHAT_SUPPORT_PRODUCT="OpenShift Container Platform"
      REDHAT_SUPPORT_PRODUCT_VERSION="4.13"
      OPENSHIFT_VERSION="4.13"
      RHEL_VERSION="9"
      OSTREE_VERSION="413.92.202303061740-0"

            yquinn@redhat.com Yanir Quinn
            zgreenbe@redhat.com Ziv Greenberg
            Mallapadi Niranjan Mallapadi Niranjan
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

              Created:
              Updated:
              Resolved: