Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-1246

[4.9] Baremetal Provisioning fails on HP Gen9 systems due to eTag handling


    • Important
    • None
    • 1
    • Metal Platform 224, Metal Platform 225, Metal Platform 226, Metal Platform 227, Metal Platform 228, Metal Platform 229
    • 6
    • Rejected
    • False
    • Hide


    • Red Hat OpenShift Container Platform

      Description of problem:

      In OpenShift 4.10 (and backported to 4.9, I think 4.9.13?) we introduced some new changes to sushy in an attempt to maximise compatibility with newer hardware platforms via Redfish with better eTag management, however we accidentally broke compatibility with older hardware platforms, namely HP Gen9 via iLO4 systems, and I believe some other HPE hardware has been affected also. This has caused a regression and has resulted in customers no longer being able to manage or provision new clusters with these systems.
      We've validated that a test patch (https://review.opendev.org/c/openstack/sushy/+/856123) is able to workaround this problem with a failback to no eTag validation and should allow customers to continue to utilise their systems.

      Version-Release number of selected component (if applicable):

      4.9.13+ we believe.

      How reproducible:


      Steps to Reproduce:

      1. Attempt an openshift-install with 4.9.13+ via redfish
      2. Validate that nodes cannot be provisioned due to 412 errors (eTag)
      3. Alternatively bring a HP Gen9 system into management on a pre-installed cluster, or upgrade to 4.9.13+ on a previously working environment with such hardware.

      Actual results:

      2022-07-10 09:25:03.872 1 WARNING sushy.exceptions [req-363d0a13-99d4-44c2-8e7c-928b325f9b75 ironic-user - - - -] Error response from PATCH https://<ip-address>/redfish/v1/Systems/1/ with status code 412 has no JSON body: simplejson.errors.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
      2022-07-10 09:25:03.873 1 DEBUG sushy.exceptions [req-363d0a13-99d4-44c2-8e7c-928b325f9b75 ironic-user - - - -] HTTP response for PATCH https://<ip-address>/redfish/v1/Systems/1/: status code: 412, error: unknown error, extended: none __init__ /usr/lib/python3.6/site-packages/sushy/exceptions.py:122
      2022-07-10 09:25:03.873 1 ERROR ironic.drivers.modules.redfish.management [req-363d0a13-99d4-44c2-8e7c-928b325f9b75 ironic-user - - - -] Setting boot mode to uefi failed for node 5abf3a2c-c662-48c3-a509-613e1d47606b. Error: HTTP PATCH https://<ip-address>/redfish/v1/Systems/1/ returned code 412. unknown error Extended information: none: sushy.exceptions.HTTPError: HTTP PATCH https://<ip-address>/redfish/v1/Systems/1/ returned code 412. unknown error Extended information: none
      2022-07-10 09:25:03.874 1 INFO ironic.drivers.modules.redfish.management [req-363d0a13-99d4-44c2-8e7c-928b325f9b75 ironic-user - - - -] Attempt to set boot mode on node 5abf3a2c-c662-48c3-a509-613e1d47606b failed to set boot mode as the node does not appear to support overriding the boot mode. Possibly partial Redfish implementation?

      Expected results:

      Node can be managed just fine :-)

      Additional info:

      Potential fix: https://review.opendev.org/c/openstack/sushy/+/856123
      Similar/linked issues: https://bugzilla.redhat.com/show_bug.cgi?id=2084059, https://bugzilla.redhat.com/show_bug.cgi?id=2103710, https://issues.redhat.com/browse/OCPBUGS-602, and https://issues.redhat.com/browse/METAL-343

            janders@redhat.com Jacob Anders
            rhn-sso-roxenham Rhys Oxenham (Inactive)
            0 Vote for this issue
            7 Start watching this issue