Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-1246

[4.9] Baremetal Provisioning fails on HP Gen9 systems due to eTag handling

    XMLWordPrintable

Details

    • Important
    • Metal Platform 224, Metal Platform 225, Metal Platform 226, Metal Platform 227, Metal Platform 228
    • 1
    • Rejected
    • Hide

      None

      Show
      None
    • Red Hat OpenShift Container Platform

    Description

      Description of problem:

      In OpenShift 4.10 (and backported to 4.9, I think 4.9.13?) we introduced some new changes to sushy in an attempt to maximise compatibility with newer hardware platforms via Redfish with better eTag management, however we accidentally broke compatibility with older hardware platforms, namely HP Gen9 via iLO4 systems, and I believe some other HPE hardware has been affected also. This has caused a regression and has resulted in customers no longer being able to manage or provision new clusters with these systems.
      
      We've validated that a test patch (https://review.opendev.org/c/openstack/sushy/+/856123) is able to workaround this problem with a failback to no eTag validation and should allow customers to continue to utilise their systems.

      Version-Release number of selected component (if applicable):

      4.9.13+ we believe.

      How reproducible:

      Consistently

      Steps to Reproduce:

      1. Attempt an openshift-install with 4.9.13+ via redfish
      2. Validate that nodes cannot be provisioned due to 412 errors (eTag)
      3. Alternatively bring a HP Gen9 system into management on a pre-installed cluster, or upgrade to 4.9.13+ on a previously working environment with such hardware.
      

      Actual results:

      2022-07-10 09:25:03.872 1 WARNING sushy.exceptions [req-363d0a13-99d4-44c2-8e7c-928b325f9b75 ironic-user - - - -] Error response from PATCH https://<ip-address>/redfish/v1/Systems/1/ with status code 412 has no JSON body: simplejson.errors.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
      2022-07-10 09:25:03.873 1 DEBUG sushy.exceptions [req-363d0a13-99d4-44c2-8e7c-928b325f9b75 ironic-user - - - -] HTTP response for PATCH https://<ip-address>/redfish/v1/Systems/1/: status code: 412, error: unknown error, extended: none __init__ /usr/lib/python3.6/site-packages/sushy/exceptions.py:122
      2022-07-10 09:25:03.873 1 ERROR ironic.drivers.modules.redfish.management [req-363d0a13-99d4-44c2-8e7c-928b325f9b75 ironic-user - - - -] Setting boot mode to uefi failed for node 5abf3a2c-c662-48c3-a509-613e1d47606b. Error: HTTP PATCH https://<ip-address>/redfish/v1/Systems/1/ returned code 412. unknown error Extended information: none: sushy.exceptions.HTTPError: HTTP PATCH https://<ip-address>/redfish/v1/Systems/1/ returned code 412. unknown error Extended information: none
      2022-07-10 09:25:03.874 1 INFO ironic.drivers.modules.redfish.management [req-363d0a13-99d4-44c2-8e7c-928b325f9b75 ironic-user - - - -] Attempt to set boot mode on node 5abf3a2c-c662-48c3-a509-613e1d47606b failed to set boot mode as the node does not appear to support overriding the boot mode. Possibly partial Redfish implementation?

      Expected results:

      Node can be managed just fine :-)

      Additional info:

      Potential fix: https://review.opendev.org/c/openstack/sushy/+/856123
      
      Similar/linked issues: https://bugzilla.redhat.com/show_bug.cgi?id=2084059, https://bugzilla.redhat.com/show_bug.cgi?id=2103710, https://issues.redhat.com/browse/OCPBUGS-602, and https://issues.redhat.com/browse/METAL-343

      Attachments

        Issue Links

          Activity

            People

              janders@redhat.com Jacob Anders
              rhn-sso-roxenham Rhys Oxenham
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated: