Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-30065

BMH ends up in a "preparation error" state if we modify HostFirmwareSettings shortly after it's created.

    XMLWordPrintable

Details

    • Important
    • No
    • 5
    • Metal Platform 251, Metal Platform 252
    • 2
    • Rejected
    • False
    • Hide

      None

      Show
      None

    Description

      OCP: 4.15.0-rc.7

      multicluster-engine.v2.5.0-89

      Machine: Dell PowerEdge R640 with:

      BIOS Version 2.20.1
      iDRAC Firmware Version 7.00.00.171

       

      Note this issue:  https://issues.redhat.com/browse/MGMT-17005

      Because of the above issue we attempted instead of created the HFS and then creating the BMH, first to create the BMH and after the HFS gets created, we applied the following changes:

       

      apiVersion: metal3.io/v1alpha1
      kind: HostFirmwareSettings
      metadata:
        name: r640-u05
        namespace: qe2
      spec:
        settings:
          SecureBoot: Disabled
       

       

       

      The BMH creation was attempted using both drivers: redfish-virtualmedia and idrac-virtalmedia.

       

       

      apiVersion: metal3.io/v1alpha1
      kind: BareMetalHost
      metadata:
        annotations:
          bmac.agent-install.openshift.io/hostname: r640-u05
          bmac.agent-install.openshift.io/role: master
        labels:
          infraenvs.agent-install.openshift.io: qe2
        name: r640-u05
        namespace: qe2
      spec:
        automatedCleaningMode: disabled
        bmc:
          address: idrac-virtualmedia://<ip>/redfish/v1/Systems/System.Embedded.1
          credentialsName: bmc-secret1
          disableCertificateVerification: true
        bootMACAddress: 98:03:9b:61:7c:61
        online: true
        rootDeviceHints:
          deviceName: "/dev/sda"
       

      Results in:

       

       

       

      Node 3c807435-9549-4cdc-912d-c42ee8890d67 failed step {'args': {'settings': [{'name': 'SecureBoot', 'value': 'Disabled'}]}, 'interface': 'bios', 'step': 'apply_configuration', 'abortable': False, 'priority': 0}: Redfish exception occurred. Error: In system 4c4c4544-0037-3610-8053-b1c04f325732 for node 3c807435-9549-4cdc-912d-c42ee8890d67 all managers failed: set virtual boot device. Errors: ["Manager 3257324f-c0b1-5380-3610-00374c4c4544: HTTP POST https://10.6.49.58/redfish/v1/Managers/iDRAC.Embedded.1/Actions/Oem/EID_674_Manager.ImportSystemConfiguration returned code 409. Base.1.12.GeneralError: A job operation is already running. Retry the operation after the existing job is completed. Extended information: [{'Message': 'A job operation is already running. Retry the operation after the existing job is completed.', 'MessageArgs': [], 'MessageArgs@odata.count': 0, 'MessageId': 'IDRAC.2.8.RAC0679', 'RelatedProperties': [], 'RelatedProperties@odata.count': 0, 'Resolution': 'Wait until the running job is completed or delete the scheduled job and retry the operation.', 'Severity': 'Warning'}]"] 

       

       

       

       

      apiVersion: metal3.io/v1alpha1
      kind: BareMetalHost
      metadata:
        annotations:
          bmac.agent-install.openshift.io/hostname: r640-u05
          bmac.agent-install.openshift.io/role: master
        labels:
          infraenvs.agent-install.openshift.io: qe2
        name: r640-u05
        namespace: qe2
      spec:
        automatedCleaningMode: disabled
        bmc:
          address: redfish-virtualmedia://10.6.49.58/redfish/v1/Systems/System.Embedded.1
          credentialsName: bmc-secret1
          disableCertificateVerification: true
        bootMACAddress: 98:03:9b:61:7c:61
        online: true
        rootDeviceHints:
          deviceName: "/dev/sda" 

      Results: in

       

       

      oc get bmh 
      NAME       STATE       CONSUMER   ONLINE   ERROR               AGE
      r640-u05   preparing              true     preparation error   4h57m
      
      
      Status:
        Error Count:    1
        Error Message:  Redfish BIOS apply_configuration step failed. Attributes {'SecureBoot': 'Disabled'} are not updated.
        Error Type:     preparation error
       

       

       

      If we look at the console, we see an attempt to run the change in HFS via the lifecycle controller. But it doesn't work.

      The idrac shows:

      JID_091209658935	Configure: BIOS.Setup.1-1	Running (34%)
      	JID_091205108525	Configure: Import Server Configuration Profile	Completed (100%)
       

      Attachments

        Activity

          People

            janders@redhat.com Jacob Anders
            achuzhoy@redhat.com Alexander Chuzhoy
            Jad Haj Yahya Jad Haj Yahya
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated: