Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-60664

[4.19] bootimage needs a refresh for linux-firmware updates required for GNR-D hardware

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Critical Critical
    • None
    • 4.19
    • RHCOS
    • Quality / Stability / Reliability
    • False
    • Hide

      None

      Show
      None
    • 1
    • Moderate
    • None
    • x86_64
    • None
    • None
    • Ready to Pick
    • 1
    • In Progress
    • Release Note Not Required
    • None
    • None
    • None
    • None
    • None

      Description of problem:

      We're enabling some early Intel 6th Generation Xeon hardware from partners called GNR-D that requires an update to the RHCOS boot images.  The presently running versions for 4.19: https://mirror.openshift.com/pub/openshift-v4/dependencies/rhcos/4.19/latest/ do not contain fixes pushed as a part of https://issues.redhat.com/browse/RHEL-92555, and we're having provisioning trouble as a result.
      
      We would like to request that RHCOS re-spin the 4.19 images so that this hardware can be discovered properly during provisioning, picking up fixes from RHEL-92555.
      
      

       

      Problem in depth:

      Presently with the shipping images (4.19.0) we see interfaces for an onboard NIC (E825C) come up in safe mode (as a consequence of not having the proper DDP firmware contained in the linux-firmware package) and instantiate themselves as:
      
      eno1np0 through eno7np0
      
      Log from the host:
      Aug 15 22:27:18 api.sno132.outbound.borball.bos2.lab node-image-pull.sh[5681]: Checking out node image content
      Aug 15 22:27:19 api.sno132.outbound.borball.bos2.lab systemd[1]: Finished Node Image Pull.
      Aug 15 22:27:19 api.sno132.outbound.borball.bos2.lab systemd[1]: Starting Node Image Overlay...
      Aug 15 22:27:19 api.sno132.outbound.borball.bos2.lab node-image-overlay.sh[5953]: Overlaying node image content
      
      The linux-firmware package is seemingly upgraded (per logs):
      Fri Aug 15 22:27:19 UTC 2025
      linux-firmware-whence-20250415-146.5.el9_5.noarch
      linux-firmware-20250415-146.5.el9_5.noarch
      Fri Aug 15 22:27:21 UTC 2025
      linux-firmware-whence-20250604-151.2.el9_6.noarch
      linux-firmware-20250604-151.2.el9_6.noarch
      
      However, this is insufficient. We need to have the proper firmware built as a part of the initrd / initramfs, so that early initialization and loading of the DDP firmware for these nics after the bootloader runs ensures that they are discovered properly in the bootimage and in subsequent boots. The proper interfaces should be:
      
      eno1 through eno7
      
      Otherwise, incorrect network configuration results and the node is offline on the network (both ipv4 and ipv6) after the first reboot.

      Version-Release number of selected component (if applicable):

      4.19.0    

      How reproducible:

      Every time on this hardware   

      Steps to Reproduce:

          1.Deploy OpenShift 4.19.5+ on this hardware using the Agent Based Installer or Red Hat ZTP through the assisted service / MCE / ACM using nmstate for the network configuration and 4.19.0 boot images from mirror.openshift.com
          2. Observe the server's first boot that the boot image / discovery ISO  provision, however it doesn't load proper DDP firmware and the E825 NIC comes up in "safe mode" for its interfaces (eno1np0 through eno7np0), which is undesirable
          3. The node reboots after successful provisioning and comes up with the proper DDP firmware (linux-firmware as of 4.19.5 is fixed with the proper firmware), however networkmanager and sysfs recognizes different interfaces (eno1 through eno7) and the node cannot be accessed over the network
          

      Actual results:

      Node comes up and is inaccessible over the network    

      Expected results:

      Node is provisioned with the same interface in the bootimage / discovery ISO and the final deployment (eno1 through eno7).    

      Additional info:

          

              aaradhak@redhat.com Aashish Radhakrishnan
              dcain@redhat.com Dave Cain
              None
              Dave Cain, XIAOBO ZHAI
              Michael Nguyen Michael Nguyen
              None
              Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

                Created:
                Updated: