Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-70226

HPE iLO 6: NetworkAdapter resource ID changes unpredictably across reboots for same physical NIC

XMLWordPrintable

    • None
    • False
    • Hide

      None

      Show
      None
    • 1
    • 1
    • None
    • None
    • None
    • None
    • Rejected
    • Metal Platform 282
    • 1
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Description of problem:

      On HPE iLO 6 systems, the Redfish /redfish/v1/Chassis/1/NetworkAdapters/{Id}
      resource ID for a physical NIC changes unpredictably between different values
      (e.g., DE080000 and DE009000) across system reboots, even though the physical
      hardware (identified by SerialNumber) remains constant.
      
      This causes firmware management systems to:
      - Create duplicate database entries for the same physical NIC
      - Lose track of firmware versions as IDs change
      - Be unable to correlate firmware update requests to physical hardware
      
      The SerialNumber, Model, and PartNumber fields remain stable, confirming it's
      the same physical hardware, but the Redfish resource path changes.
      

      Version-Release number of selected component (if applicable):

      Hardware: HPE ProLiant DL110 Gen11
      BMC: HPE iLO 6 v1.68, v1.70 (issue observed on both versions)
      NIC: Intel E810-XXVDA4T
      Redfish Version: 1.16.0
      

      How reproducible:

      Reproducible but non-deterministic. NetworkAdapter ID changes occur during
      reboots, but not every reboot triggers a change.
      
      Test results:
      - Session 1: ID remained stable (DE080000) across 4 reboots
      - Session 2: ID changed 3 times across 4 reboots
        - Reboot 1 (BIOS FW): No change (stayed DE080000)
        - Reboot 2 (BMC FW): Changed DE080000 → DE009000
        - Reboot 3 (NIC FW): Changed DE009000 → DE080000
        - Reboot 4 (teardown): Changed DE080000 → DE009000
      
      Independent monitoring script confirmed all three ID changes in Session 2 with
      aligned timestamps.
      

      Steps to Reproduce:

      1. Query /redfish/v1/Chassis/1/NetworkAdapters
         Note the NetworkAdapter ID in Members array (e.g., DE080000)
      
      2. Query /redfish/v1/Chassis/1/NetworkAdapters/DE080000
         Record the SerialNumber value
      
      3. Perform system operation that requires reboot:
         - BMC firmware update with host reboot, OR
         - NIC firmware update with host reboot, OR
         - Regular system reboot
      
      4. After reboot, query /redfish/v1/Chassis/1/NetworkAdapters again
      
      5. Observe that Members array shows different ID (e.g., DE009000)
      
      6. Query the new path /redfish/v1/Chassis/1/NetworkAdapters/DE009000
         Confirm SerialNumber is identical to step 2
      

      Actual results:

      NetworkAdapter resource ID changes between reboots for same physical NIC:
      
      Example from multi-component firmware update session:
      - Initial state:    /redfish/v1/Chassis/1/NetworkAdapters/DE080000
      - After BMC reboot: /redfish/v1/Chassis/1/NetworkAdapters/DE009000 (changed)
      - After NIC reboot: /redfish/v1/Chassis/1/NetworkAdapters/DE080000 (changed back)
      - After teardown:   /redfish/v1/Chassis/1/NetworkAdapters/DE009000 (changed again)
      
      All queries confirmed identical SerialNumber (value unchanged)
      Three ID changes observed in a single update session.
      
      Changes occur during reboot process, not while system is running.
      No pattern detected correlating to firmware version, reboot type, or system state.
      

      Expected results:

      The NetworkAdapter resource ID should remain stable across reboots for a given
      physical NIC. If the physical hardware location/slot doesn't change, the Redfish
      resource path should not change.
      
      Industry comparison: Dell iDRAC uses slot-based stable IDs like "NIC.Slot.3"
      which persist across all reboots and firmware updates.
      
      Stable ID enables:
      - Reliable firmware tracking across update sessions
      - Accurate inventory management
      - Correlation between NetworkAdapter and FirmwareInventory resources
      - Infrastructure-as-Code automation referencing specific NICs
      

      Additional info:

      WORKAROUND:
      Use the SerialNumber field instead of Id for stable NIC identification:
        {
          "Id": "DE080000",                    // UNSTABLE - changes across reboots
          "SerialNumber": "<serial-number>",   // STABLE - use this for tracking
          "Model": "Intel(R) Ethernet Network Adapter E810-XXVDA4T",
          "PartNumber": "<part-number>"
        }
      
      Firmware management systems should:
      1. Use SerialNumber as primary key for NIC identification
      2. Store current Id for queries but don't rely on it for correlation
      3. Handle Id changes gracefully by re-discovering NICs by SerialNumber
      
      SUGGESTED FIX OPTIONS:
      Option 1 (Preferred): Make NetworkAdapter IDs stable based on physical slot
        - Example: Use slot-based naming like "Slot1", "PCIe1", "Adapter1"
        - Mirrors Dell iDRAC approach: "NIC.Slot.3"
      
      Option 2: Include SerialNumber in the ID
        - Example: "SN<serial>" or "E810-<serial>"
        - Guarantees stability but may have compatibility concerns
      
      Option 3: Add StableId or Location field to NetworkAdapter schema
        - Preserve current Id behavior for backwards compatibility
        - Add new stable identifier field for management applications
      
      ROOT CAUSE (SUSPECTED):
      NetworkAdapter ID appears dynamically generated during BMC/system initialization,
      possibly based on PCIe enumeration order which may vary. ID is not derived from
      stable hardware identifiers (slot position, SerialNumber, MAC address).
      
      IMPACT SCOPE:
      This issue does NOT affect:
      - Network connectivity or NIC functionality
      - Firmware update success (updates work regardless of ID changes)
      - SerialNumber, Model, PartNumber stability
      
      The issue only affects Redfish resource path stability for management/automation.
      
      SUPPORTING EVIDENCE:
      - Detailed timeline document with log analysis available upon request
      - Independent monitoring script confirmed all ID changes with aligned timestamps
      - Two complete firmware update sessions showing contrasting behavior
      

              imelofer Iury Gregory Melo Ferreira
              dpenney1@redhat.com Don Penney
              None
              None
              Jad Haj Yahya Jad Haj Yahya
              None
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Created:
                Updated: