Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-38142

baremetal-operator fails to resolve BMC hostname

XMLWordPrintable

    • Important
    • None
    • 1
    • Metal Platform 258, Metal Platform 259, Metal Platform 260
    • 3
    • False
    • Hide

      None

      Show
      None

      Description of problem:

      To summarize, when we meet the following three conditions, baremetal nodes cannot boot due to a hostname resolution failure.

      • HubCluster is IPv4/IPv6 Dual Stack
      • BMC of managed baremetal hosts are IPv6 single stack
      • A hostname is used instead of An IP address in "spec.bmc.address" of BMH resource
      • The hostname is resolved only to IPv6 address, not IPv4

      According to the following update, the provisioning service checks the BMC address scheme on the target and provides a matching URL for the installation media:

      When we create a BMH resource, spec.bmc.address will be an URL of the BMC.
      However, when we put a hostname instead of an IP address in the spec.bmc.address like the following example,

       

      <Example BMH definition>
      apiVersion: metal3.io/v1alpha1
      kind: BareMetalHost
        :
      spec:
        bmc:
          address: redfish://bmc.hostname.example.com:443/redfish/v1/Systems/1

      we observe the following error.

      $ oc logs -n openshift-machine-api metal3-baremetal-operator-6779dff98c-9djz7
      
      {"level":"info","ts":1721660334.9622784,"logger":"provisioner.ironic","msg":"Failed to look up the IP address for BMC hostname","host":"myenv~mybmh","hostname":"redfish://bmc.hostname.example.com:443/redfish/v1/Systems/1"} 

      Because of name resolution failure, baremetal-operator cannot determine if the BMC is IPv4 or IPv6.
      Therefore, the IP scheme is fall-back to IPv4 and ISO images are exposed via IPv4 address even if the BMC is IPv6 single stack.
      In this case, the IPv6 BMC cannot access to the ISO image on IPv4, we observe error messages like the following example, and the baremetal host cannot boot from the ISO.

      <Error message on iDRAC>
      Unable to locate the ISO or IMG image file or folder in the network share location because the file or folder path or the user credentials entered are incorrect

      The issue is caused by the following implementation.
      The following line passes `p.bmcAddress` which is whole URL, that's why the name resolution fails.
      I think we should pass `parsedURL.Hostname()` instead, which is the hostname part of the URL.

      https://github.com/metal3-io/baremetal-operator/blob/main/pkg/provisioner/ironic/ironic.go#L657
      
      		ips, err := net.LookupIP(p.bmcAddress) 

       

      Version-Release number of selected component (if applicable):
      We observe this issue on OCP 4.14 and 4.15. But I think this issue occurs even in the latest releases.

      How reproducible:

      • HubCluster is IPv4/IPv6 Dual Stack
      • BMC of managed baremetal hosts are IPv6 single stack
      • A hostname is used instead of An IP address in "spec.bmc.address" of BMH resource
      • The hostname is resolved only to IPv6 address, not IPv4

      Steps to Reproduce:

      1. Create a HubCluster with IPv4/IPv6 Dual Stack
      2. Prepare a baremetal host and BMC with IPv6 single stack
      3. Prepare a DNS server with an AAAA record entry which resolve the BMC hostname to an IPv6 address
      4. Create a BMH resource and use the hostname in the URL of "spec.bmc.address"
      5. BMC cannot boot due to IPv4/IPv6 mismatch

      Actual results:
      Name resolution fails and the baremetal host cannot boot

      Expected results:
      Name resolution works and the baremetal host can boot

      Additional info:

       

            rhn-engineering-hpokorny Honza Pokorny
            rhn-support-yatanaka Yamato Tanaka
            Dmitry Dmitriev Dmitry Dmitriev
            Steeve Goveas
            Votes:
            0 Vote for this issue
            Watchers:
            12 Start watching this issue

              Created:
              Updated: