Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-69399

[ BareMetal IPI ] Switching provisioningNetwork from `managed` to `unmanaged` does not create a route via the provisioning IP

XMLWordPrintable

    • None
    • False
    • Hide

      None

      Show
      None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Description of problem:

      Installing a BareMetal IPI cluster using provisioningNetwork set to `Managed`, gets the cluster installed successfully and a route is created via the provisioning IP on the pod scheduling the metal3 pod. 
      
      ~~~
      Working state; Route created - 
      
      192.168.xx.0/20 dev --readacted-- proto kernel scope link src 192.168.xx.10
      
      metal3-static-ip-set container - 
      
       initContainers:
          - resources:
            ..
            name: metal3-static-ip-set
            command:
              - /set-static-ip
            env:
              - name: PROVISIONING_IP
                value: 192.168.xx.10/20
              - name: PROVISIONING_INTERFACE
      ~~~
      
      But, after switching from provisioningNetwork `Managed` to `Unmanaged`, there is no route created via the provisioning IP. Also the metal3-static-ip-set yaml has incorrect subnet set for provisioning network. The expected on is /20, but it gets /32.
      
      ~~~
      No route created with unmanaged ProvisioningNetwork :
      
      ..
      spec:
        ..
        provisioningIP: 192.168.xx.10
        provisioningNetwork: Unmanaged
        provisioningNetworkCIDR: 192.168.xx.0/20
        ..
      
        initContainers:
          - resources:
            ..
            name: metal3-static-ip-set
            command:
              - /set-static-ip
            env:
              - name: PROVISIONING_IP
                value: 192.168.xx.10
              - name: PROVISIONING_INTERFACE
                ..
      ~~~
      
      Tried creating NNCP via kubernetes nmstate operator for each master node. The metal3's metal3-static-ip-set init container crashes with: 
      
      ~~~
      + '[' -z 192.168.xx.10 ']'
      + '[' -z vlanyyy ']'
      + '[' -z vlanyyy ']'
      ++ ip -o addr show dev vlanyyy scope global
      + '[' -n '15: vlanyyy inet 192.168.xx.11/20 brd 192.168.xx.xy scope global noprefixroute vlanyyy\ valid_lft forever preferred_lft forever' ']'
      + provisioning_net_valid
      + local prov_vip_prefixlen
      ++ ipcalc --no-decorate --prefix 192.168.xx.10
      + prov_vip_prefixlen=32
      ++ ip -j -o addr show dev vlanyyy
      ++ jq -r '.[0].addr_info[] | select(.scope == "global").local'
      + for ip_addr in $(ip -j -o addr show dev "${PROVISIONING_INTERFACE}" | jq -r '.[0].addr_info[] | select(.scope == "global").local')
      + local host_ip_cidr=192.168.xx.11/32
      + ipcalc --check 192.168.xx.11/32
      ++ ipcalc --no-decorate --network 192.168.xx.11/32
      ++ ipcalc --no-decorate --network 192.168.xx.10
      + '[' 192.168.xx.11 = 192.168.xx.10 ']'
      + echo 'ERROR: provisioning interface "vlanyyy" IP addresses are all outside the provisioning subnet "192.168.xx.10"'
      ERROR: provisioning interface "vlanyyy" IP addresses are all outside the provisioning subnet "192.168.xx.10"
      + return 1
      + exit 1
      ~~~   

      Version-Release number of selected component (if applicable):

         

      How reproducible:

      Install a BareMetal IPI cluster with ProvisioningNetwork set to Managed. Switch provisioning network to unmaaged and add NNCP for master nodes for provisioning interface. 

      Steps to Reproduce:

      1. Install a BareMetal IPI cluster with ProvisioningNetwork set to Managed.    
      2. Change the provisioningNetwork to unmanaged 
      3. Add ip and routes via NNCP (kubernetes-nmstate operator) for all master nodes.   
      4. Check the the metal3 pod's metal3-static-ip-set init container.     

      Actual results:

      The metal3 pod fails with errors. 

      Expected results:

      A route should be created with the NNCP in place. 

      Additional info:

      - Discussion in slack thread - https://redhat-internal.slack.com/archives/CFP6ST0A3/p1765874702761259

              hroy@redhat.com Himanshu Roy
              rhn-support-adikulka Aditya Kulkarni
              None
              None
              Jad Haj Yahya Jad Haj Yahya
              None
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated: