Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-11258

IPI BM Dual vips deployment failed and node stuck in NotReady state

    XMLWordPrintable

Details

    • Critical
    • No
    • SDN Sprint 235, OSDOCS Sprint 236
    • 2
    • Proposed
    • False
    • Hide

      None

      Show
      None

    Description

      Description of problem:

      IPI BM Dual vips deployment failed, see node status below:
      
      oc describe nodes openshift-qe-013.arm.eng.rdu2.redhat.comName:               openshift-qe-013.arm.eng.rdu2.redhat.comRoles:              control-plane,masterLabels:             beta.kubernetes.io/arch=amd64beta.kubernetes.io/os=linuxkubernetes.io/arch=amd64kubernetes.io/hostname=openshift-qe-013.arm.eng.rdu2.redhat.comkubernetes.io/os=linuxnode-role.kubernetes.io/control-plane=node-role.kubernetes.io/master=node.openshift.io/os_id=rhcosAnnotations:        k8s.ovn.org/node-gateway-router-lrp-ifaddr: {"ipv4":"100.64.0.4/16","ipv6":"fd98::4/64"}                    k8s.ovn.org/node-subnets: {"default":["10.129.0.0/23","fd02:0:0:2::/64"]}                    volumes.kubernetes.io/controller-managed-attach-detach: trueCreationTimestamp:  Sun, 02 Apr 2023 07:23:06 -0400Taints:             node-role.kubernetes.io/master:NoSchedulenode.kubernetes.io/not-ready:NoScheduleUnschedulable:      falseLease:  HolderIdentity:  openshift-qe-013.arm.eng.rdu2.redhat.com  AcquireTime:     <unset>  RenewTime:       Sun, 02 Apr 2023 07:47:05 -0400Conditions:  Type             Status  LastHeartbeatTime                 LastTransitionTime                Reason                       Message  ----             ------  -----------------                 ------------------                ------                       -------  MemoryPressure   False   Sun, 02 Apr 2023 07:46:04 -0400   Sun, 02 Apr 2023 07:23:06 -0400   KubeletHasSufficientMemory   kubelet has sufficient memory available  DiskPressure     False   Sun, 02 Apr 2023 07:46:04 -0400   Sun, 02 Apr 2023 07:23:06 -0400   KubeletHasNoDiskPressure     kubelet has no disk pressure  PIDPressure      False   Sun, 02 Apr 2023 07:46:04 -0400   Sun, 02 Apr 2023 07:23:06 -0400   KubeletHasSufficientPID      kubelet has sufficient PID available  Ready            False   Sun, 02 Apr 2023 07:46:04 -0400   Sun, 02 Apr 2023 07:23:06 -0400   KubeletNotReady              container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: No CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?Addresses:  InternalIP:  10.1.235.25  Hostname:    openshift-qe-013.arm.eng.rdu2.redhat.comCapacity:  cpu:                32  ephemeral-storage:  937101636Ki  hugepages-1Gi:      0  hugepages-2Mi:      0  memory:             131459212Ki  pods:               250Allocatable:  cpu:                31500m  ephemeral-storage:  862559124484  hugepages-1Gi:      0  hugepages-2Mi:      0  memory:             130308236Ki  pods:               250System Info:  Machine ID:                    ac13b999f3024ed48b6b31589ad06f63  System UUID:                   4c4c4544-004e-4e10-8044-b5c04f513533  Boot ID:                       03e5d0b8-c385-400c-aff9-8b0d6d63b0de  Kernel Version:                5.14.0-284.4.1.el9_2.x86_64  OS Image:                      Red Hat Enterprise Linux CoreOS 413.92.202303310708-0 (Plow)  Operating System:              linux  Architecture:                  amd64  Container Runtime Version:     cri-o://1.26.2-7.rhaos4.13.gitc0557b8.el9  Kubelet Version:               v1.26.2+7195e44  Kube-Proxy Version:            v1.26.2+7195e44Non-terminated Pods:             (9 in total)  Namespace                      Name                                                   CPU Requests  CPU Limits  Memory Requests  Memory Limits  Age  ---------                      ----                                                   ------------  ----------  ---------------  -------------  ---  openshift-kni-infra            coredns-openshift-qe-013.arm.eng.rdu2.redhat.com       200m (0%)     0 (0%)      400Mi (0%)       0 (0%)         23m  openshift-kni-infra            haproxy-openshift-qe-013.arm.eng.rdu2.redhat.com       200m (0%)     0 (0%)      400Mi (0%)       0 (0%)         22m  openshift-kni-infra            keepalived-openshift-qe-013.arm.eng.rdu2.redhat.com    200m (0%)     0 (0%)      400Mi (0%)       0 (0%)         23m  openshift-multus               multus-9chgx                                           10m (0%)      0 (0%)      65Mi (0%)        0 (0%)         22m  openshift-multus               multus-additional-cni-plugins-sntgg                    10m (0%)      0 (0%)      10Mi (0%)        0 (0%)         22m  openshift-multus               network-metrics-daemon-c7xkh                           20m (0%)      0 (0%)      120Mi (0%)       0 (0%)         22m  openshift-network-diagnostics  network-check-target-8d46p                             10m (0%)      0 (0%)      15Mi (0%)        0 (0%)         22m  openshift-ovn-kubernetes       ovnkube-master-scscw                                   60m (0%)      0 (0%)      1520Mi (1%)      0 (0%)         22m  openshift-ovn-kubernetes       ovnkube-node-nhrw6                                     50m (0%)      0 (0%)      660Mi (0%)       0 (0%)         22mAllocated resources:  (Total limits may be over 100 percent, i.e., overcommitted.)  Resource           Requests     Limits  --------           --------     ------  cpu                760m (2%)    0 (0%)  memory             3590Mi (2%)  0 (0%)  ephemeral-storage  0 (0%)       0 (0%)  hugepages-1Gi      0 (0%)       0 (0%)  hugepages-2Mi      0 (0%)       0 (0%)Events:  Type     Reason                Age                 From             Message  ----     ------                ----                ----             -------  Normal   RegisteredNode        23m                 node-controller  Node openshift-qe-013.arm.eng.rdu2.redhat.com event: Registered Node openshift-qe-013.arm.eng.rdu2.redhat.com in Controller  Normal   RegisteredNode        7m23s               node-controller  Node openshift-qe-013.arm.eng.rdu2.redhat.com event: Registered Node openshift-qe-013.arm.eng.rdu2.redhat.com in Controller  Warning  ErrorReconcilingNode  58s (x35 over 20m)  controlplane     [k8s.ovn.org/node-chassis-id annotation not found for node openshift-qe-013.arm.eng.rdu2.redhat.com, macAddress annotation not found for node "openshift-qe-013.arm.eng.rdu2.redhat.com" , k8s.ovn.org/l3-gateway-config annotation not found for node "openshift-qe-013.arm.eng.rdu2.redhat.com"]

      Version-Release number of selected component (if applicable):

      4.13.0-0.nightly-2023-04-01-062001

      How reproducible:

      Always

      Steps to Reproduce:

      1. Run IPI BM dual vips deployment
      2.
      3.
      

      Actual results:

      Failed

      Expected results:

      Should pass

      Additional info:

       

       

      Attachments

        Issue Links

          Activity

            People

              ffernand@redhat.com Flavio Fernandes (Inactive)
              jadha Jad Haj Yahya
              Jad Haj Yahya Jad Haj Yahya
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: