Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-10625

Issue with kubernetes-nmstate-operator v4.13 on Power platform

XMLWordPrintable

    • Quality / Stability / Reliability
    • False
    • Hide

      None

      Show
      None
    • None
    • Moderate
    • No
    • None
    • Approved
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Description of problem:

      Issue with kubernetes-nmstate-operator v4.13 on Power platform.
      
      On Power cluster:kubernetes-nmstate-operator.4.13.0-202303180002 is successfully installed.
      
      # oc get csv -n openshift-nmstate
      NAME                                              DISPLAY                       VERSION               REPLACES   PHASE
      kubernetes-nmstate-operator.4.13.0-202303180002   Kubernetes NMState Operator   4.13.0-202303180002              Succeeded
      
      On listing the pods in 'openshift-nmstate' namespace, it shows 'nmstate-console-plugin' in "CrashLoopBackOff" state.
      
      
      # oc get pods -n openshift-nmstate
      NAME                                     READY   STATUS             RESTARTS         AGE
      nmstate-cert-manager-55794f58c5-fzgm8    1/1     Running            0                49m
      nmstate-console-plugin-b45b7d98f-d6dcp   0/1     CrashLoopBackOff   16 (2m33s ago)   60m
      nmstate-handler-c7446                    1/1     Running            2 (49m ago)      69m
      nmstate-handler-czqqm                    1/1     Running            2 (39m ago)      69m
      nmstate-handler-rjd9d                    1/1     Running            2 (54m ago)      69m
      nmstate-handler-xnnwd                    1/1     Running            2 (60m ago)      69m
      nmstate-handler-xv72x                    1/1     Running            2 (61m ago)      69m
      nmstate-operator-6b66b56bcd-4l8nn        1/1     Running            0                48m
      nmstate-webhook-844c5b8676-5qww9         1/1     Running            0                60m
      nmstate-webhook-844c5b8676-rzxzx         1/1     Running            0                49m
      
      
      2 issues here -
      On describing the pod, it shows exec format error and
      Image being pulled is -
      Image:          quay.io/nmstate/nmstate-console-plugin:release-1.0.0
      Image ID:       quay.io/nmstate/nmstate-console-plugin@sha256:68c2af903035761c4812614c27d3023518a719f71ba2cd7c7ab8be80c17ff469
      
      
      # oc logs nmstate-console-plugin-b45b7d98f-d6dcp -n openshift-nmstate
      exec /usr/bin/container-entrypoint: exec format error
      
      
      # oc describe pod nmstate-console-plugin-b45b7d98f-8m7xt -n openshift-nmstate
      Name:             nmstate-console-plugin-b45b7d98f-8m7xt
      Namespace:        openshift-nmstate
      Priority:         0
      Service Account:  default
      Node:             worker-1/9.114.99.201
      Start Time:       Tue, 21 Mar 2023 06:07:10 -0400
      Labels:           app=nmstate-console-plugin
                        pod-template-hash=b45b7d98f
      Annotations:      k8s.v1.cni.cncf.io/network-status:
                          [{
                              "name": "openshift-sdn",
                              "interface": "eth0",
                              "ips": [
                                  "10.131.0.21"
                              ],
                              "default": true,
                              "dns": {}
                          }]
                        openshift.io/scc: restricted-v2
                        seccomp.security.alpha.kubernetes.io/pod: runtime/default
      Status:           Running
      IP:               10.131.0.21
      IPs:
        IP:           10.131.0.21
      Controlled By:  ReplicaSet/nmstate-console-plugin-b45b7d98f
      Containers:
        nmstate-console-plugin:
          Container ID:   cri-o://f6450fda3963bb5223bbb3c89aa32e6601c851b7deb475426073d01787df1613
          Image:          quay.io/nmstate/nmstate-console-plugin:release-1.0.0
          Image ID:       quay.io/nmstate/nmstate-console-plugin@sha256:68c2af903035761c4812614c27d3023518a719f71ba2cd7c7ab8be80c17ff469
          Port:           9443/TCP
          Host Port:      0/TCP
          State:          Waiting
            Reason:       CrashLoopBackOff
          Last State:     Terminated
            Reason:       Error
            Exit Code:    1
            Started:      Tue, 21 Mar 2023 06:39:06 -0400
            Finished:     Tue, 21 Mar 2023 06:39:06 -0400
          Ready:          False
          Restart Count:  11
          Requests:
            cpu:        10m
            memory:     50Mi
          Environment:  <none>
          Mounts:
            /etc/nginx/nginx.conf from nginx-conf (ro,path="nginx.conf")
            /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-fvdjq (ro)
            /var/serving-cert from plugin-serving-cert (ro)
      Conditions:
        Type              Status
        Initialized       True
        Ready             False
        ContainersReady   False
        PodScheduled      True
      Volumes:
        plugin-serving-cert:
          Type:        Secret (a volume populated by a Secret)
          SecretName:  plugin-serving-cert
          Optional:    false
        nginx-conf:
          Type:      ConfigMap (a volume populated by a ConfigMap)
          Name:      nginx-conf
          Optional:  false
        kube-api-access-fvdjq:
          Type:                    Projected (a volume that contains injected data from multiple sources)
          TokenExpirationSeconds:  3607
          ConfigMapName:           kube-root-ca.crt
          ConfigMapOptional:       <nil>
          DownwardAPI:             true
          ConfigMapName:           openshift-service-ca.crt
          ConfigMapOptional:       <nil>
      QoS Class:                   Burstable
      Node-Selectors:              <none>
      Tolerations:                 node.kubernetes.io/memory-pressure:NoSchedule op=Exists
                                   node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                                   node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
      Events:
        Type     Reason          Age                  From               Message
        ----     ------          ----                 ----               -------
        Normal   Scheduled       36m                  default-scheduler  Successfully assigned openshift-nmstate/nmstate-console-plugin-b45b7d98f-8m7xt to worker-1
        Warning  FailedMount     36m (x5 over 36m)    kubelet            MountVolume.SetUp failed for volume "plugin-serving-cert" : secret "plugin-serving-cert" not found
        Normal   AddedInterface  36m                  multus             Add eth0 [10.131.0.21/23] from openshift-sdn
        Normal   Pulled          36m                  kubelet            Successfully pulled image "quay.io/nmstate/nmstate-console-plugin:release-1.0.0" in 352.586388ms (352.618117ms including waiting)
        Normal   Pulled          36m                  kubelet            Successfully pulled image "quay.io/nmstate/nmstate-console-plugin:release-1.0.0" in 375.187429ms (375.200967ms including waiting)
        Normal   Pulled          35m                  kubelet            Successfully pulled image "quay.io/nmstate/nmstate-console-plugin:release-1.0.0" in 315.81314ms (315.835916ms including waiting)
        Normal   Pulling         35m (x4 over 36m)    kubelet            Pulling image "quay.io/nmstate/nmstate-console-plugin:release-1.0.0"
        Normal   Pulled          35m                  kubelet            Successfully pulled image "quay.io/nmstate/nmstate-console-plugin:release-1.0.0" in 353.527434ms (353.610925ms including waiting)
        Normal   Created         35m (x4 over 36m)    kubelet            Created container nmstate-console-plugin
        Normal   Started         35m (x4 over 36m)    kubelet            Started container nmstate-console-plugin
        Warning  BackOff         88s (x158 over 36m)  kubelet            Back-off restarting failed container nmstate-console-plugin in pod nmstate-console-plugin-b45b7d98f-8m7xt_openshift-nmstate(e982af99-7a3c-4dc4-a9bf-bce8b3e03619)
      
      
      

      Version-Release number of selected component (if applicable):

      How reproducible:

       

      Steps to Reproduce:

      1. install kubernetes-nmstate-operator on OCP 4.13 cluster deployed on Power platform
      2. Check the pods in the "openshift-nmstate" namespace
      3. pod "nmstate-console-plugin" shows in "CrashLoopBackOff" state 

      Actual results:

       

      Expected results:

      "kubernetes-nmstate-operator" should install successfully on Power and all the pods in "openshift-nmstate" namespace should be in "Running" state.

      Additional info:

       

              bnemec@redhat.com Benjamin Nemec
              sbobade Swapnil Bobade (Inactive)
              None
              None
              Qiong Wang Qiong Wang
              None
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

                Created:
                Updated: