Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-33866

nmstate-metrics pod crashed on arm cluster

XMLWordPrintable

    • Important
    • No
    • Rejected
    • False
    • Hide

      None

      Show
      None

      Description of problem:

          $ oc get pod -n openshift-nmstate
      NAME                                      READY   STATUS             RESTARTS      AGE
      nmstate-cert-manager-64877d455b-4v92z     1/1     Running            0             29m
      nmstate-console-plugin-694cb9d8d9-rchjd   1/1     Running            0             30m
      nmstate-handler-cvrqm                     1/1     Running            1             3h11m
      nmstate-handler-gns94                     1/1     Running            1             3h11m
      nmstate-handler-lrfpx                     1/1     Running            1             3h11m
      nmstate-handler-tbbpc                     1/1     Running            1             3h11m
      nmstate-handler-vlpgc                     1/1     Running            1             3h11m
      nmstate-metrics-74dd87877f-4d4q2          1/2     CrashLoopBackOff   2 (15s ago)   40s
      nmstate-operator-56bb5c84bc-xsqzq         1/1     Running            0             29m
      nmstate-webhook-595946f6bf-bq9cz          1/1     Running            0             29m
      nmstate-webhook-595946f6bf-hwmcm          1/1     Running            0             20m
      
      
      
      $ oc describe pod nmstate-metrics-74dd87877f-bfszm -n openshift-nmstate
      Name:                 nmstate-metrics-74dd87877f-bfszm
      Namespace:            openshift-nmstate
      Priority:             2000000000
      Priority Class Name:  system-cluster-critical
      Service Account:      nmstate-handler
      Node:                 master-01.ci-op-blyfkg30.ocpqe.arm.eng.rdu2.redhat.com/192.168.83.74
      Start Time:           Fri, 17 May 2024 18:35:52 +0800
      Labels:               app=kubernetes-nmstate
                            component=kubernetes-nmstate-metrics
                            name=nmstate-metrics
                            pod-template-hash=74dd87877f
                            prometheus.nmstate.io=true
      Annotations:          description: kubernetes-nmstate-metrics dump nmstate metrics
                            k8s.ovn.org/pod-networks:
                              {"default":{"ip_addresses":["10.129.0.31/23","fd01:0:0:2::1c/64"],"mac_address":"0a:58:0a:81:00:1f","gateway_ips":["10.129.0.1","fd01:0:0:...
                            k8s.v1.cni.cncf.io/network-status:
                              [{
                                  "name": "ovn-kubernetes",
                                  "interface": "eth0",
                                  "ips": [
                                      "10.129.0.31",
                                      "fd01:0:0:2::1c"
                                  ],
                                  "mac": "0a:58:0a:81:00:1f",
                                  "default": true,
                                  "dns": {}
                              }]
                            openshift.io/scc: restricted-v2
                            seccomp.security.alpha.kubernetes.io/pod: runtime/default
      Status:               Running
      SeccompProfile:       RuntimeDefault
      IP:                   10.129.0.31
      IPs:
        IP:           10.129.0.31
        IP:           fd01:0:0:2::1c
      Controlled By:  ReplicaSet/nmstate-metrics-74dd87877f
      Containers:
        nmstate-metrics:
          Container ID:  cri-o://8ef3d2a1fac135027e6c1ff6bcaa2f4dbaf67d06021423c5b737236190651b2f
          Image:         registry.redhat.io/openshift4/ose-kubernetes-nmstate-handler-rhel9@sha256:53078535516187a294686bef2cab57e36aa02c2ed39ff31a34ee72e23faa00ce
          Image ID:      registry.redhat.io/openshift4/ose-kubernetes-nmstate-handler-rhel9@sha256:53078535516187a294686bef2cab57e36aa02c2ed39ff31a34ee72e23faa00ce
          Port:          <none>
          Host Port:     <none>
          Command:
            manager
          Args:
            --zap-time-encoding=iso8601
          State:          Running
            Started:      Fri, 17 May 2024 18:35:57 +0800
          Ready:          True
          Restart Count:  0
          Requests:
            cpu:     30m
            memory:  20Mi
          Environment:
            WATCH_NAMESPACE:      
            POD_NAME:             nmstate-metrics-74dd87877f-bfszm (v1:metadata.name)
            POD_NAMESPACE:        openshift-nmstate (v1:metadata.namespace)
            RUN_METRICS_MANAGER:  
            OPERATOR_NAME:        nmstate
            ENABLE_PROFILER:      False
            PROFILER_PORT:        6060
          Mounts:
            /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-4dftd (ro)
        kube-rbac-proxy:
          Container ID:  cri-o://0482edf4094781a1aeebeb964090f1c5f4ff397fd2616f4f7bcedd9b0be29ff2
          Image:         quay.io/openshift/origin-kube-rbac-proxy:4.10.0
          Image ID:      quay.io/openshift/origin-kube-rbac-proxy@sha256:baedb268ac66456018fb30af395bb3d69af5fff3252ff5d549f0231b1ebb6901
          Port:          8443/TCP
          Host Port:     0/TCP
          Args:
            --logtostderr
            --secure-listen-address=:8443
            --upstream=http://127.0.0.1:8089
          State:      Terminated
            Reason:   Error
            Message:  exec /usr/bin/kube-rbac-proxy: exec format error
      
            Exit Code:  1
            Started:    Fri, 17 May 2024 18:57:07 +0800
            Finished:   Fri, 17 May 2024 18:57:07 +0800
          Last State:   Terminated
            Reason:     Error
            Message:    exec /usr/bin/kube-rbac-proxy: exec format error
      
            Exit Code:    1
            Started:      Fri, 17 May 2024 18:52:05 +0800
            Finished:     Fri, 17 May 2024 18:52:05 +0800
          Ready:          False
          Restart Count:  9
          Requests:
            cpu:        10m
            memory:     20Mi
          Environment:  <none>

      Version-Release number of selected component (if applicable):

          4.16.0-0.nightly-multi-2024-05-16-165918
       dualstack cluster

      How reproducible:

       

      Steps to Reproduce:

          1. setup dualstack cluster and setup nmstate operator
          2. oc get pod -n openshift-nmstate
          3.
          

      Actual results:

          mstate-metrics-74dd87877f-4d4q2          1/2     CrashLoopBackOff   2 (15s ago)   40s

      Expected results:

          

      Additional info:

          

              mkowalsk@redhat.com Mat Kowalski
              zzhao1@redhat.com Zhanqi Zhao
              Qiong Wang Qiong Wang
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Created:
                Updated: