Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-59498

[Multus] `/usr/src/whereabouts/bin/node-slice-controller` not found

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Duplicate
    • Icon: Critical Critical
    • None
    • 4.18
    • Networking / multus
    • None
    • Quality / Stability / Reliability
    • False
    • Hide

      None

      Show
      None
    • None
    • Critical
    • Yes
    • None
    • None
    • None
    • CORENET Sprint 274
    • 1
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Description of problem:

      During the verifcation for https://issues.redhat.com/browse/OCPBUGS-57643, after whereabouts-shim enabled, whereabouts-controller pod is not Running.
      This is a regression issue. 

      Version-Release number of selected component (if applicable):

      4.18.0-0.nightly-2025-07-16-110135

      How reproducible:

      Always

      Steps to Reproduce:

      1. Enable whereabouts-shim in 4.18 cluster.
      2. whereabouts-controller pod is not Running

      Actual results:

      $ oc get clusterversionNAME      VERSION                              AVAILABLE   PROGRESSING   SINCE   STATUSversion   4.18.0-0.nightly-2025-07-16-110135   True        False         28m     Cluster version is 4.18.0-0.nightly-2025-07-16-110135
      
      $ oc get networks.operator.openshift.io cluster -o yamlapiVersion: operator.openshift.io/v1kind: Networkmetadata:  creationTimestamp: "2025-07-17T13:17:51Z"  generation: 3  name: cluster  resourceVersion: "35123"  uid: 93d42b71-acce-48d4-98e4-d0a7e643e79fspec:  additionalNetworks:  - name: whereabouts-shim    namespace: default    rawCNIConfig: |-      {       "name": "whereabouts-shim",       "cniVersion": "0.4.0",       "type": "bridge",       "ipam": {         "type": "whereabouts"       }      }    type: Raw  clusterNetwork:  - cidr: 10.128.0.0/14    hostPrefix: 23  defaultNetwork:    ovnKubernetesConfig:      egressIPConfig: {}      gatewayConfig:        ipv4: {}        ipv6: {}        routingViaHost: false      genevePort: 6081      ipsecConfig:        mode: Disabled      mtu: 1400      policyAuditConfig:        destination: "null"        maxFileSize: 50        maxLogFiles: 5        rateLimit: 20        syslogFacility: local0    type: OVNKubernetes  deployKubeProxy: false  disableMultiNetwork: false  disableNetworkDiagnostics: false  logLevel: Normal  managementState: Managed  observedConfig: null  operatorLogLevel: Normal  serviceNetwork:  - 172.30.0.0/16  unsupportedConfigOverrides: null  useMultiNetworkPolicy: falsestatus:  conditions:  - lastTransitionTime: "2025-07-17T13:17:51Z"    message: ""    reason: ""    status: "False"    type: ManagementStateDegraded  - lastTransitionTime: "2025-07-17T13:18:25Z"    message: ""    reason: ""    status: "False"    type: Degraded  - lastTransitionTime: "2025-07-17T13:17:51Z"    message: ""    reason: ""    status: "True"    type: Upgradeable  - lastTransitionTime: "2025-07-17T14:04:39Z"    message: Deployment "/openshift-multus/whereabouts-controller" is not available      (awaiting 1 nodes)    reason: Deploying    status: "True"    type: Progressing  - lastTransitionTime: "2025-07-17T13:20:24Z"    message: ""    reason: ""    status: "True"    type: Available  readyReplicas: 0  version: 4.18.0-0.nightly-2025-07-16-110135
      $ oc get pod | grep whereabouts-controllerwhereabouts-controller-6897c7f966-7bqts        0/1     CreateContainerError   0             3m10s$ oc describe pod whereabouts-controller-6897c7f966-7bqtsName:                 whereabouts-controller-6897c7f966-7bqtsNamespace:            openshift-multusPriority:             2000001000Priority Class Name:  system-node-criticalService Account:      multus-ancillary-toolsNode:                 weliang-4-18-2np9c-worker-2-5qx2z/10.242.65.5Start Time:           Thu, 17 Jul 2025 10:09:46 -0400Labels:               app=whereabouts-controller                      pod-template-hash=6897c7f966Annotations:          k8s.ovn.org/pod-networks:                        {"default":{"ip_addresses":["10.129.2.15/23"],"mac_address":"0a:58:0a:81:02:0f","gateway_ips":["10.129.2.1"],"routes":[{"dest":"10.128.0.0...                      k8s.v1.cni.cncf.io/network-status:                        [{                            "name": "ovn-kubernetes",                            "interface": "eth0",                            "ips": [                                "10.129.2.15"                            ],                            "mac": "0a:58:0a:81:02:0f",                            "default": true,                            "dns": {}                        }]Status:               PendingIP:                   10.129.2.15IPs:  IP:           10.129.2.15Controlled By:  ReplicaSet/whereabouts-controller-6897c7f966Containers:  whereabouts:    Container ID:      Image:         quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e787c5a6f8dbd129f2914274a081e3fb239a4a409d5fe668d07c6cb30d190296    Image ID:          Port:          <none>    Host Port:     <none>    Command:      /usr/src/whereabouts/bin/node-slice-controller    State:          Waiting      Reason:       CreateContainerError    Ready:          False    Restart Count:  0    Limits:      cpu:     100m      memory:  200Mi    Requests:      cpu:     100m      memory:  100Mi    Environment:      NODENAME:                (v1:spec.nodeName)      WHEREABOUTS_NAMESPACE:  openshift-multus (v1:metadata.namespace)    Mounts:      /host/etc/cni/net.d from cni-net-dir (rw)      /host/etc/cni/net.d/whereabouts.d/whereabouts.conf from whereabouts-configmap (rw,path="whereabouts.conf")      /host/opt/cni/bin from cnibin (rw)      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-hwrdg (ro)Conditions:  Type                        Status  PodReadyToStartContainers   True   Initialized                 True   Ready                       False   ContainersReady             False   PodScheduled                True Volumes:  cnibin:    Type:          HostPath (bare host directory volume)    Path:          /var/lib/cni/bin    HostPathType:  Directory  cni-net-dir:    Type:          HostPath (bare host directory volume)    Path:          /etc/kubernetes/cni/net.d    HostPathType:  Directory  whereabouts-configmap:    Type:      ConfigMap (a volume populated by a ConfigMap)    Name:      whereabouts-config    Optional:  false  kube-api-access-hwrdg:    Type:                    Projected (a volume that contains injected data from multiple sources)    TokenExpirationSeconds:  3607    ConfigMapName:           kube-root-ca.crt    ConfigMapOptional:       <nil>    DownwardAPI:             true    ConfigMapName:           openshift-service-ca.crt    ConfigMapOptional:       <nil>QoS Class:                   BurstableNode-Selectors:              <none>Tolerations:                 node.kubernetes.io/memory-pressure:NoSchedule op=Exists                             node.kubernetes.io/not-ready:NoExecute op=Exists for 300s                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300sEvents:  Type     Reason          Age                   From               Message  ----     ------          ----                  ----               -------  Normal   Scheduled       3m16s                 default-scheduler  Successfully assigned openshift-multus/whereabouts-controller-6897c7f966-7bqts to weliang-4-18-2np9c-worker-2-5qx2z  Normal   AddedInterface  3m16s                 multus             Add eth0 [10.129.2.15/23] from ovn-kubernetes  Warning  Failed          73s (x12 over 3m16s)  kubelet            Error: container create failed: executable file `/usr/src/whereabouts/bin/node-slice-controller` not found: No such file or directory  Normal   Pulled          59s (x13 over 3m16s)  kubelet            Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e787c5a6f8dbd129f2914274a081e3fb239a4a409d5fe668d07c6cb30d190296" already present on machine
      $ oc delete pod whereabouts-controller-6897c7f966-7bqtspod "whereabouts-controller-6897c7f966-7bqts" deleted
      $ oc get pod | grep whereabouts-controllerwhereabouts-controller-6897c7f966-mf9df        0/1     CreateContainerError   0             22s$ oc describe pod whereabouts-controller-6897c7f966-mf9dfName:                 whereabouts-controller-6897c7f966-mf9dfNamespace:            openshift-multusPriority:             2000001000Priority Class Name:  system-node-criticalService Account:      multus-ancillary-toolsNode:                 weliang-4-18-2np9c-worker-1-b9nzq/10.242.1.4Start Time:           Thu, 17 Jul 2025 10:13:39 -0400Labels:               app=whereabouts-controller                      pod-template-hash=6897c7f966Annotations:          k8s.ovn.org/pod-networks:                        {"default":{"ip_addresses":["10.131.0.18/23"],"mac_address":"0a:58:0a:83:00:12","gateway_ips":["10.131.0.1"],"routes":[{"dest":"10.128.0.0...                      k8s.v1.cni.cncf.io/network-status:                        [{                            "name": "ovn-kubernetes",                            "interface": "eth0",                            "ips": [                                "10.131.0.18"                            ],                            "mac": "0a:58:0a:83:00:12",                            "default": true,                            "dns": {}                        }]Status:               PendingIP:                   10.131.0.18IPs:  IP:           10.131.0.18Controlled By:  ReplicaSet/whereabouts-controller-6897c7f966Containers:  whereabouts:    Container ID:      Image:         quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e787c5a6f8dbd129f2914274a081e3fb239a4a409d5fe668d07c6cb30d190296    Image ID:          Port:          <none>    Host Port:     <none>    Command:      /usr/src/whereabouts/bin/node-slice-controller    State:          Waiting      Reason:       CreateContainerError    Ready:          False    Restart Count:  0    Limits:      cpu:     100m      memory:  200Mi    Requests:      cpu:     100m      memory:  100Mi    Environment:      NODENAME:                (v1:spec.nodeName)      WHEREABOUTS_NAMESPACE:  openshift-multus (v1:metadata.namespace)    Mounts:      /host/etc/cni/net.d from cni-net-dir (rw)      /host/etc/cni/net.d/whereabouts.d/whereabouts.conf from whereabouts-configmap (rw,path="whereabouts.conf")      /host/opt/cni/bin from cnibin (rw)      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-9pf6j (ro)Conditions:  Type                        Status  PodReadyToStartContainers   True   Initialized                 True   Ready                       False   ContainersReady             False   PodScheduled                True Volumes:  cnibin:    Type:          HostPath (bare host directory volume)    Path:          /var/lib/cni/bin    HostPathType:  Directory  cni-net-dir:    Type:          HostPath (bare host directory volume)    Path:          /etc/kubernetes/cni/net.d    HostPathType:  Directory  whereabouts-configmap:    Type:      ConfigMap (a volume populated by a ConfigMap)    Name:      whereabouts-config    Optional:  false  kube-api-access-9pf6j:    Type:                    Projected (a volume that contains injected data from multiple sources)    TokenExpirationSeconds:  3607    ConfigMapName:           kube-root-ca.crt    ConfigMapOptional:       <nil>    DownwardAPI:             true    ConfigMapName:           openshift-service-ca.crt    ConfigMapOptional:       <nil>QoS Class:                   BurstableNode-Selectors:              <none>Tolerations:                 node.kubernetes.io/memory-pressure:NoSchedule op=Exists                             node.kubernetes.io/not-ready:NoExecute op=Exists for 300s                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300sEvents:  Type     Reason          Age               From               Message  ----     ------          ----              ----               -------  Normal   Scheduled       35s               default-scheduler  Successfully assigned openshift-multus/whereabouts-controller-6897c7f966-mf9df to weliang-4-18-2np9c-worker-1-b9nzq  Normal   AddedInterface  35s               multus             Add eth0 [10.131.0.18/23] from ovn-kubernetes  Normal   Pulled          8s (x5 over 35s)  kubelet            Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:e787c5a6f8dbd129f2914274a081e3fb239a4a409d5fe668d07c6cb30d190296" already present on machine  Warning  Failed          8s (x5 over 35s)  kubelet            Error: container create failed: executable file `/usr/src/whereabouts/bin/node-slice-controller` not found: No such file or directory$ 
      
      

      Expected results:

      whereabouts-controller pod is running

      Additional info:

      This issue is blocking the verification for  https://issues.redhat.com/browse/OCPBUGS-57643

              bpickard@redhat.com Ben Pickard
              weliang1@redhat.com Weibin Liang
              None
              None
              Weibin Liang Weibin Liang
              None
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated:
                Resolved: