Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-35412

Multi-Arch: Operator Hub shows operators that are not built for s390x

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Minor Minor
    • None
    • 4.16.z
    • None
    • No
    • False
    • Hide

      None

      Show
      None

      Description: Operator hub shows 503 operators when tried to install a few operator fails because of invalid image used on compute nodes.

      Pre-conditons:
      Deploy an multiarch x86 ocp cluster with 2 KVM s390x nodes as additonal compute nodes with 416-rc-3 build.
      oc version
      Client Version: 4.16.0-rc.3
      Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
      Server Version: 4.16.0-rc.3
      Kubernetes Version: v1.29.5+87992f4

      Steps to reproduce:
      1. Install 3scale (community catalog source ) API Management operator from console
      2. Once installed check yaml labels: it says s390x is supported
      3. Operator failed to deploy
      4. Tried 3scale (community) , BusinessAutomation (Redhat catalog source) & NLPLAB operato (community)

      Expected: Only operators which are supported by the worker node platforms which is amd64 & s390x in this cluster should be displayed on operator hub.

      Logs:
      NAME READY STATUS RESTARTS AGE
      threescale-operator-controller-manager-v2-bcbcfc97d-pvmx5 0/1 Error 12 36m

      NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
      threescale-operator-controller-manager-v2-bcbcfc97d-pvmx5 0/1 CrashLoopBackOff 12 (5m1s ago) 41m 10.129.2.7 worker-3.multiarch.lnxero1.boe <none> <none>

      oc get csv

      NAME DISPLAY VERSION REPLACES PHASE
      3scale-community-operator.v0.10.1 3scale API Management 0.10.1 3scale-community-operator.v0.9.0 Failed

      4. Image of the container says amd64 insead of s390x
      oc get csv 3scale-community-operator.v0.10.1 -oyaml shows s390x supported

      labels:
      olm.managed: "true"
      operatorframework.io/arch.amd64: supported
      operatorframework.io/arch.ppc64le: supported
      operatorframework.io/arch.s390x: supported
      operators.coreos.com/3scale-community-operator.openshift-operators: ""

      [root@worker-3 ~]# podman inspect 6bd26afe9252 |grep -i arch
      "Architecture": "amd64",
      [root@worker-3 ~]#
      [root@worker-3 ~]#
      [root@worker-3 ~]#
      [root@worker-3 ~]# podman images |grep -i 6bd26afe9252
      quay.io/3scale/3scale-operator v0.10.1 6bd26afe9252 15 months ago 61 MB

      ========== NLPLAB operator ===========
      Now using project "johnsnowlabs-operator" on server "https://api.multiarch.lnxero1.boe:6443".
      [root@bastion ~]# la
      Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
      NAME READY STATUS RESTARTS AGE
      pod/annotationlab-controller-manager-7f47c66486-8gcmd 1/2 CrashLoopBackOff 8 (70s ago) 17m

      NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
      service/annotationlab-controller-manager-metrics-service ClusterIP 172.30.9.165 <none> 8443/TCP 17m

      NAME READY UP-TO-DATE AVAILABLE AGE
      deployment.apps/annotationlab-controller-manager 0/1 1 0 17m

      NAME DESIRED CURRENT READY AGE
      replicaset.apps/annotationlab-controller-manager-7f47c66486 1 1 0 17m
      [root@bastion ~]# lcsv
      NAME DISPLAY VERSION REPLACES PHASE
      3scale-community-operator.v0.10.1 3scale API Management 0.10.1 3scale-community-operator.v0.9.0 Failed
      annotationlab.v5.7.0 NLPLab 5.7.0 annotationlab.v5.4.1 Failed

      [root@worker-2 ~]# podman images |grep -i 5653c10fab31
      registry.johnsnowlabs.com/annotationlab/annotationlab-operator v5.7.0 5653c10fab31 5 months ago 170 MB
      [root@worker-2 ~]# podman inspect 5653c10fab31|grep -i arch
      "architecture": "x86_64",
      "Architecture": "amd64",
      "architecture": "x86_64",
      "created_by": "/bin/sh -c #(nop) LABEL \"distribution-scope\"=\"public\" \"vendor\"=\"Red Hat, Inc.\" \"build-date\"=\"2023-03-28T04:02:11\" \"architecture\"=\"x86_64\" \"vcs-type\"=\"git\" \"vcs-ref\"=\"146fdafc2595e26f5f9c1b9a2b3f36bbca8237e4\" \"io.k8s.description\"=\"The Universal Base Image Minimal is a stripped down image that uses microdnf as a package manager. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly.\" \"url\"=\"https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8-minimal/images/8.7-1107\"",

      Business Automation
      7.13.5-3 provided by Red Hat

      Events:
      Type Reason Age From Message
      ---- ------ ---- ---- -------
      Normal Scheduled 3m16s default-scheduler Successfully assigned default/business-automation-operator-746c569ccf-k287s to worker-3.multiarch.lnxero1.boe
      Normal AddedInterface 3m16s multus Add eth0 [10.129.2.8/23] from ovn-kubernetes
      Normal Pulling 104s (x4 over 3m16s) kubelet Pulling image "registry.redhat.io/rhpam-7/rhpam-rhel8-operator@sha256:df3954934e18a75299ee31453891e066c2eb3f49fa7ae8133c202f692436fea6"
      Warning Failed 103s (x4 over 3m16s) kubelet Failed to pull image "registry.redhat.io/rhpam-7/rhpam-rhel8-operator@sha256:df3954934e18a75299ee31453891e066c2eb3f49fa7ae8133c202f692436fea6": choosing image instance: no image found in manifest list for architecture s390x, variant "", OS linux
      Warning Failed 103s (x4 over 3m16s) kubelet Error: ErrImagePull
      Warning Failed 90s (x6 over 3m15s) kubelet Error: ImagePullBackOff
      Normal BackOff 77s (x7 over 3m15s) kubelet Back-off pulling image "registry.redhat.io/rhpam-7/rhpam-rhel8-operator@sha256:df3954934e18a75299ee31453891e066c2eb3f49fa7ae8133c202f692436fea6"
      [root@bastion ~]#
      [root@bastion ~]# la
      Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
      NAME READY STATUS RESTARTS AGE
      pod/business-automation-operator-746c569ccf-k287s 0/1 ImagePullBackOff 0 4m35s

      NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
      service/kubernetes ClusterIP 172.30.0.1 <none> 443/TCP 91m
      service/openshift ExternalName <none> kubernetes.default.svc.cluster.local <none> 88m

      NAME READY UP-TO-DATE AVAILABLE AGE
      deployment.apps/business-automation-operator 0/1 1 0 4m35s

      NAME DESIRED CURRENT READY AGE
      replicaset.apps/business-automation-operator-746c569ccf 1 1 0 4m35s

            jpoulin Jeremy Poulin
            apuranda Amrut Purandare
            Doug Slavens Doug Slavens
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated: