-
Bug
-
Resolution: Done
-
Minor
-
None
-
4.16.z
-
None
-
No
-
False
-
Description: Operator hub shows 503 operators when tried to install a few operator fails because of invalid image used on compute nodes.
Pre-conditons:
Deploy an multiarch x86 ocp cluster with 2 KVM s390x nodes as additonal compute nodes with 416-rc-3 build.
oc version
Client Version: 4.16.0-rc.3
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: 4.16.0-rc.3
Kubernetes Version: v1.29.5+87992f4
Steps to reproduce:
1. Install 3scale (community catalog source ) API Management operator from console
2. Once installed check yaml labels: it says s390x is supported
3. Operator failed to deploy
4. Tried 3scale (community) , BusinessAutomation (Redhat catalog source) & NLPLAB operato (community)
Expected: Only operators which are supported by the worker node platforms which is amd64 & s390x in this cluster should be displayed on operator hub.
Logs:
NAME READY STATUS RESTARTS AGE
threescale-operator-controller-manager-v2-bcbcfc97d-pvmx5 0/1 Error 12 36m
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
threescale-operator-controller-manager-v2-bcbcfc97d-pvmx5 0/1 CrashLoopBackOff 12 (5m1s ago) 41m 10.129.2.7 worker-3.multiarch.lnxero1.boe <none> <none>
oc get csv
NAME DISPLAY VERSION REPLACES PHASE
3scale-community-operator.v0.10.1 3scale API Management 0.10.1 3scale-community-operator.v0.9.0 Failed
4. Image of the container says amd64 insead of s390x
oc get csv 3scale-community-operator.v0.10.1 -oyaml shows s390x supported
labels:
olm.managed: "true"
operatorframework.io/arch.amd64: supported
operatorframework.io/arch.ppc64le: supported
operatorframework.io/arch.s390x: supported
operators.coreos.com/3scale-community-operator.openshift-operators: ""
[root@worker-3 ~]# podman inspect 6bd26afe9252 |grep -i arch
"Architecture": "amd64",
[root@worker-3 ~]#
[root@worker-3 ~]#
[root@worker-3 ~]#
[root@worker-3 ~]# podman images |grep -i 6bd26afe9252
quay.io/3scale/3scale-operator v0.10.1 6bd26afe9252 15 months ago 61 MB
========== NLPLAB operator ===========
Now using project "johnsnowlabs-operator" on server "https://api.multiarch.lnxero1.boe:6443".
[root@bastion ~]# la
Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
NAME READY STATUS RESTARTS AGE
pod/annotationlab-controller-manager-7f47c66486-8gcmd 1/2 CrashLoopBackOff 8 (70s ago) 17m
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/annotationlab-controller-manager-metrics-service ClusterIP 172.30.9.165 <none> 8443/TCP 17m
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/annotationlab-controller-manager 0/1 1 0 17m
NAME DESIRED CURRENT READY AGE
replicaset.apps/annotationlab-controller-manager-7f47c66486 1 1 0 17m
[root@bastion ~]# lcsv
NAME DISPLAY VERSION REPLACES PHASE
3scale-community-operator.v0.10.1 3scale API Management 0.10.1 3scale-community-operator.v0.9.0 Failed
annotationlab.v5.7.0 NLPLab 5.7.0 annotationlab.v5.4.1 Failed
[root@worker-2 ~]# podman images |grep -i 5653c10fab31
registry.johnsnowlabs.com/annotationlab/annotationlab-operator v5.7.0 5653c10fab31 5 months ago 170 MB
[root@worker-2 ~]# podman inspect 5653c10fab31|grep -i arch
"architecture": "x86_64",
"Architecture": "amd64",
"architecture": "x86_64",
"created_by": "/bin/sh -c #(nop) LABEL \"distribution-scope\"=\"public\" \"vendor\"=\"Red Hat, Inc.\" \"build-date\"=\"2023-03-28T04:02:11\" \"architecture\"=\"x86_64\" \"vcs-type\"=\"git\" \"vcs-ref\"=\"146fdafc2595e26f5f9c1b9a2b3f36bbca8237e4\" \"io.k8s.description\"=\"The Universal Base Image Minimal is a stripped down image that uses microdnf as a package manager. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly.\" \"url\"=\"https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8-minimal/images/8.7-1107\"",
Business Automation
7.13.5-3 provided by Red Hat
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 3m16s default-scheduler Successfully assigned default/business-automation-operator-746c569ccf-k287s to worker-3.multiarch.lnxero1.boe
Normal AddedInterface 3m16s multus Add eth0 [10.129.2.8/23] from ovn-kubernetes
Normal Pulling 104s (x4 over 3m16s) kubelet Pulling image "registry.redhat.io/rhpam-7/rhpam-rhel8-operator@sha256:df3954934e18a75299ee31453891e066c2eb3f49fa7ae8133c202f692436fea6"
Warning Failed 103s (x4 over 3m16s) kubelet Failed to pull image "registry.redhat.io/rhpam-7/rhpam-rhel8-operator@sha256:df3954934e18a75299ee31453891e066c2eb3f49fa7ae8133c202f692436fea6": choosing image instance: no image found in manifest list for architecture s390x, variant "", OS linux
Warning Failed 103s (x4 over 3m16s) kubelet Error: ErrImagePull
Warning Failed 90s (x6 over 3m15s) kubelet Error: ImagePullBackOff
Normal BackOff 77s (x7 over 3m15s) kubelet Back-off pulling image "registry.redhat.io/rhpam-7/rhpam-rhel8-operator@sha256:df3954934e18a75299ee31453891e066c2eb3f49fa7ae8133c202f692436fea6"
[root@bastion ~]#
[root@bastion ~]# la
Warning: apps.openshift.io/v1 DeploymentConfig is deprecated in v4.14+, unavailable in v4.10000+
NAME READY STATUS RESTARTS AGE
pod/business-automation-operator-746c569ccf-k287s 0/1 ImagePullBackOff 0 4m35s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubernetes ClusterIP 172.30.0.1 <none> 443/TCP 91m
service/openshift ExternalName <none> kubernetes.default.svc.cluster.local <none> 88m
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/business-automation-operator 0/1 1 0 4m35s
NAME DESIRED CURRENT READY AGE
replicaset.apps/business-automation-operator-746c569ccf 1 1 0 4m35s