Uploaded image for project: 'Project Quay'
  1. Project Quay
  2. PROJQUAY-2476

Bridge operator pod stuck in 'CreateContainerError' status

    XMLWordPrintable

Details

    • False
    • False
    • Quay Enterprise
    • 0

    Description

      Description of problem:

       When install bridge operator via OLM, the Bridge operator pod stuck in 'CreateContainerError' status, cannot be running

      Version-Release number of selected component (if applicable):

      quay-bridge-operator-bundle-container-v3.6.0-14

      • Index image v4.5: registry-proxy.engineering.redhat.com/rh-osbs/iib:104363
      • Index image v4.6: registry-proxy.engineering.redhat.com/rh-osbs/iib:104367
      • Index image v4.7: registry-proxy.engineering.redhat.com/rh-osbs/iib:104373
      • Index image v4.8: registry-proxy.engineering.redhat.com/rh-osbs/iib:104376
      • Index image v4.9: registry-proxy.engineering.redhat.com/rh-osbs/iib:104381

      quay-bridge-operator-container-v3.6.0-17

      • registry-proxy.engineering.redhat.com/rh-osbs/quay-quay-bridge-operator-rhel8@sha256:5179892b4c73ec3e9b8fe89bc2996d1e2ec4ab230d5837859e2f5339803128c3

       

      How reproducible:

      always

       

      Steps to Reproduce:
      1. create custom catalogsource from index image

      apiVersion: operators.coreos.com/v1alpha1
      kind: CatalogSource
      metadata:
        name: bridge-operator-cs
        namespace: openshift-marketplace
      spec:
        sourceType: grpc
        image: brew.registry.redhat.io/rh-osbs/iib:104376
        displayName: Bridge36 Operator Catalog
        publisher: Quay
      

      2, Open openshift webconsole, install bridge operator by operatorhub
      3. Check operator status
      4. Check operator pod status

      Actual results:

      Bridge operator pod stuck in 'CreateContainerError' status

       

      Expected results:

       Operator pod is running, operator install succeed

      Additional info:

      $ oc get po -n openshift-operators
      NAME                                                       READY   STATUS                 RESTARTS   AGE
      quay-bridge-operator-controller-manager-7bb9bc45d6-4n97l   1/2     CreateContainerError   0          7m13s
      
      
      
      $ oc describe pod -n openshift-operators
      Name:         quay-bridge-operator-controller-manager-7bb9bc45d6-4n97l
      Namespace:    openshift-operators
      Priority:     0
      Node:         ip-10-0-138-80.us-east-2.compute.internal/10.0.138.80
      Start Time:   Wed, 01 Sep 2021 17:18:59 +0800
      Labels:       control-plane=controller-manager
                    pod-template-hash=7bb9bc45d6
      Annotations:  alm-examples:
                      [
                        {
                          "apiVersion": "redhatcop.redhat.io/v1",
                          "kind": "QuayIntegration",
                          "metadata": {
                            "name": "quay"
                          },
                          "spec": {
                            "clusterID": "openshift",
                            "credentialsSecret": {
                              "name": "quay-credentials",
                              "namespace": "openshift-operators"
                            },
                            "insecureRegistry": false
                          }
                        }
                      ]
                    capabilities: Full Lifecycle
                    categories: OpenShift Optional
                    containerImage: registry.redhat.io/quay/quay-bridge-operator-rhel8@sha256:5179892b4c73ec3e9b8fe89bc2996d1e2ec4ab230d5837859e2f5339803128c3
                    createdAt: 2021-08-31 12:08 UTC
                    description: Enhance OCP using Red Hat Quay container registry
                    k8s.v1.cni.cncf.io/network-status:
                      [{
                          "name": "openshift-sdn",
                          "interface": "eth0",
                          "ips": [
                              "10.129.2.15"
                          ],
                          "default": true,
                          "dns": {}
                      }]
                    k8s.v1.cni.cncf.io/networks-status:
                      [{
                          "name": "openshift-sdn",
                          "interface": "eth0",
                          "ips": [
                              "10.129.2.15"
                          ],
                          "default": true,
                          "dns": {}
                      }]
                    olm.operatorGroup: global-operators
                    olm.operatorNamespace: openshift-operators
                    olm.skipRange: >=3.5.x <3.6.0
                    olm.targetNamespaces: 
                    olmcahash: ec6c64c9d15474119d18753795b12f93298a54949f19b4f1d276f0bc674b9573
                    openshift.io/scc: restricted
                    operatorframework.io/properties:
                      {"properties":[{"type":"olm.gvk","value":{"group":"redhatcop.redhat.io","kind":"QuayIntegration","version":"v1"}},{"type":"olm.package","v...
                    operators.openshift.io/infrastructure-features: ["disconnected"]
                    operators.operatorframework.io/builder: operator-sdk-v1.9.0+git
                    operators.operatorframework.io/project_layout: go.kubebuilder.io/v3
                    repository: https://github.com/quay/quay-bridge-operator
      Status:       Pending
      IP:           10.129.2.15
      IPs:
        IP:           10.129.2.15
      Controlled By:  ReplicaSet/quay-bridge-operator-controller-manager-7bb9bc45d6
      Containers:
        kube-rbac-proxy:
          Container ID:  cri-o://7e8bc26f1bb158df8e8c9055df612c850111e8d7e0e779d6727a1bc55adc4950
          Image:         registry.redhat.io/openshift4/ose-kube-rbac-proxy@sha256:71c2b03f01b0af6cb349e06bc2dc035a81e10f420ffb6358a4fdc5494bb57171
          Image ID:      quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:71c2b03f01b0af6cb349e06bc2dc035a81e10f420ffb6358a4fdc5494bb57171
          Port:          8443/TCP
          Host Port:     0/TCP
          Args:
            --secure-listen-address=0.0.0.0:8443
            --upstream=http://127.0.0.1:8080/
            --logtostderr=true
            --v=10
          State:          Running
            Started:      Wed, 01 Sep 2021 17:19:06 +0800
          Ready:          True
          Restart Count:  0
          Environment:
            OPERATOR_CONDITION_NAME:  quay-bridge-operator.v3.6.0
          Mounts:
            /apiserver.local.config/certificates from apiservice-cert (rw)
            /tmp/k8s-webhook-server/serving-certs from webhook-cert (rw)
            /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-lj8wl (ro)
        manager:
          Container ID:  
          Image:         registry.redhat.io/quay/quay-bridge-operator-rhel8@sha256:5179892b4c73ec3e9b8fe89bc2996d1e2ec4ab230d5837859e2f5339803128c3
          Image ID:      
          Port:          9443/TCP
          Host Port:     0/TCP
          Command:
            /manager
          Args:
            --health-probe-bind-address=:8081
            --metrics-bind-address=127.0.0.1:8080
            --leader-elect
          State:          Waiting
            Reason:       CreateContainerError
          Ready:          False
          Restart Count:  0
          Limits:
            cpu:     100m
            memory:  30Mi
          Requests:
            cpu:      100m
            memory:   20Mi
          Liveness:   http-get http://:8081/healthz delay=15s timeout=1s period=20s #success=1 #failure=3
          Readiness:  http-get http://:8081/readyz delay=5s timeout=1s period=10s #success=1 #failure=3
          Environment:
            OPERATOR_CONDITION_NAME:  quay-bridge-operator.v3.6.0
          Mounts:
            /apiserver.local.config/certificates from apiservice-cert (ro)
            /tmp/k8s-webhook-server/serving-certs from webhook-cert (rw)
            /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-lj8wl (ro)
      Conditions:
        Type              Status
        Initialized       True 
        Ready             False 
        ContainersReady   False 
        PodScheduled      True 
      Volumes:
        apiservice-cert:
          Type:        Secret (a volume populated by a Secret)
          SecretName:  quay-bridge-operator-controller-manager-service-cert
          Optional:    false
        webhook-cert:
          Type:        Secret (a volume populated by a Secret)
          SecretName:  quay-bridge-operator-controller-manager-service-cert
          Optional:    false
        kube-api-access-lj8wl:
          Type:                    Projected (a volume that contains injected data from multiple sources)
          TokenExpirationSeconds:  3607
          ConfigMapName:           kube-root-ca.crt
          ConfigMapOptional:       <nil>
          DownwardAPI:             true
          ConfigMapName:           openshift-service-ca.crt
          ConfigMapOptional:       <nil>
      QoS Class:                   Burstable
      Node-Selectors:              <none>
      Tolerations:                 node.kubernetes.io/memory-pressure:NoSchedule op=Exists
                                   node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                                   node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
      Events:
        Type     Reason          Age                  From               Message
        ----     ------          ----                 ----               -------
        Normal   Scheduled       3m10s                default-scheduler  Successfully assigned openshift-operators/quay-bridge-operator-controller-manager-7bb9bc45d6-4n97l to ip-10-0-138-80.us-east-2.compute.internal
        Normal   AddedInterface  3m8s                 multus             Add eth0 [10.129.2.15/23] from openshift-sdn
        Normal   Pulling         3m8s                 kubelet            Pulling image "registry.redhat.io/openshift4/ose-kube-rbac-proxy@sha256:71c2b03f01b0af6cb349e06bc2dc035a81e10f420ffb6358a4fdc5494bb57171"
        Normal   Pulling         3m3s                 kubelet            Pulling image "registry.redhat.io/quay/quay-bridge-operator-rhel8@sha256:5179892b4c73ec3e9b8fe89bc2996d1e2ec4ab230d5837859e2f5339803128c3"
        Normal   Created         3m3s                 kubelet            Created container kube-rbac-proxy
        Normal   Started         3m3s                 kubelet            Started container kube-rbac-proxy
        Normal   Pulled          3m3s                 kubelet            Successfully pulled image "registry.redhat.io/openshift4/ose-kube-rbac-proxy@sha256:71c2b03f01b0af6cb349e06bc2dc035a81e10f420ffb6358a4fdc5494bb57171" in 4.87105609s
        Normal   Pulled          2m57s                kubelet            Successfully pulled image "registry.redhat.io/quay/quay-bridge-operator-rhel8@sha256:5179892b4c73ec3e9b8fe89bc2996d1e2ec4ab230d5837859e2f5339803128c3" in 6.042565442s
        Warning  Failed          2m57s                kubelet            Error: container create failed: time="2021-09-01T09:19:12Z" level=error msg="container_linux.go:367: starting container process caused: exec: \"/manager\": stat /manager: no such file or directory"
        Warning  Failed          2m56s                kubelet            Error: container create failed: time="2021-09-01T09:19:13Z" level=error msg="container_linux.go:367: starting container process caused: exec: \"/manager\": stat /manager: no such file or directory"
        Warning  Failed          2m42s                kubelet            Error: container create failed: time="2021-09-01T09:19:27Z" level=error msg="container_linux.go:367: starting container process caused: exec: \"/manager\": stat /manager: no such file or directory"
        Warning  Failed          2m27s                kubelet            Error: container create failed: time="2021-09-01T09:19:42Z" level=error msg="container_linux.go:367: starting container process caused: exec: \"/manager\": stat /manager: no such file or directory"
        Warning  Failed          2m14s                kubelet            Error: container create failed: time="2021-09-01T09:19:55Z" level=error msg="container_linux.go:367: starting container process caused: exec: \"/manager\": stat /manager: no such file or directory"
        Warning  Failed          2m                   kubelet            Error: container create failed: time="2021-09-01T09:20:09Z" level=error msg="container_linux.go:367: starting container process caused: exec: \"/manager\": stat /manager: no such file or directory"
        Warning  Failed          107s                 kubelet            Error: container create failed: time="2021-09-01T09:20:22Z" level=error msg="container_linux.go:367: starting container process caused: exec: \"/manager\": stat /manager: no such file or directory"
        Warning  Failed          91s                  kubelet            Error: container create failed: time="2021-09-01T09:20:38Z" level=error msg="container_linux.go:367: starting container process caused: exec: \"/manager\": stat /manager: no such file or directory"
        Warning  Failed          78s                  kubelet            Error: container create failed: time="2021-09-01T09:20:51Z" level=error msg="container_linux.go:367: starting container process caused: exec: \"/manager\": stat /manager: no such file or directory"
        Normal   Pulled          63s (x9 over 2m56s)  kubelet            Container image "registry.redhat.io/quay/quay-bridge-operator-rhel8@sha256:5179892b4c73ec3e9b8fe89bc2996d1e2ec4ab230d5837859e2f5339803128c3" already present on machine
        Warning  Failed          63s                  kubelet            (combined from similar events): Error: container create failed: time="2021-09-01T09:21:06Z" level=error msg="container_linux.go:367: starting container process caused: exec: \"/manager\": stat /manager: no such file or directory"
      

      Attachments

        Issue Links

          Activity

            People

              hgovinda Harish Govindarajulu
              rhn-support-dyan Dongbo Yan
              Dongbo Yan Dongbo Yan
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: