Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-20086

Operators fail to install because Catalogsources stuck in CONNECTING/TRANSIENT_FAILURE

XMLWordPrintable

    • No
    • Rejected
    • False
    • Hide

      None

      Show
      None

      Description of problem:

      While installing and applying the du profile to many disconnected clusters via ACM/ZTP, 2 clusters failed to install any du profile operators because their catalogsources remained stuck in CONNECTING/TRANSIENT_FAILURE state.  This prevented those clusters from completely rolling out the du profile.

      Version-Release number of selected component (if applicable):

      Hub OCP - 4.14.0-rc.2
      Deployed clusters 4.14.0-rc.2
      ACM - 2.9.0-DOWNSTREAM-2023-09-27-22-12-46

      How reproducible:

      2 out of 4 failures related to the du profile being applied. (2 out of 1816 clusters which had the du profile applied to)

      Steps to Reproduce:

      1.
      2.
      3.
      

      Actual results:

       

      Expected results:

       

      Additional info:

      Affected clusters:
      standard-00172 - 5 node cluster
      vm01131 - Single Node OpenShift
      
      # oc --kubeconfig /root/hv-vm/kc/standard-00172/kubeconfig get catalogsource -n openshift-marketplace -o yaml
      apiVersion: v1
      items:
      - apiVersion: operators.coreos.com/v1alpha1
        kind: CatalogSource
        metadata:
          annotations:
            scale-test-label-1: value3
            scale-test-label-2: value4
            target.workload.openshift.io/management: '{"effect": "PreferredDuringScheduling"}'
          creationTimestamp: "2023-10-02T22:52:44Z"
          generation: 1
          name: rh-du-operators
          namespace: openshift-marketplace
          resourceVersion: "908795"
          uid: f851a104-7a05-4b98-b2b0-1f18e82b3035
        spec:
          displayName: disconnected-redhat-operators
          image: e27-h01-000-r650.rdu2.scalelab.redhat.com:5000/olm-mirror/redhat-operator-index:v4.13
          publisher: Red Hat
          sourceType: grpc
          updateStrategy:
            registryPoll:
              interval: 1h
        status:
          connectionState:
            address: rh-du-operators.openshift-marketplace.svc:50051
            lastConnect: "2023-10-04T14:48:55Z"
            lastObservedState: TRANSIENT_FAILURE
          latestImageRegistryPoll: "2023-10-04T13:53:05Z"
          registryService:
            createdAt: "2023-10-02T22:52:45Z"
            port: "50051"
            protocol: grpc
            serviceName: rh-du-operators
            serviceNamespace: openshift-marketplace
      kind: List
      metadata:
        resourceVersion: ""
      
      # oc --kubeconfig /root/hv-vm/kc/vm01131/kubeconfig get catalogsource -n openshift-marketplace -o yaml
      apiVersion: v1
      items:
      - apiVersion: operators.coreos.com/v1alpha1
        kind: CatalogSource
        metadata:
          annotations:
            scale-test-label-1: value3
            scale-test-label-2: value4
            target.workload.openshift.io/management: '{"effect": "PreferredDuringScheduling"}'
          creationTimestamp: "2023-10-03T02:28:33Z"
          generation: 1
          name: rh-du-operators
          namespace: openshift-marketplace
          resourceVersion: "517268"
          uid: cc30a95e-63b0-4b61-8f98-a0cbd809fd3f
        spec:
          displayName: disconnected-redhat-operators
          image: e27-h01-000-r650.rdu2.scalelab.redhat.com:5000/olm-mirror/redhat-operator-index:v4.13
          publisher: Red Hat
          sourceType: grpc
          updateStrategy:
            registryPoll:
              interval: 1h
        status:
          connectionState:
            address: rh-du-operators.openshift-marketplace.svc:50051
            lastConnect: "2023-10-04T14:48:13Z"
            lastObservedState: CONNECTING
          latestImageRegistryPoll: "2023-10-04T14:28:37Z"
          registryService:
            createdAt: "2023-10-03T02:28:33Z"
            port: "50051"
            protocol: grpc
            serviceName: rh-du-operators
            serviceNamespace: openshift-marketplace
      kind: List
      metadata:
        resourceVersion: ""
      

       

            krizza@redhat.com Kevin Rizza
            akrzos@redhat.com Alex Krzos
            Jian Zhang Jian Zhang
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated:
              Resolved: