Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-37286

redhat-operators catalog failing and cannot install redhat operators in a new installation

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Not a Bug
    • Icon: Normal Normal
    • None
    • 4.15.0
    • OLM
    • None
    • Important
    • None
    • Rejected
    • False
    • Hide

      None

      Show
      None

      Description of problem:

      When we try to install operators from other catalogs (community, certified...) they fail as well without showing the same error, but when we disable the redhat-operators catalog, they install successfully. This seems to be in line with the code where the error gets initiated, where apparently a faulty catalog would block installations from others:

      Version-Release number of selected component (if applicable):

          4.15.19

      How reproducible:

          100%

      Steps to Reproduce:

      - Installed fresh cluster with agent-based installer - A three node cluster with static IPs 
      - Cluster up and running normally 
      - Tried with versions 4.15.19 and 4.15.20, same result. Upgrading 4.15.19 to 4.16.0. No fix. 
      - Install LSO operator fails (Red Hat catalog) with error: https://github.com/openshift/operator-framework-olm/blob/28befe20a9f97db2a4a47c655b483de3197622da/staging/operator-lifecycle-manager/pkg/controller/registry/resolver/cache/cache.go#L142 
      - Install ODF operator fails (Red Hat catalog) with the same error 
      - Install Nvidia GPU operator fails (Certified catalog) with an error on Red Hat catalog 
      - After see that the error is triggered here: https://github.com/openshift/operator-framework-olm/blob/28befe20a9f97db2a4a47c655b483de3197622da/staging/operator-lifecycle-manager/pkg/controller/registry/resolver/cache/cache.go#L413 . We disable Red Hat Catalog and Nvidia GPU operator install correctly. Seems one catalog blocked the other     

      Actual results:

      oc get pods -n openshift-marketplace
      NAME                                    READY   STATUS     RESTARTS   AGE
      certified-operators-2n5j9               1/1     Running    0          1h
      community-operators-lx6v9               1/1     Running    0          1h
      marketplace-operator-7f6676d76c-8qnrp   1/1     Running    0          1h
      redhat-marketplace-zz5xk                1/1     Running    0          1h
      redhat-operators-xtsg7                  0/1     Init:1/2   0          24m
      redhat-operators-xzl5d                  1/1     Running    0          34m

      Expected results:

          redhat-operator-x pod should run fine and operator installation should take place

      Additional info:

          Workaround for the issue:
          1. Disable the  redhat-operator catalogue
          2. Create a custom catalogue pointing to redhat-operator index image
          3. edit the registry poll    apiVersion: operators.coreos.com/v1alpha1
             kind: CatalogSource
             metadata:
               name: custom-redhat-catalog
               namespace: openshift-marketplace
             spec:
               displayName: Red Hat Catalog
               image: 'registry.redhat.io/redhat/redhat-operator-index:v4.15'
               publisher: ''
               sourceType: grpc
               updateStrategy:
                 registryPoll:
                   interval: 30m     
      
          4. Try installing the operator it will work
      
      

       

       

            agreene1991 Alexander Greene (Inactive)
            rhn-support-gmahto Gagan Mahto
            Jian Zhang Jian Zhang
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

              Created:
              Updated:
              Resolved: