Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-38035

IBM Operator Index Image fails with "cache requires rebuild: cache reports digest as xxx, but computed digest is yyy"

XMLWordPrintable

    • +
    • Critical
    • None
    • False
    • Hide

      None

      Show
      None
    • Hide
      * Previously, when you mirrored Operator catalogs with the `--rebuild-catalogs` argument, catalog cache was recreated on the local machine. This required extraction and use of the `opm` binary from the catalog image, which caused failure of either the mirroring operation or the catalog source. These failures would happen because the supported operating system and the platform of the `opm` binary caused a mismatch with the operating system and platform of `oc-mirror`. With this release, the value of `true` is applied to the `--rebuild-catalogs` argument by default so that any catalog rebuilds do not re-create internal cache. Additionally, this release updates the image from `opm serve /configs --cache-dir=/tmp/cache` to `opm serve /configs` so that the creation of cache happens at pod startup. Cache at startup might increase pod startup time. (link:https://issues.redhat.com/browse/OCPBUGS-38035[*OCPBUGS-38035*])
      Show
      * Previously, when you mirrored Operator catalogs with the `--rebuild-catalogs` argument, catalog cache was recreated on the local machine. This required extraction and use of the `opm` binary from the catalog image, which caused failure of either the mirroring operation or the catalog source. These failures would happen because the supported operating system and the platform of the `opm` binary caused a mismatch with the operating system and platform of `oc-mirror`. With this release, the value of `true` is applied to the `--rebuild-catalogs` argument by default so that any catalog rebuilds do not re-create internal cache. Additionally, this release updates the image from `opm serve /configs --cache-dir=/tmp/cache` to `opm serve /configs` so that the creation of cache happens at pod startup. Cache at startup might increase pod startup time. (link: https://issues.redhat.com/browse/OCPBUGS-38035 [* OCPBUGS-38035 *])
    • Bug Fix
    • Done

      This is a clone of issue OCPBUGS-37667. The following is the description of the original issue:

      Description of problem:

      After successfully mirroring the ibm-ftm-operator via the latest oc-mirror command to internal registry and applying the newly generated IBM CatalogSource YAML file. The created catalog pod in the openShift-marketplace namespace enters CrashLoopBackOff.
      
      Customer is trying to mirror operators and  list the catalogue the command has no issues, but catalog pod is crashing with the following error:
      ~~~
      time="2024-07-10T13:43:07Z" level=info msg="starting pprof endpoint" address="localhost:6060"
      time="2024-07-10T13:43:08Z" level=fatal msg="cache requires rebuild: cache reports digest as \"e891bfd5a4cb5702\", but computed digest is \"1922475dc0ee190c\""
      ~~~
      
          

      Version-Release number of selected component (if applicable):

      oc-mirror 4.16
      OCP 4.14.z
      
          

      How reproducible:

      
      
          

      Steps to Reproduce:

          1. Create catalog image with the following imagesetconfiguration:
      ~~~
      kind: ImageSetConfiguration
      apiVersion: mirror.openshift.io/v1alpha2
      archiveSize: 4
      storageConfig:
        registry:
          imageURL: <internal-registry>:Port/oc-mirror-metadata/12july24
          skipTLS: false
      mirror:
        platform:
          architectures:
            - "amd64"
          channels:
          - name: stable-4.14
            minVersion: 4.14.11
            maxVersion: 4.14.30
            type: ocp
            shortestPath: true
          graph: true
        operators:
        - catalog: icr.io/cpopen/ibm-operator-catalog:v1.22
          packages:
          - name: ibm-ftm-operator
            channels:
            - name: v4.4
      ~~~
          2.  Run the following command:
      ~~~
      /oc-mirror --config=./imageset-config.yaml docker://Internal-registry:Port --rebuild-catalogs
      ~~~
          3. Create catalogsourcepod under openshift-marketplace namespace:
      ~~~
       cat oc-mirror-workspace/results-1721222945/catalogSource-cs-ibm-operator-catalog.yaml
      apiVersion: operators.coreos.com/v1alpha1
      kind: CatalogSource
      metadata:
        name: cs-ibm-operator-catalog
        namespace: openshift-marketplace
      spec:
        image: Internal-registry:Port/cpopen/ibm-operator-catalog:v1.22
        sourceType: grpc
      ~~~
          

      Actual results:

      
      catalog pod is crashing with the following error:
      ~~~
      time="2024-07-10T13:43:07Z" level=info msg="starting pprof endpoint" address="localhost:6060"
      time="2024-07-10T13:43:08Z" level=fatal msg="cache requires rebuild: cache reports digest as \"e891bfd5a4cb5702\", but computed digest is \"1922475dc0ee190c\""
      ~~~
      
          

      Expected results:

      The pod should run without any issue. 
      
          

      Additional info:

      1. The issue is reproducible with the OCP 4.14.14 and OCP 4.14.29
      2. Customer is already using oc-mirror 4.16:
      ~~~
      ./oc-mirror version
      WARNING: This version information is deprecated and will be replaced with the output from --short. Use --output=yaml|json to get the full version.
      Client Version: version.Info{Major:"", Minor:"", GitVersion:"4.16.0-202407030803.p0.g394b1f8.assembly.stream.el9-394b1f8", GitCommit:"394b1f814f794f4f01f473212c9a7695726020bf", GitTreeState:"clean", BuildDate:"2024-07-03T10:18:49Z", GoVersion:"go1.21.11 (Red Hat 1.21.11-1.module+el8.10.0+21986+2112108a) X:strictfipsruntime", Compiler:"gc", Platform:"linux/amd64"}
      ~~~
      3. Customer tried with workaround described in the KB[1]: https://access.redhat.com/solutions/7006771 but no luck
      4. Customer also tried to set the OPM_BINARY, but didn't work. They download  OPM with respective arch: https://github.com/operator-framework/operator-registry/releases rename the downloaded binary as opm and set below variable before executing oc-mirror
      OPM_BINARY=/path/to/opm
      
          

            luzuccar@redhat.com Luigi Mario Zuccarelli
            openshift-crt-jira-prow OpenShift Prow Bot
            ying zhou ying zhou
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

              Created:
              Updated:
              Resolved: