Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-23327

file path used for oci images can result in an error

    XMLWordPrintable

Details

    • No
    • CLID Sprint 246
    • 1
    • Rejected
    • False
    • Hide

      None

      Show
      None
    • Hide
      When using oc-mirror for mirroring local oci operator catalogs that are located in a hidden folder, oc-mirror was failing with an error similar to:
      error: ".hidden_folder/data/publish/latest/catalog-oci/manifest-list/kubebuilder/kube-rbac-proxy@sha256:db06cc4c084dd0253134f156dddaaf53ef1c3fb3cc809e5d81711baa4029ea4c" is not a valid image reference: invalid reference format

      This is now fixed by calculating the references to the images included in a local oci catalog differently, so that the path to the local catalog does not cause problems during mirroring.
      Show
      When using oc-mirror for mirroring local oci operator catalogs that are located in a hidden folder, oc-mirror was failing with an error similar to: error: ".hidden_folder/data/publish/latest/catalog-oci/manifest-list/kubebuilder/kube-rbac-proxy@sha256:db06cc4c084dd0253134f156dddaaf53ef1c3fb3cc809e5d81711baa4029ea4c" is not a valid image reference: invalid reference format This is now fixed by calculating the references to the images included in a local oci catalog differently, so that the path to the local catalog does not cause problems during mirroring.
    • Bug Fix
    • Done

    Description

      Description of problem:

      When executing oc mirror using an oci path, you can end up with in an error state when the destination is a file://<path> destination (i.e. mirror to disk).
      

      Version-Release number of selected component (if applicable):

      4.14.2
      

      How reproducible:

      always
      

      Steps to Reproduce:

      
      At IBM we use the ibm-pak tool to generate a OCI catalog, but this bug is reproducible using a simple skopeo copy. Once you've copied the image locally you can move it around using file system copy commands to test this in different ways.
      
      1. Make a directory structure like this to simulate how ibm-pak creates its own catalogs. The problem seems to be related to the path you use, so this represents the failure case:
      
      mkdir -p /root/.ibm-pak/data/publish/latest/catalog-oci/manifest-list
      
      2. make a location where the local storage will live:
      
      mkdir -p /root/.ibm-pak/oc-mirror-storage
      
      3. Next, copy the image locally using skopeo:
      
      skopeo copy docker://icr.io/cpopen/ibm-zcon-zosconnect-catalog@sha256:8d28189637b53feb648baa6d7e3dd71935656a41fd8673292163dd750ef91eec oci:///root/.ibm-pak/data/publish/latest/catalog-oci/manifest-list --all --format v2s2
      
      4. You can copy the OCI catalog content to a location where things will work properly so you can see a working example:
      
      cp -r /root/.ibm-pak/data/publish/latest/catalog-oci/manifest-list /root/ibm-zcon-zosconnect-catalog
      
      5. You'll need an ISC... I've included both the oci references in the example (the commented out one works, but the oci:///root/.ibm-pak/data/publish/latest/catalog-oci/manifest-list reference fails).
      
      kind: ImageSetConfiguration
      apiVersion: mirror.openshift.io/v1alpha2
      mirror:
        operators:
        - catalog: oci:///root/.ibm-pak/data/publish/latest/catalog-oci/manifest-list
        #- catalog: oci:///root/ibm-zcon-zosconnect-catalog
          packages:
          - name: ibm-zcon-zosconnect
            channels:
            - name: v1.0
          full: true
          targetTag: 27ba8e
          targetCatalog: ibm-catalog
      storageConfig:
        local:
          path: /root/.ibm-pak/oc-mirror-storage
      
      6. run oc mirror (remember the ISC has oci refs for good and bad scenarios). You may want to change your working directory to different locations between running the good/bad examples.
      
      oc mirror --config /root/.ibm-pak/data/publish/latest/image-set-config.yaml "file://zcon --dest-skip-tls --max-per-registry=6
      
      
      
      
      

      Actual results:

      
      Logging to .oc-mirror.log
      Found: zcon/oc-mirror-workspace/src/publish
      Found: zcon/oc-mirror-workspace/src/v2
      Found: zcon/oc-mirror-workspace/src/charts
      Found: zcon/oc-mirror-workspace/src/release-signatures
      error: ".ibm-pak/data/publish/latest/catalog-oci/manifest-list/kubebuilder/kube-rbac-proxy@sha256:db06cc4c084dd0253134f156dddaaf53ef1c3fb3cc809e5d81711baa4029ea4c" is not a valid image reference: invalid reference format
      
      
      

      Expected results:

      
      Simple example where things were working with the oci:///root/ibm-zcon-zosconnect-catalog reference (this was executed in the same workspace so no new images were detected).
      
      Logging to .oc-mirror.log
      Found: zcon/oc-mirror-workspace/src/publish
      Found: zcon/oc-mirror-workspace/src/v2
      Found: zcon/oc-mirror-workspace/src/charts
      Found: zcon/oc-mirror-workspace/src/release-signatures
      3 related images processed in 668.063974ms
      Writing image mapping to zcon/oc-mirror-workspace/operators.1700092336/manifests-ibm-zcon-zosconnect-catalog/mapping.txt
      No new images detected, process stopping
      
      

      Additional info:

      
      I debugged the error that happened and captured one of the instances where the ParseReference call fails. This is only for reference to help narrow down the issue.
      
      github.com/openshift/oc/pkg/cli/image/imagesource.ParseReference (/root/go/src/openshift/oc-mirror/vendor/github.com/openshift/oc/pkg/cli/image/imagesource/reference.go:111)
      github.com/openshift/oc-mirror/pkg/image.ParseReference (/root/go/src/openshift/oc-mirror/pkg/image/image.go:79)
      github.com/openshift/oc-mirror/pkg/cli/mirror.(*MirrorOptions).addRelatedImageToMapping (/root/go/src/openshift/oc-mirror/pkg/cli/mirror/fbc_operators.go:194)
      github.com/openshift/oc-mirror/pkg/cli/mirror.(*OperatorOptions).plan.func3 (/root/go/src/openshift/oc-mirror/pkg/cli/mirror/operator.go:575)
      golang.org/x/sync/errgroup.(*Group).Go.func1 (/root/go/src/openshift/oc-mirror/vendor/golang.org/x/sync/errgroup/errgroup.go:75)
      runtime.goexit (/usr/local/go/src/runtime/asm_amd64.s:1594)
      
      Also, I wanted to point out that because we use a period in the path (i.e. .ibm-pak) I wonder if that's causing the issue? This is just a guess and something to consider. *FOLLOWUP* ... I just removed the period from ".ibm-pak" and that seemed to make the error go away.
      
      

      Attachments

        Issue Links

          Activity

            People

              skhoury@redhat.com Sherine Khoury
              jhunkins@redhat.com John Hunkins
              ying zhou ying zhou
              Votes:
              1 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated: