Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-17546

pod catalogsource generated by oc-mirror will crashloopBackOff randomly

XMLWordPrintable

    • Important
    • No
    • CFE Sprint 241, CFE Sprint 242
    • 2
    • Rejected
    • False
    • Hide

      None

      Show
      None
    • NA

      Description of problem:

      By mirror2disk + disk2mirror workflow or mirror2mirror workflow, the catalogsource pod sometimes will crashloopBackOff and cache integrity not verified
      

      Version-Release number of selected component (if applicable):

       

      How reproducible:

      Sometimes 

      Steps to Reproduce:

      1.  Clone the repo to https://github.com/openshift/oc-mirror/pull/651 and build oc-mirror;
      2. Copy the catalog index to localhost ;
      `skopeo copy --all  --format oci  docker://registry.redhat.io/redhat/redhat-operator-index:v4.12 oci:///home/ocmirrortest/test825/oci-412-multi  --remove-signatures`
      3.  Run the oc-mirror command :
      cat config-412.yaml 
      apiVersion: mirror.openshift.io/v1alpha2
      kind: ImageSetConfiguration
      archiveSize: 16
      mirror:
        operators:
        - catalog: oci:///home/ocmirrortest/test825/oci-412-multi
          full: false # only mirror the latest versions
          packages:
          - name: cluster-logging
        - catalog: registry.redhat.io/redhat/community-operator-index:v4.12
          full: false # only mirror the latest versions
          packages:
          - name: namespace-configuration-operator
      4.  `oc-mirror --config config-412.yaml  file://`
      5.   `oc-mirror --from mirror_seq1_000000.tar docker://ec2-3-23-92-68.us-east-2.compute.amazonaws.com:5000/412oci --dest-skip-tls`
      6.   `oc create -f oc-mirror-workspace/results-1691638259/catalogSource-cs-community-operator-index.yaml`
             `oc create -f oc-mirror-workspace/results-1691638259/catalogSource-cs-ocmirrortest-test825-oci-412-multi.yaml `
             `oc create -f oc-mirror-workspace/results-1691638259/imageContentSourcePolicy.yaml`
      

      Actual results:

      [root@preserve-fedora36 CFE-825]# oc get pod
      NAME                                          READY   STATUS             RESTARTS      AGE
      cs-community-operator-index-q7grd             0/1     CrashLoopBackOff   5 (34s ago)   3m43s
      cs-ocmirrortest-test825-oci-412-multi-vqtf4   0/1     CrashLoopBackOff   5 (25s ago)   3m32s
      marketplace-operator-7db46798cd-jzjl7         1/1     Running            0             14m
      qe-app-registry-b7jhx                         1/1     Running            0             14m
      qe-app-registry-r7nx4                         1/1     Terminating        0             12s
      
      [root@preserve-fedora36 CFE-825]# oc logs pod/cs-community-operator-index-q7grd
      time="2023-08-10T03:39:37Z" level=fatal msg="cache requires rebuild: cache reports digest as \"6e9f392faaa4058b\", but computed digest is \"1964c079afb52b4f\""
      [root@preserve-fedora36 CFE-825]# oc logs pod/cs-ocmirrortest-test825-oci-412-multi-vqtf4 
      time="2023-08-10T03:39:46Z" level=fatal msg="cache requires rebuild: cache reports digest as \"6302a81eb16ce9f1\", but computed digest is \"8224901ed28d8455\""
       

      Expected results:

      catlogsource pod running well 

        1. must-gather-cluster-ko.tar
          25.34 MB
        2. must-gather-cluster-ok.tar
          27.03 MB
        3. cache-when-ok.tar
          2.76 MB
        4. cache-when-ko.tar
          2.76 MB

              rh-ee-aguidi Alex Guidi
              yinzhou@redhat.com ying zhou
              ying zhou ying zhou
              Sherine Khoury
              Votes:
              2 Vote for this issue
              Watchers:
              9 Start watching this issue

                Created:
                Updated:
                Resolved: