Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-48430

oc-mirror v2 automatically deleting the manifests which are generated under working-dir/cluster-resources while generating delete-images.yaml

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done-Errata
    • Icon: Major Major
    • None
    • 4.17
    • oc-mirror
    • None
    • CLID Sprint 265
    • 1
    • False
    • Hide

      None

      Show
      None
    • Hide
      * Previously, when using `oc-mirror --v2 delete --generate`command, the contents of the working-dir/cluster-resources folder were cleared. With this fix, the working-dir/cluster-resources folder is not cleaned when the delete feature is used. (link:https://issues.redhat.com/browse/OCPBUGS-48430[*OCPBUGS-48430*])
      Show
      * Previously, when using `oc-mirror --v2 delete --generate`command, the contents of the working-dir/cluster-resources folder were cleared. With this fix, the working-dir/cluster-resources folder is not cleaned when the delete feature is used. (link: https://issues.redhat.com/browse/OCPBUGS-48430 [* OCPBUGS-48430 *])
    • Bug Fix
    • Done

      This is a clone of issue OCPBUGS-44714. The following is the description of the original issue:
      —
      Description of problem:

      While generating delete-images.yaml for the pruning of images using oc-mirror v2, the manifest which are generated under working-dir/cluster-resources (IDMS,ITMS etc) are getting deleted automatically   

      Version-Release number of selected component (if applicable):

      4.17

      How reproducible:

      100% reproducible

      Steps to Reproduce:

      1- Create a DeleteImageSetConfiguration file like below

      apiVersion: mirror.openshift.io/v2alpha1
kind: DeleteImageSetConfiguration
delete:
  platform:
    channels:
    - name: stable-4.17
      minVersion: 4.17.3
      maxVersion: 4.17.3
  operators:
    - catalog: registry.redhat.io/redhat/redhat-operator-index:v4.17
      packages:
       - name: aws-load-balancer-operator
       - name: node-observability-operator
       - name: 3scale-operator
  additionalImages:
   - name: registry.redhat.io/ubi8/ubi:latest
   - name: registry.redhat.io/ubi9/ubi@sha256:20f695d2a91352d4eaa25107535126727b5945bff38ed36a3e59590f495046f0

      2- ensure that the manifest are generated by the oc-mirror are present in the working-dir/cluster-resources 

       

       ls -lrth /opt/417ocmirror/working-dir/cluster-resources/
total 16K

-rw-r--r--. 1 root root 491 Nov 18 21:57 itms-oc-mirror.yaml
-rw-r--r--. 1 root root 958 Nov 18 21:57 idms-oc-mirror.yaml
-rw-r--r--. 1 root root 322 Nov 18 21:57 updateService.yaml
-rw-r--r--. 1 root root 268 Nov 18 21:57 cs-redhat-operator-index-v4-17.yaml

      3- Generate the delete-images.yaml using below command

       

      ./oc-mirror  delete --config ./deleteimageset.yaml --workspace file:///opt/417ocmirror --v2 --generate docker://bastionmirror.amuhamme.upi:8443/417images2024/11/18 23:53:12  [WARN]   : ⚠️  --v2 flag identified, flow redirected to the oc-mirror v2 version. This is Tech Preview, it is still under development and it is not production ready.
2024/11/18 23:53:12  [INFO]   : 👋 Hello, welcome to oc-mirror
2024/11/18 23:53:12  [INFO]   : ⚙️  setting up the environment for you...
2024/11/18 23:53:12  [INFO]   : 🔀 workflow mode: diskToMirror / delete
2024/11/18 23:53:12  [INFO]   : 🕵️  going to discover the necessary images...
2024/11/18 23:53:12  [INFO]   : 🔍 collecting release images...
2024/11/18 23:53:12  [INFO]   : 🔍 collecting operator images...
2024/11/18 23:53:13  [INFO]   : 🔍 collecting additional images...
2024/11/18 23:53:13  [INFO]   : 📄 Generating delete file...
2024/11/18 23:53:13  [INFO]   : /opt/417ocmirror/working-dir/delete file created
2024/11/18 23:53:13  [INFO]   : delete time     : 712.42082ms
2024/11/18 23:53:13  [INFO]   : 👋 Goodbye, thank you for using oc-mirror

      4- Verify after generating the delete-images.yaml  the manifests present in the working-dir/cluster-resources/ got deleted.

       

      # ls -lrth /opt/417ocmirror/working-dir/cluster-resources/
total 0

# ls -lrth /opt/417ocmirror/working-dir/delete
total 72K
-rwxr-xr-x. 1 root root 65K Nov 18 23:53 delete-images.yaml
-rwxr-xr-x. 1 root root 617 Nov 18 23:53 delete-imageset-config.yaml

      Actual results:

      Generating delete-images.yaml is deleting the manifest under working-dir/cluster-resources/

      Expected results:

      Generating delete-images.yaml  should not delete the manifest under working-dir/cluster-resources/

      Additional info:

            [OCPBUGS-48430] oc-mirror v2 automatically deleting the manifests which are generated under working-dir/cluster-resources while generating delete-images.yaml

            Errata Tool added a comment -

            Since the problem described in this issue should be resolved in a recent advisory, it has been closed.

            For information on the advisory (Important: OpenShift Container Platform 4.18.1 bug fix and security update), and where to find the updated files, follow the link below.

            If the solution does not work for you, open a new bug report.
            https://access.redhat.com/errata/RHSA-2024:6122

            Errata Tool added a comment - Since the problem described in this issue should be resolved in a recent advisory, it has been closed. For information on the advisory (Important: OpenShift Container Platform 4.18.1 bug fix and security update), and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2024:6122

            Ying Zhou added a comment -

            tested with latest version, can't reproduce the issue now:

            [cloud-user@preserve-oc-mirror-vm yinzhou]$ ll out/working-dir/cluster-resources/
total 24
-rw-r--r--. 1 cloud-user cloud-user  236 Jan 20 07:09 cs-redhat-operator-index-latest.yaml
-rw-r--r--. 1 cloud-user cloud-user  241 Jan 20 07:09 cs-redhat-operator-index-v4-15.yaml
-rw-r--r--. 1 cloud-user cloud-user  819 Jan 20 07:09 idms-oc-mirror.yaml
-rw-r--r--. 1 cloud-user cloud-user  452 Jan 20 07:09 itms-oc-mirror.yaml
-rw-r--r--. 1 cloud-user cloud-user 1478 Jan 20 07:09 signature-configmap.json
-rw-r--r--. 1 cloud-user cloud-user 1464 Jan 20 07:09 signature-configmap.yaml
[cloud-user@preserve-oc-mirror-vm yinzhou]$ ./oc-mirror -c config-44799-d.yaml delete --generate --workspace file://out --v2 docker://localhost:5000 --dest-tls-verify=false2025/01/20 07:09:58  [INFO]   : 👋 Hello, welcome to oc-mirror
2025/01/20 07:09:58  [INFO]   : ⚙️  setting up the environment for you...
2025/01/20 07:09:58  [INFO]   : 🔀 workflow mode: diskToMirror / delete
2025/01/20 07:09:58  [INFO]   : 🕵  going to discover the necessary images...
2025/01/20 07:09:58  [INFO]   : 🔍 collecting release images...
2025/01/20 07:09:58  [INFO]   : 🔍 collecting operator images...
 ✓   (1s) Collecting catalog oci:///home/cloud-user/redhat-operator-index 
 ✓   (0s) Collecting catalog registry.redhat.io/redhat/redhat-operator-index:v4.15 
2025/01/20 07:10:00  [INFO]   : 🔍 collecting additional images...
2025/01/20 07:10:00  [INFO]   : 🔍 collecting helm images...
2025/01/20 07:10:00  [INFO]   : 📄 Generating delete file...
2025/01/20 07:10:00  [INFO]   : out/working-dir/delete file created
2025/01/20 07:10:00  [INFO]   : delete time     : 2.333376953s
2025/01/20 07:10:00  [INFO]   : 👋 Goodbye, thank you for using oc-mirror
[cloud-user@preserve-oc-mirror-vm yinzhou]$ ll out/working-dir/cluster-resources/
total 24
-rw-r--r--. 1 cloud-user cloud-user  236 Jan 20 07:09 cs-redhat-operator-index-latest.yaml
-rw-r--r--. 1 cloud-user cloud-user  241 Jan 20 07:09 cs-redhat-operator-index-v4-15.yaml
-rw-r--r--. 1 cloud-user cloud-user  819 Jan 20 07:09 idms-oc-mirror.yaml
-rw-r--r--. 1 cloud-user cloud-user  452 Jan 20 07:09 itms-oc-mirror.yaml
-rw-r--r--. 1 cloud-user cloud-user 1478 Jan 20 07:09 signature-configmap.json
-rw-r--r--. 1 cloud-user cloud-user 1464 Jan 20 07:09 signature-configmap.yaml
[cloud-user@preserve-oc-mirror-vm yinzhou]$ ./oc-mirror version
W0120 07:10:09.834356  570053 mirror.go:102] ⚠️  oc-mirror v1 is deprecated (starting in 4.18 release) and will be removed in a future release - please migrate to oc-mirror --v2WARNING: This version information is deprecated and will be replaced with the output from --short. Use --output=yaml|json to get the full version.
Client Version: version.Info{Major:"", Minor:"", GitVersion:"4.18.0-202501171402.p0.g9dde8cc.assembly.stream.el9-9dde8cc", GitCommit:"9dde8cc20bd64043ebec9f4dcf602b6e58d2de03", GitTreeState:"clean", BuildDate:"2025-01-17T15:15:37Z", GoVersion:"go1.22.9 (Red Hat 1.22.9-2.el9_5) X:strictfipsruntime", Compiler:"gc", Platform:"linux/amd64"}

            Ying Zhou added a comment - tested with latest version, can't reproduce the issue now: [cloud-user@preserve-oc-mirror-vm yinzhou]$ ll out/working-dir/cluster-resources/ total 24 -rw-r--r--. 1 cloud-user cloud-user  236 Jan 20 07:09 cs-redhat- operator -index-latest.yaml -rw-r--r--. 1 cloud-user cloud-user  241 Jan 20 07:09 cs-redhat- operator -index-v4-15.yaml -rw-r--r--. 1 cloud-user cloud-user  819 Jan 20 07:09 idms-oc-mirror.yaml -rw-r--r--. 1 cloud-user cloud-user  452 Jan 20 07:09 itms-oc-mirror.yaml -rw-r--r--. 1 cloud-user cloud-user 1478 Jan 20 07:09 signature-configmap.json -rw-r--r--. 1 cloud-user cloud-user 1464 Jan 20 07:09 signature-configmap.yaml [cloud-user@preserve-oc-mirror-vm yinzhou]$ ./oc-mirror -c config-44799-d.yaml delete --generate --workspace file: //out --v2 docker://localhost:5000 --dest-tls-verify=false2025/01/20 07:09:58  [INFO]   : 👋 Hello, welcome to oc-mirror 2025/01/20 07:09:58  [INFO]   : ⚙️  setting up the environment for you... 2025/01/20 07:09:58  [INFO]   : 🔀 workflow mode: diskToMirror / delete 2025/01/20 07:09:58  [INFO]   : 🕵  going to discover the necessary images... 2025/01/20 07:09:58  [INFO]   : 🔍 collecting release images... 2025/01/20 07:09:58  [INFO]   : 🔍 collecting operator images...  ✓   (1s) Collecting catalog oci: ///home/cloud-user/redhat- operator -index   ✓   (0s) Collecting catalog registry.redhat.io/redhat/redhat- operator -index:v4.15  2025/01/20 07:10:00  [INFO]   : 🔍 collecting additional images... 2025/01/20 07:10:00  [INFO]   : 🔍 collecting helm images... 2025/01/20 07:10:00  [INFO]   : 📄 Generating delete file... 2025/01/20 07:10:00  [INFO]   : out/working-dir/delete file created 2025/01/20 07:10:00  [INFO]   : delete time     : 2.333376953s 2025/01/20 07:10:00  [INFO]   : 👋 Goodbye, thank you for using oc-mirror [cloud-user@preserve-oc-mirror-vm yinzhou]$ ll out/working-dir/cluster-resources/ total 24 -rw-r--r--. 1 cloud-user cloud-user  236 Jan 20 07:09 cs-redhat- operator -index-latest.yaml -rw-r--r--. 1 cloud-user cloud-user  241 Jan 20 07:09 cs-redhat- operator -index-v4-15.yaml -rw-r--r--. 1 cloud-user cloud-user  819 Jan 20 07:09 idms-oc-mirror.yaml -rw-r--r--. 1 cloud-user cloud-user  452 Jan 20 07:09 itms-oc-mirror.yaml -rw-r--r--. 1 cloud-user cloud-user 1478 Jan 20 07:09 signature-configmap.json -rw-r--r--. 1 cloud-user cloud-user 1464 Jan 20 07:09 signature-configmap.yaml [cloud-user@preserve-oc-mirror-vm yinzhou]$ ./oc-mirror version W0120 07:10:09.834356  570053 mirror.go:102] ⚠️  oc-mirror v1 is deprecated (starting in 4.18 release) and will be removed in a future release - please migrate to oc-mirror --v2WARNING: This version information is deprecated and will be replaced with the output from -- short . Use --output=yaml|json to get the full version. Client Version: version.Info{Major: "", Minor:" ", GitVersion:" 4.18.0-202501171402.p0.g9dde8cc.assembly.stream.el9-9dde8cc ", GitCommit:" 9dde8cc20bd64043ebec9f4dcf602b6e58d2de03 ", GitTreeState:" clean ", BuildDate:" 2025-01-17T15:15:37Z ", GoVersion:" go1.22.9 (Red Hat 1.22.9-2.el9_5) X:strictfipsruntime ", Compiler :" gc ", Platform:" linux/amd64"}

            Rama Kasturi Narra added a comment - - edited

            Reproduced and verified bug via pre-merge test and i see that issue has been fixed

            steps used to reproduce the issue:
            ===========================
            1. perform mirror2disk + disk2mirror using the imageSetConfig in the bug
            2. Now run command to generate the delete-images.yaml file
            3. User can see that resources under cluster-resources directory no longer exist.

            with the fix PR:
            ==================
            1. perform mirror2disk + disk2mirror using the imageSetConfig given below
            2. Now run command to generate the delete-images.yaml file
            3. user can see that none of the files under the working-dir are deleted, plus a delete dir gets created as well under the working dir

            [fedora@knarra-fedora ~]$ ./oc-mirror version
W0116 12:35:39.844322  128855 mirror.go:102] 

⚠️  oc-mirror v1 is deprecated (starting in 4.18 release) and will be removed in a future release - please migrate to oc-mirror --v2

WARNING: This version information is deprecated and will be replaced with the output from --short. Use --output=yaml|json to get the full version.
Client Version: version.Info{Major:"", Minor:"", GitVersion:"v0.2.0-alpha.1-340-g3474130", GitCommit:"34741306", GitTreeState:"clean", BuildDate:"2025-01-16T09:57:17Z", GoVersion:"go1.23.4", Compiler:"gc", Platform:"linux/amd64"}

            [fedora@knarra-fedora ~]$ ls -l bug1/working-dir/
total 0
drwxr-xr-x. 1 fedora fedora   0 Jan 16 11:07 cluster-resources
drwxr-xr-x. 1 fedora fedora  26 Jan 16 11:08 helm
drwxr-xr-x. 1 fedora fedora   0 Jan 16 11:07 hold-operator
drwxr-xr-x. 1 fedora fedora  64 Jan 16 11:07 hold-release
drwxr-xr-x. 1 fedora fedora  50 Jan 16 11:07 logs
drwxr-xr-x. 1 fedora fedora  42 Jan 16 11:07 operator-catalogs
drwxr-xr-x. 1 fedora fedora  22 Jan 16 11:07 release-images
drwxr-xr-x. 1 fedora fedora 170 Jan 16 11:07 signatures


[fedora@knarra-fedora ~]$ ls -l bug1/working-dir/
total 0
drwxr-xr-x. 1 fedora fedora 242 Jan 16 12:20 cluster-resources
drwxr-xr-x. 1 fedora fedora  26 Jan 16 11:08 helm
drwxr-xr-x. 1 fedora fedora   0 Jan 16 11:07 hold-operator
drwxr-xr-x. 1 fedora fedora  64 Jan 16 11:07 hold-release
drwxr-xr-x. 1 fedora fedora  50 Jan 16 12:17 logs
drwxr-xr-x. 1 fedora fedora  42 Jan 16 11:07 operator-catalogs
drwxr-xr-x. 1 fedora fedora  22 Jan 16 11:07 release-images
drwxr-xr-x. 1 fedora fedora 170 Jan 16 11:07 signatures


[fedora@knarra-fedora ~]$ ls -l bug1/working-dir/
total 0
drwxr-xr-x. 1 fedora fedora 242 Jan 16 12:20 cluster-resources
drwxr-xr-x. 1 fedora fedora  90 Jan 16 12:26 delete
drwxr-xr-x. 1 fedora fedora  26 Jan 16 11:08 helm
drwxr-xr-x. 1 fedora fedora   0 Jan 16 11:07 hold-operator
drwxr-xr-x. 1 fedora fedora  64 Jan 16 11:07 hold-release
drwxr-xr-x. 1 fedora fedora  50 Jan 16 12:26 logs
drwxr-xr-x. 1 fedora fedora  42 Jan 16 11:07 operator-catalogs
drwxr-xr-x. 1 fedora fedora  22 Jan 16 11:07 release-images
drwxr-xr-x. 1 fedora fedora 170 Jan 16 11:07 signatures

            [fedora@knarra-fedora ~]$ cat /tmp/isc1.yaml 
apiVersion: mirror.openshift.io/v2alpha1
kind: ImageSetConfiguration
mirror:
  platform:
    channels:
    - name: stable-4.17
      minVersion: 4.17.3
      maxVersion: 4.17.3
  operators:
    - catalog: registry.redhat.io/redhat/redhat-operator-index:v4.17
      packages:
       - name: aws-load-balancer-operator
       - name: node-observability-operator
       - name: 3scale-operator
  additionalImages:
   - name: registry.redhat.io/ubi8/ubi:latest
   - name: registry.redhat.io/ubi9/ubi@sha256:20f695d2a91352d4eaa25107535126727b5945bff38ed36a3e59590f495046f0

            Rama Kasturi Narra added a comment - - edited Reproduced and verified bug via pre-merge test and i see that issue has been fixed steps used to reproduce the issue: =========================== 1. perform mirror2disk + disk2mirror using the imageSetConfig in the bug 2. Now run command to generate the delete-images.yaml file 3. User can see that resources under cluster-resources directory no longer exist. with the fix PR: ================== 1. perform mirror2disk + disk2mirror using the imageSetConfig given below 2. Now run command to generate the delete-images.yaml file 3. user can see that none of the files under the working-dir are deleted, plus a delete dir gets created as well under the working dir [fedora@knarra-fedora ~]$ ./oc-mirror version W0116 12:35:39.844322 128855 mirror.go:102] ⚠️ oc-mirror v1 is deprecated (starting in 4.18 release) and will be removed in a future release - please migrate to oc-mirror --v2 WARNING: This version information is deprecated and will be replaced with the output from -- short . Use --output=yaml|json to get the full version. Client Version: version.Info{Major: "", Minor:" ", GitVersion:" v0.2.0-alpha.1-340-g3474130 ", GitCommit:" 34741306 ", GitTreeState:" clean ", BuildDate:" 2025-01-16T09:57:17Z ", GoVersion:" go1.23.4 ", Compiler :" gc ", Platform:" linux/amd64"} [fedora@knarra-fedora ~]$ ls -l bug1/working-dir/ total 0 drwxr-xr-x. 1 fedora fedora 0 Jan 16 11:07 cluster-resources drwxr-xr-x. 1 fedora fedora 26 Jan 16 11:08 helm drwxr-xr-x. 1 fedora fedora 0 Jan 16 11:07 hold- operator drwxr-xr-x. 1 fedora fedora 64 Jan 16 11:07 hold-release drwxr-xr-x. 1 fedora fedora 50 Jan 16 11:07 logs drwxr-xr-x. 1 fedora fedora 42 Jan 16 11:07 operator -catalogs drwxr-xr-x. 1 fedora fedora 22 Jan 16 11:07 release-images drwxr-xr-x. 1 fedora fedora 170 Jan 16 11:07 signatures [fedora@knarra-fedora ~]$ ls -l bug1/working-dir/ total 0 drwxr-xr-x. 1 fedora fedora 242 Jan 16 12:20 cluster-resources drwxr-xr-x. 1 fedora fedora 26 Jan 16 11:08 helm drwxr-xr-x. 1 fedora fedora 0 Jan 16 11:07 hold- operator drwxr-xr-x. 1 fedora fedora 64 Jan 16 11:07 hold-release drwxr-xr-x. 1 fedora fedora 50 Jan 16 12:17 logs drwxr-xr-x. 1 fedora fedora 42 Jan 16 11:07 operator -catalogs drwxr-xr-x. 1 fedora fedora 22 Jan 16 11:07 release-images drwxr-xr-x. 1 fedora fedora 170 Jan 16 11:07 signatures [fedora@knarra-fedora ~]$ ls -l bug1/working-dir/ total 0 drwxr-xr-x. 1 fedora fedora 242 Jan 16 12:20 cluster-resources drwxr-xr-x. 1 fedora fedora 90 Jan 16 12:26 delete drwxr-xr-x. 1 fedora fedora 26 Jan 16 11:08 helm drwxr-xr-x. 1 fedora fedora 0 Jan 16 11:07 hold- operator drwxr-xr-x. 1 fedora fedora 64 Jan 16 11:07 hold-release drwxr-xr-x. 1 fedora fedora 50 Jan 16 12:26 logs drwxr-xr-x. 1 fedora fedora 42 Jan 16 11:07 operator -catalogs drwxr-xr-x. 1 fedora fedora 22 Jan 16 11:07 release-images drwxr-xr-x. 1 fedora fedora 170 Jan 16 11:07 signatures [fedora@knarra-fedora ~]$ cat /tmp/isc1.yaml apiVersion: mirror.openshift.io/v2alpha1 kind: ImageSetConfiguration mirror: platform: channels: - name: stable-4.17 minVersion: 4.17.3 maxVersion: 4.17.3 operators: - catalog: registry.redhat.io/redhat/redhat- operator -index:v4.17 packages: - name: aws-load-balancer- operator - name: node-observability- operator - name: 3scale- operator additionalImages: - name: registry.redhat.io/ubi8/ubi:latest - name: registry.redhat.io/ubi9/ubi@sha256:20f695d2a91352d4eaa25107535126727b5945bff38ed36a3e59590f495046f0

              rdossant Rafael Fonseca dos Santos
              openshift-crt-jira-prow OpenShift Prow Bot
              Rama Kasturi Narra Rama Kasturi Narra
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated:
                Resolved: