Uploaded image for project: 'OpenShift Virtualization'
  1. OpenShift Virtualization
  2. CNV-68996

[Multiarch] Arch-specific DataSources (arm64) persist after removing arm64 nodes

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Major Major
    • CNV v4.20.z
    • CNV v4.20.0
    • Storage Platform
    • None
    • Quality / Stability / Reliability
    • 0.42
    • False
    • Hide

      None

      Show
      None
    • False
    • None
    • Release Notes
    • Hide
      Known issue: duplicate boot sources after enabling enableMultiArchBootImageImport must be deleted

      If you enable spec.featureGates.enableMultiArchBootImageImport after boot sources were imported, OpenShift Virtualization recreates the boot source import resources using architecture-suffixed names. For example: fedora-amd64, fedora-arm64, fedora-s390x depending on what the cluster supports. The older, non-suffixed boot source resources are not automatically removed and remain in the openshift-virtualization-os-images namespace. This can result in duplicate boot sources displaying in the Web Console and the CLI. Storage usage can potentially increase because extra PVCs or VolumeSnapshots are still present.

      Workaround: Delete the leftover boot source resources. First, identify the DataSource objects that are currently active. These are the DataSource objects that resolve to the PVC/VolumeSnapshot that you want to keep. Next, delete the older/stale DataSource objects and the PVCs/VolumeSnapshots that they reference.
      Show
      Known issue: duplicate boot sources after enabling enableMultiArchBootImageImport must be deleted If you enable spec.featureGates.enableMultiArchBootImageImport after boot sources were imported, OpenShift Virtualization recreates the boot source import resources using architecture-suffixed names. For example: fedora-amd64, fedora-arm64, fedora-s390x depending on what the cluster supports. The older, non-suffixed boot source resources are not automatically removed and remain in the openshift-virtualization-os-images namespace. This can result in duplicate boot sources displaying in the Web Console and the CLI. Storage usage can potentially increase because extra PVCs or VolumeSnapshots are still present. Workaround: Delete the leftover boot source resources. First, identify the DataSource objects that are currently active. These are the DataSource objects that resolve to the PVC/VolumeSnapshot that you want to keep. Next, delete the older/stale DataSource objects and the PVCs/VolumeSnapshots that they reference.
    • Known Issue
    • Proposed
    • Important
    • None

      Description of problem:

      We have a cluster with arm64 and amd64. When i delete arm64 nodes , I see datasources still stay which are of no use since we don't have an arm node to utilise them 

Test case 2: delete single node arm 
 
$  oc get nodes -o json | jq -r '.items[] | [.metadata.name,
  (if .metadata.labels["node-role.kubernetes.io/master"] != null then "master"
   elif .metadata.labels["node-role.kubernetes.io/worker"] != null then "worker"
   else "other" end),
  .status.nodeInfo.architecture] | @tsv'
ip-10-0-11-154.us-east-2.compute.internal    master    amd64
ip-10-0-38-209.us-east-2.compute.internal    master    amd64
ip-10-0-61-33.us-east-2.compute.internal    worker    amd64
ip-10-0-93-238.us-east-2.compute.internal    master    amd64
 
 
$ oc get datasources -A
NAMESPACE                            NAME                    AGE
openshift-virtualization-os-images   centos-stream10         8h
openshift-virtualization-os-images   centos-stream10-amd64   8h
openshift-virtualization-os-images   centos-stream10-arm64   8h
openshift-virtualization-os-images   centos-stream9          8h
openshift-virtualization-os-images   centos-stream9-amd64    8h
openshift-virtualization-os-images   centos-stream9-arm64    119s
openshift-virtualization-os-images   fedora                  8h
openshift-virtualization-os-images   fedora-amd64            8h
openshift-virtualization-os-images   fedora-arm64            119s
openshift-virtualization-os-images   rhel10                  8h
openshift-virtualization-os-images   rhel10-amd64            8h
openshift-virtualization-os-images   rhel10-arm64            8h
openshift-virtualization-os-images   rhel7                   8h
openshift-virtualization-os-images   rhel7-amd64             8h
openshift-virtualization-os-images   rhel8                   8h
openshift-virtualization-os-images   rhel8-amd64             8h
openshift-virtualization-os-images   rhel8-arm64             8h
openshift-virtualization-os-images   rhel9                   8h
openshift-virtualization-os-images   rhel9-amd64             8h
openshift-virtualization-os-images   rhel9-arm64             119s
openshift-virtualization-os-images   win10                   8h
openshift-virtualization-os-images   win10-amd64             8h
openshift-virtualization-os-images   win11                   8h
openshift-virtualization-os-images   win11-amd64             8h
openshift-virtualization-os-images   win2k16                 8h
openshift-virtualization-os-images   win2k16-amd64           8h
openshift-virtualization-os-images   win2k19                 8h
openshift-virtualization-os-images   win2k19-amd64           8h
openshift-virtualization-os-images   win2k22                 8h
openshift-virtualization-os-images   win2k22-amd64           8h
openshift-virtualization-os-images   win2k25                 8h
openshift-virtualization-os-images   win2k25-amd64           8h
 
Result: datasources continue to stay

      Version-Release number of selected component (if applicable):

      4.20
Deployed: OCP-4.20.0-ec.6Deployed: CNV-v4.20.0.rhel9-152

      How reproducible:

       

      Steps to Reproduce:

      1.delete an arm node
2.
3.

      Actual results:

      datasources objects stayed

      Expected results:

      datasources objects should go

      Additional info:

       

        1. arm_node_removal_before_after
          53 kB

              alitke@redhat.com Adam Litke
              gkapoor@redhat.com Geetika Kapoor
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated: