Uploaded image for project: 'OpenShift Virtualization'
  1. OpenShift Virtualization
  2. CNV-35429

[2250659] Validate Disk Failover Test fails during Windows Shared Cluster validation

XMLWordPrintable

    • Urgent

      Description of problem:
      Validate Disk Failover Test fails during Windows Shared Cluster validation when attempting to write file data to partition table entry claiming the disk structure is corrupted or unreadable.

      Version-Release number of selected component (if applicable):
      oc version
      Client Version: 4.14.0-rc.2
      Kustomize Version: v5.0.1
      Server Version: 4.14.0-rc.2
      Kubernetes Version: v1.27.6+1648878
      [cloud-user@ocp-psi-executor ~]$ oc get csv -n openshift-cnv
      NAME DISPLAY VERSION REPLACES PHASE
      kubevirt-hyperconverged-operator.v4.14.1 OpenShift Virtualization 4.14.1 kubevirt-hyperconverged-operator.v4.14.0 Succeeded
      openshift-pipelines-operator-rh.v1.11.0 Red Hat OpenShift Pipelines 1.11.0 Succeeded

      How reproducible:
      100%

      Steps to Reproduce:
      1. Map an iscsi LUN from the Netapp Storage Provider to 3 worker nodes
      2. Create a PV referencing the iscsi LUN and detailing the 3 worker nodes
      3. Create a shared PVC based on the iscsi LUN with volumeMode block and accessMode ReadWriteMany
      4. Create 3 Windows12 R2 vms each with their own OS disk and referencing the 2nd shared iscsi LUN based PVC
      5. Install the virtio-win-guest-tools to update the drivers on each of the VMs
      6. Via Disk Manager access the iscsi LUN, create a partition on all available space and install NTFS
      7. Install Windows Shared Cluster software on each of the 3 Windows VMS
      8. Install and configure Active Directory/DNS Domain Controller on one of the VMs
      9. Run the Failover Cluster - Validate Configuration tool and select to run only the Storage - Validate Disk Failover Test
      10. The Validate Disk Failover Test fails when attempting to write file data to the partition table entry claiming the disk structure is corrupted or unreadable.

      Actual results:
      The Validate Disk Failover Test fails when attempting to write file data to the partition table entry claiming the disk structure is corrupted or unreadable.

      Expected results:
      The Validate Disk Failover Test should succeed when attempting to write file data to the partition table entry.
      Additional info:

      Storage class yaml:
      ------------------------------------------
      apiVersion: storage.k8s.io/v1
      kind: StorageClass
      metadata:
      name: local-scsi
      provisioner: kubernetes.io/no-provisioner
      volumeBindingMode: Immediate

      PV Yaml:
      ------------------------------------------
      apiVersion: v1
      kind: PersistentVolume
      metadata:
      name: iscsi-pv-root
      spec:
      capacity:
      storage: 70Gi
      accessModes:

      • ReadWriteMany
        storageClassName: local-scsi
        iscsi:
        targetPortal: 10.9.96.31:3260
        iqn: iqn.1992-08.com.netapp:sn.438c2b596a3811e894b800a098da27d5:vs.4
        lun: 0
        volumeMode: Block
        nodeAffinity:
        required:
        nodeSelectorTerms:
      • matchExpressions:
      • key: kubernetes.io/hostname
        operator: In
        values:
      • stg03-kevin-zrzbv-worker-0-2c2q7
      • stg03-kevin-zrzbv-worker-0-9wssf
      • stg03-kevin-zrzbv-worker-0-zwwg9

      Shared PVC Yaml:
      ---------------------------------------------------------
      apiVersion: v1
      kind: PersistentVolumeClaim
      metadata:
      name: scsi-pvc
      spec:
      volumeMode: Block
      storageClassName: local-scsi
      accessModes:

      • ReadWriteMany
        resources:
        requests:
        storage: 70G

      VM1 Yaml:
      ----------------------------------------------------------

      oc get vm vm-win12-datavolume -oyaml
      apiVersion: kubevirt.io/v1
      kind: VirtualMachine
      metadata:
      annotations:
      kubemacpool.io/transaction-timestamp: "2023-11-19T11:43:28.558198691Z"
      kubevirt.io/latest-observed-api-version: v1
      kubevirt.io/storage-observed-api-version: v1
      creationTimestamp: "2023-10-30T11:50:59Z"
      finalizers:

      • kubevirt.io/virtualMachineControllerFinalize
        generation: 56
        labels:
        kubevirt.io/vm: vm-win12-datavolume
        name: vm-win12-datavolume
        namespace: default
        resourceVersion: "91170595"
        uid: 0da144e8-42bc-4e94-bc20-fa453cba028e
        spec:
        dataVolumeTemplates:
      • metadata:
        creationTimestamp: null
        name: win12-dv
        spec:
        pvc:
        accessModes:
      • ReadWriteOnce
        resources:
        requests:
        storage: 60Gi
        storageClassName: ocs-storagecluster-ceph-rbd
        volumeMode: Block
        source:
        http:
        url: http://10.19.3.125/pub/users/joherr/os/bootsource_images/win2012r2.qcow2.gz
        running: true
        template:
        metadata:
        creationTimestamp: null
        labels:
        kubevirt.io/vm: vm-win12-datavolume
        spec:
        architecture: amd64
        domain:
        devices:
        disks:
      • disk:
        bus: sata
        name: datavolumedisk1
      • errorPolicy: report
        lun:
        bus: scsi
        reservation: true
        name: scsi-disk
      • cdrom:
        bus: sata
        name: windows-drivers-disk
        interfaces:
      • bridge: {}
        macAddress: 02:7e:7e:00:00:05
        model: e1000e
        name: nic-conservation-lungfish
        machine:
        type: pc-q35-rhel9.2.0
        resources:
        requests:
        memory: 10Gi
        networks:
      • multus:
        networkName: l2-cluster-net
        name: nic-conservation-lungfish
        terminationGracePeriodSeconds: 0
        volumes:
      • dataVolume:
        name: win12-dv
        name: datavolumedisk1
      • name: scsi-disk
        persistentVolumeClaim:
        claimName: scsi-pvc
      • containerDisk:
        image: registry.redhat.io/container-native-virtualization/virtio-win-rhel9@sha256:3a83562ffa9e9c3438eecd7b0833e303778691d8e17ddb41f09fce598ed9a01b
        name: windows-drivers-disk
        status:
        conditions:
      • lastProbeTime: null
        lastTransitionTime: "2023-11-20T12:07:24Z"
        status: "True"
        type: Ready
      • lastProbeTime: null
        lastTransitionTime: null
        message: 'cannot migrate VMI: PVC win12-dv is not shared, live migration requires
        that all PVCs must be shared (using ReadWriteMany access mode)'
        reason: DisksNotLiveMigratable
        status: "False"
        type: LiveMigratable
      • lastProbeTime: "2023-11-20T12:08:33Z"
        lastTransitionTime: null
        status: "True"
        type: AgentConnected
        created: true
        desiredGeneration: 56
        observedGeneration: 56
        printableStatus: Running
        ready: true
        volumeSnapshotStatuses:
      • enabled: true
        name: datavolumedisk1
      • enabled: false
        name: scsi-disk
        reason: 'No VolumeSnapshotClass: Volume snapshots are not configured for this
        StorageClass [local-scsi] [scsi-disk]'
      • enabled: false
        name: windows-drivers-disk
        reason: Snapshot is not supported for this volumeSource type [windows-drivers-disk]

      VM2 Yaml:
      ------------------------------------------------------------

      oc get vm vm-win12-datavolume-b -oyaml
      apiVersion: kubevirt.io/v1
      kind: VirtualMachine
      metadata:
      annotations:
      kubemacpool.io/transaction-timestamp: "2023-11-19T11:43:49.226565165Z"
      kubevirt.io/latest-observed-api-version: v1
      kubevirt.io/storage-observed-api-version: v1
      creationTimestamp: "2023-10-30T16:14:32Z"
      finalizers:

      • kubevirt.io/virtualMachineControllerFinalize
        generation: 70
        labels:
        kubevirt.io/vm: vm-win12-datavolume-b
        name: vm-win12-datavolume-b
        namespace: default
        resourceVersion: "91141914"
        uid: 81f583e1-5285-47e3-bcf1-1acf50ad033d
        spec:
        dataVolumeTemplates:
      • metadata:
        creationTimestamp: null
        name: win12-dv-b
        spec:
        pvc:
        accessModes:
      • ReadWriteOnce
        resources:
        requests:
        storage: 60Gi
        storageClassName: ocs-storagecluster-ceph-rbd
        volumeMode: Block
        source:
        http:
        url: http://10.19.3.125/pub/users/joherr/os/bootsource_images/win2012r2.qcow2.gz
        running: true
        template:
        metadata:
        creationTimestamp: null
        labels:
        kubevirt.io/vm: vm-win12-datavolume-b
        spec:
        architecture: amd64
        domain:
        devices:
        disks:
      • disk:
        bus: sata
        name: datavolumedisk1-b
      • errorPolicy: report
        lun:
        bus: scsi
        reservation: true
        name: scsi-disk
      • cdrom:
        bus: sata
        name: windows-drivers-disk
        interfaces:
      • bridge: {}
        macAddress: 02:7e:7e:00:00:03
        model: e1000e
        name: nic-contemporary-chinchilla
        machine:
        type: pc-q35-rhel9.2.0
        resources:
        requests:
        memory: 8Gi
        networks:
      • multus:
        networkName: l2-cluster-net
        name: nic-contemporary-chinchilla
        terminationGracePeriodSeconds: 0
        volumes:
      • dataVolume:
        name: win12-dv-b
        name: datavolumedisk1-b
      • name: scsi-disk
        persistentVolumeClaim:
        claimName: scsi-pvc
      • containerDisk:
        image: registry.redhat.io/container-native-virtualization/virtio-win-rhel9@sha256:3a83562ffa9e9c3438eecd7b0833e303778691d8e17ddb41f09fce598ed9a01b
        name: windows-drivers-disk
        status:
        conditions:
      • lastProbeTime: null
        lastTransitionTime: "2023-11-20T11:43:03Z"
        status: "True"
        type: Ready
      • lastProbeTime: null
        lastTransitionTime: null
        message: 'cannot migrate VMI: PVC win12-dv-b is not shared, live migration requires
        that all PVCs must be shared (using ReadWriteMany access mode)'
        reason: DisksNotLiveMigratable
        status: "False"
        type: LiveMigratable
      • lastProbeTime: "2023-11-20T11:44:00Z"
        lastTransitionTime: null
        status: "True"
        type: AgentConnected
        created: true
        desiredGeneration: 70
        observedGeneration: 70
        printableStatus: Running
        ready: true
        volumeSnapshotStatuses:
      • enabled: true
        name: datavolumedisk1-b
      • enabled: false
        name: scsi-disk
        reason: 'No VolumeSnapshotClass: Volume snapshots are not configured for this
        StorageClass [local-scsi] [scsi-disk]'
      • enabled: false
        name: windows-drivers-disk
        reason: Snapshot is not supported for this volumeSource type [windows-drivers-disk]

      VM3 Yaml:
      ---------------------------------------------------------

      oc get vm vm-win12-datavolume-c -oyaml
      apiVersion: kubevirt.io/v1
      kind: VirtualMachine
      metadata:
      annotations:
      kubemacpool.io/transaction-timestamp: "2023-11-19T11:43:58.073946416Z"
      kubevirt.io/latest-observed-api-version: v1
      kubevirt.io/storage-observed-api-version: v1
      creationTimestamp: "2023-10-30T18:14:40Z"
      finalizers:

      • kubevirt.io/virtualMachineControllerFinalize
        generation: 50
        labels:
        kubevirt.io/vm: vm-win12-datavolume-c
        name: vm-win12-datavolume-c
        namespace: default
        resourceVersion: "91139114"
        uid: b1e0d451-9e13-47e9-9bf2-0d7c904229f8
        spec:
        dataVolumeTemplates:
      • metadata:
        creationTimestamp: null
        name: win12-dv-c
        spec:
        pvc:
        accessModes:
      • ReadWriteOnce
        resources:
        requests:
        storage: 60Gi
        storageClassName: ocs-storagecluster-ceph-rbd
        volumeMode: Block
        source:
        http:
        url: http://10.19.3.125/pub/users/joherr/os/bootsource_images/win2012r2.qcow2.gz
        running: true
        template:
        metadata:
        creationTimestamp: null
        labels:
        kubevirt.io/vm: vm-win12-datavolume-c
        spec:
        architecture: amd64
        domain:
        devices:
        disks:
      • disk:
        bus: sata
        name: datavolumedisk1-c
      • errorPolicy: report
        lun:
        bus: scsi
        reservation: true
        name: scsi-disk
      • cdrom:
        bus: sata
        name: windows-drivers-disk
        interfaces:
      • bridge: {}
        macAddress: 02:7e:7e:00:00:06
        model: e1000e
        name: nic-willing-jackal
        machine:
        type: pc-q35-rhel9.2.0
        resources:
        requests:
        memory: 7Gi
        networks:
      • multus:
        networkName: l2-cluster-net
        name: nic-willing-jackal
        terminationGracePeriodSeconds: 0
        volumes:
      • dataVolume:
        name: win12-dv-c
        name: datavolumedisk1-c
      • name: scsi-disk
        persistentVolumeClaim:
        claimName: scsi-pvc
      • containerDisk:
        image: registry.redhat.io/container-native-virtualization/virtio-win-rhel9@sha256:3a83562ffa9e9c3438eecd7b0833e303778691d8e17ddb41f09fce598ed9a01b
        name: windows-drivers-disk
        status:
        conditions:
      • lastProbeTime: null
        lastTransitionTime: "2023-11-20T11:40:52Z"
        status: "True"
        type: Ready
      • lastProbeTime: null
        lastTransitionTime: null
        message: 'cannot migrate VMI: PVC win12-dv-c is not shared, live migration requires
        that all PVCs must be shared (using ReadWriteMany access mode)'
        reason: DisksNotLiveMigratable
        status: "False"
        type: LiveMigratable
      • lastProbeTime: "2023-11-20T11:41:38Z"
        lastTransitionTime: null
        status: "True"
        type: AgentConnected
        created: true
        desiredGeneration: 50
        observedGeneration: 50
        printableStatus: Running
        ready: true
        volumeSnapshotStatuses:
      • enabled: true
        name: datavolumedisk1-c
      • enabled: false
        name: scsi-disk
        reason: 'No VolumeSnapshotClass: Volume snapshots are not configured for this
        StorageClass [local-scsi] [scsi-disk]'
      • enabled: false
        name: windows-drivers-disk
        reason: Snapshot is not supported for this volumeSource type [windows-drivers-disk]

            alitke@redhat.com Adam Litke
            kgoldbla Kevin Alon Goldblatt
            Dalia Frank Dalia Frank
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: