Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-1488

4.11: [IBMCloud] The udevadm utility is missing in the IBM Cloud VPC block storage IPI image

XMLWordPrintable

    • Moderate
    • None
    • False
    • Hide

      None

      Show
      None
    • N/A
    • Release Note Not Required

      This is a clone of issue OCPBUGS-1460. The following is the description of the original issue:

      This is a clone of issue OCPBUGS-416. The following is the description of the original issue:

      Description of problem:
      The pod fails to mount the PVC using IBM Cloud VPC block storage.

      Version-Release number of selected component (if applicable):

      How reproducible:
      The steps can be followed here: from this link
      https://cloud.ibm.com/docs/openshift?topic=openshift-vpc-block
      .
      The error occurs when the application pod tried to mount the VPC.

      Steps to Reproduce:
      Describe above.

      Actual results:
      Events:
      Type Reason Age From Message
      ---- ------ ---- ---- -------
      Normal Scheduled 26m default-scheduler Successfully assigned default/test to a100-huge-m25p7-worker-3-with-secondary-xdwvl
      Normal SuccessfulAttachVolume 26m attachdetach-controller AttachVolume.Attach succeeded for volume "pvc-8721c341-739d-4607-bbcb-9dcf66ef6dba"
      Warning FailedMount 26m (x2 over 26m) kubelet MountVolume.MountDevice failed for volume "pvc-8721c341-739d-4607-bbcb-9dcf66ef6dba" : rpc error: code = Internal desc = {RequestID: ffbb97b4-e4d0-4016-87a9-dc46f80c5478 , Code: FormatAndMountFailed, Description: Failed to format '/dev/disk/by-id/virtio-0777-6872e22d-5c00-4' and mount it at '/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-8721c341-739d-4607-bbcb-9dcf66ef6dba/globalmount', BackendError: format of disk "/dev/disk/by-id/virtio-0777-6872e22d-5c00-4" failed: type"ext4") target"/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-8721c341-739d-4607-bbcb-9dcf66ef6dba/globalmount") options"defaults") errcode:(exit status 1) output:(mke2fs 1.45.6 (20-Mar-2020)
      The file /dev/disk/by-id/virtio-0777-6872e22d-5c00-4 does not exist and no size was specified.
      ) , Action: Please check if there is any error in POD describe related with volume attach}
      Warning FailedMount 22m kubelet Unable to attach or mount volumes: unmounted volumes=[bs-pvc], unattached volumes=[kube-api-access-6bgvj bs-pvc]: timed out waiting for the condition
      Warning FailedMount 4m11s (x9 over 24m) kubelet Unable to attach or mount volumes: unmounted volumes=[bs-pvc], unattached volumes=[bs-pvc kube-api-access-6bgvj]: timed out waiting for the condition
      Warning FailedMount 3m51s (x17 over 26m) kubelet MountVolume.MountDevice failed for volume "pvc-8721c341-739d-4607-bbcb-9dcf66ef6dba" : rpc error: code = Internal desc = {RequestID: 1a12a7c5-3bd0-41cf-b8a9-90dd3224c2fb , Code: FormatAndMountFailed, Description: Failed to format '/dev/disk/by-id/virtio-0777-6872e22d-5c00-4' and mount it at '/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-8721c341-739d-4607-bbcb-9dcf66ef6dba/globalmount', BackendError: format of disk "/dev/disk/by-id/virtio-0777-6872e22d-5c00-4" failed: type"ext4") target"/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-8721c341-739d-4607-bbcb-9dcf66ef6dba/globalmount") options"defaults") errcode:(exit status 1) output:(mke2fs 1.45.6 (20-Mar-2020)

      Expected results:
      The pod should successfully mount the PVC

      Additional info:
      Had a debugging session with Sameer Shaikh and Arashad Ahamad from the IBM VPC block storage team. The conclusion is that the udevadm utility is missing in the IPI image used by the IBM Cloud VPC block storage CSI.

      1. oc exec -it ibm-vpc-block-csi-controller-5fbb46bdc6-k7kpf -n openshift-cluster-csi-drivers -c iks-vpc-block-driver bash
        kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD][COMMAND] instead.
        bash-4.4$ which udevadm
        which: no udevadm in (/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin)

            rhn-engineering-jsafrane Jan Safranek
            openshift-crt-jira-prow OpenShift Prow Bot
            Chao Yang Chao Yang
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: