Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-1067

[vsphere-CSI-Driver-Operator] The storageclass "thin-csi" could not be re-created after deleting

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Undefined Undefined
    • 4.12.z
    • 4.11
    • Storage / Operators
    • None
    • Moderate
    • None
    • False
    • Hide

      None

      Show
      None

      Description of problem:

      The storageclass "thin-csi" is created by vsphere-CSI-Driver-Operator, after deleting it manually, it should be re-created immediately. 

      Version-Release number of selected component (if applicable):

      4.11.4

      How reproducible:

      Always

      Steps to Reproduce:

      1. Check storageclass in running cluster, thin-csi is present:
      $ oc get sc 
      NAME             PROVISIONER                    RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
      thin (default)   kubernetes.io/vsphere-volume   Delete          Immediate              false                  41m
      thin-csi         csi.vsphere.vmware.com         Delete          WaitForFirstConsumer   true                   38m
      
      2. Delete thin-csi storageclass:
      $ oc delete sc thin-csi
      storageclass.storage.k8s.io "thin-csi" deleted
      
      3. Check storageclass again, thin-csi is not present:
      $ oc get sc
      NAME             PROVISIONER                    RECLAIMPOLICY   VOLUMEBINDINGMODE   ALLOWVOLUMEEXPANSION   AGE
      thin (default)   kubernetes.io/vsphere-volume   Delete          Immediate           false                  50m
      
      4. Check vmware-vsphere-csi-driver-operator log:
      ......
      I0909 03:47:42.172866       1 named_certificates.go:53] "Loaded SNI cert" index=0 certName="self-signed loopback" certDetail="\"apiserver-loopback-client@1662695014\" [serving] validServingFor=[apiserver-loopback-client] issuer=\"apiserver-loopback-client-ca@1662695014\" (2022-09-09 02:43:34 +0000 UTC to 2023-09-09 02:43:34 +0000 UTC (now=2022-09-09 03:47:42.172853123 +0000 UTC))"I0909 03:49:38.294962       
      1 streamwatcher.go:111] Unexpected EOF during watch stream event decoding: unexpected EOFI0909 03:49:38.295468       
      1 streamwatcher.go:111] Unexpected EOF during watch stream event decoding: unexpected EOFI0909 03:49:38.295765       
      1 streamwatcher.go:111] Unexpected EOF during watch stream event decoding: unexpected EOF
      
      
      5. Only first time creating in vmware-vsphere-csi-driver-operator log:
      $ oc -n openshift-cluster-csi-drivers logs vmware-vsphere-csi-driver-operator-7cc6d44b5c-c8czw | grep -i "storageclass"I0909 03:46:31.865926   1 event.go:285] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"openshift-cluster-csi-drivers", Name:"vmware-vsphere-csi-driver-operator", UID:"9e0c3e2d-d403-40a1-bf69-191d7aec202b", APIVersion:"apps/v1", ResourceVersion:"", FieldPath:""}): type: 'Normal' reason: 'StorageClassCreated' Created StorageClass.storage.k8s.io/thin-csi because it was missing 

      Actual results:

      The storageclass "thin-csi" could not be re-created after deleting

      Expected results:

      The storageclass "thin-csi" should be re-created after deleting

      Additional info:

       

            [OCPBUGS-1067] [vsphere-CSI-Driver-Operator] The storageclass "thin-csi" could not be re-created after deleting

            Errata Tool added a comment -

            Since the problem described in this issue should be resolved in a recent advisory, it has been closed.

            For information on the advisory, and where to find the updated files, follow the link below.

            If the solution does not work for you, open a new bug report.
            https://access.redhat.com/errata/RHSA-2022:7399

            Errata Tool added a comment - Since the problem described in this issue should be resolved in a recent advisory, it has been closed. For information on the advisory, and where to find the updated files, follow the link below. If the solution does not work for you, open a new bug report. https://access.redhat.com/errata/RHSA-2022:7399

            Wei Duan added a comment -

            hekumar@redhat.com jdobson@redhat.com 
            Should we backport to 4.11/4.10?

            Wei Duan added a comment - hekumar@redhat.com jdobson@redhat.com   Should we backport to 4.11/4.10?

            BTW I verified that, if you wait for next sync interval, storageclass does get created. Opened https://github.com/openshift/vmware-vsphere-csi-driver-operator/pull/108 to fix this issue. It is bit ugly and I do not like it very much, But I am not sure if there is another option.

            Hemant Kumar added a comment - BTW I verified that, if you wait for next sync interval, storageclass does get created. Opened https://github.com/openshift/vmware-vsphere-csi-driver-operator/pull/108 to fix this issue. It is bit ugly and I do not like it very much, But I am not sure if there is another option.

            I think this happened after - https://github.com/openshift/vmware-vsphere-csi-driver-operator/pull/95

            We moved storageclass creation after the cluster checks are run, so as not to create SCs if cluster is not supported. Now since cluster checks are ran every 8 hours or so (if last cluster check succeeded), then we don't recreate SC because between those 8 hours we are not running any cluster checks.

            So a unfortunate side effect of conflicting concerns. If we chose to always sync storageclass even if cluster checks are not run, I suspect we will be back in same state as https://github.com/openshift/vmware-vsphere-csi-driver-operator/pull/95

            Hemant Kumar added a comment - I think this happened after - https://github.com/openshift/vmware-vsphere-csi-driver-operator/pull/95 We moved storageclass creation after the cluster checks are run, so as not to create SCs if cluster is not supported. Now since cluster checks are ran every 8 hours or so (if last cluster check succeeded), then we don't recreate SC because between those 8 hours we are not running any cluster checks. So a unfortunate side effect of conflicting concerns. If we chose to always sync storageclass even if cluster checks are not run, I suspect we will be back in same state as https://github.com/openshift/vmware-vsphere-csi-driver-operator/pull/95

            Wei Duan added a comment -

            rhn-engineering-jsafrane Hi, it may not happen to user when they don't remove the sc, but considering this is a regression issue, please help raise priority, thanks.

            Wei Duan added a comment - rhn-engineering-jsafrane Hi, it may not happen to user when they don't remove the sc, but considering this is a regression issue, please help raise priority, thanks.

              hekumar@redhat.com Hemant Kumar
              wduan@redhat.com Wei Duan
              Wei Duan Wei Duan
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Created:
                Updated:
                Resolved: