R&D - Driver Recycling with Read Only Volumes

Spike

A desired feature of the shared resource CSI driver is the ability to update the referenced Secret or ConfigMap, while the pod sees the volume as read-only.

When configuring read-only volume mounts, we lose the ability to update the volume contents across driver restarts. This is critical for us to support upgrades and driver restarts.

By comparison,

• when the volume is read-write, and we have one less linux file system layer, we can update contents after restart
• when the volume is read-only, if the pod loses permissions to the sharedConfigMap or sharedSecret, the driver can remove the content, and that is reflected in what the pod sees

Gabe tried several "driver only" approaches, even persisting the file system mounts in fstab on the host to try and preserve things.  No luck.

From what he was able to gather, something (the kubelet?) is eliminating the extra file system layer with read only volume (that extra / intermediate layer is what the driver has access to

and uses for updates, but the "top" file system layer is what the pod sees, and that is read only).

We need to solicit help from the node and storage teams as appropriate to see what if any resolution paths are possible, so we do not lose that intermediate file system layer.  Or if there is

another linux file system approach we should be taking to achive our goals.

 The current code for all this is at https://github.com/openshift/csi-driver-shared-resource/blob/master/pkg/hostpath/mount.go

Acceptance Criteria

Determine how we can support read only volumes as per the CSI spec, while ensuring the content continues to be mounted on driver restart.

The initial approach here doesn't appear to work, we need to write this up in a Google Doc.

Notes

We will likely need help from the OpenShift storage or Node teams on this, since the kubelet seems to impact our current iteration.