CSI Inline Volume Support with admission plugin

Goal: 

The goal is to provide inline volume support (also known as Ephemeral volumes) via a CSI driver/operator. This epic also track the dev of the new admission plugin required to make inline volumes safe.

Problem: 

• The only practical way to extend pods such that node local integrations can happen is with inline volumes. So if we want to integrate with IAM for per pod credentials, we need inline csi volumes. If we want to do better build cache integration, we need inline csi. 

Why is this important: 

• (from https://kubernetes-csi.github.io/docs/ephemeral-local-volumes.html) Traditionally, volumes that are backed by CSI drivers can only be used with a PersistentVolume and PersistentVolumeClaim object combination. This feature will support ephemeral storage use cases and allows CSI volumes to be specified directly in the pod specification. At runtime, nested inline volumes follow the ephemeral lifecycle of their associated pods where the driver handles all phases of volume operations as pods are created and destroyed.
• Vault integration can be implemented via in-line volumes (see https://github.com/deislabs/secrets-store-csi-driver/blob/master/README.md).
• Inline volumes would allow us to give out tokens for cloud integration and nuke cloud credential operator’s use of secrets.
• In OpenShift we already have Shared Resource CSI driver, which uses in-line CSI volumes to distribute cluster-wide secrets and/or config maps.

Dependencies (internal and external):

• CSI API

Prioritized epics + deliverables (in scope / not in scope):

• In Scope
  • A working CSI based inline volume
  • Documentation
  • Admision plugin

• Not in Scope
  • Implementing the use cases for inline volumes (i.e. integration with IAM)

Estimate (XS, S, M, L, XL, XXL):

Previous Work:

Customers:

Open questions:

Notes:

• Couple of useful links:
  • https://kubernetes-csi.github.io/docs/ephemeral-local-volumes.html
  • https://github.com/deislabs/secrets-store-csi-driver/blob/master/README.md
  • https://github.com/kubernetes/enhancements/issues/596