Azure File CSI NFS support

Feature Overview (aka. Goal Summary)  

Azure File CSI supports both SMB and NFS protocol. Currently we only support SMB and there is a strong demand from IBM and individual customers to support NFS for posix compliance reasons.

Goals (aka. expected user outcomes)

Support Azure File CSI with NFS.

The Azure File operator will not automatically create a NFS storage class, we will document how to create one.

Requirements (aka. Acceptance Criteria):

There are some concerns on the way Azure File CSI deals with NFS. They don't respect the FSGroup policy supplied in the pod definition. This breaks kubernetes convention where a pod should be able to define its own FSGroup policy, instead Azure File CSI set a per driver policy that pods can't override.

We brought up this problem to MSFT but there is no fix planned on the driver, given the pressure from the field we are going to support NFS with a on root mismatch default and document this specific behavior in our documentation.

Use Cases (Optional):

As an OCP on Azure admin i want my user to be able to consume NFS based PVs through Azure File CSI.

As an OCP on Azure user i want to attach NFS based PVs to my pods.

As an ARO customer I want to consume NFS based PVs.

Questions to Answer (Optional):

Include a list of refinement / architectural questions that may need to be answered before coding can begin.  Initial completion during Refinement status.

Out of Scope

Running two drivers, one for NFS and one for SMB to solve the FSGroup issue.

Background

Provide any additional context is needed to frame the feature.  Initial completion during Refinement status.

Customer Considerations

This feature is candidate to be backported up to 4.12 if possible.

Documentation Considerations

Document that Azure File CSI NFS is supported, how to create a storage class as well as the FSGroup issue.