Feature Overview

This feature aims to enable customers of OCP to integrate 3rd party KMS solutions for encrypting etcd values at rest in accordance with: https://kubernetes.io/docs/tasks/administer-cluster/kms-provider/

This feature makes the feature GA after the Tech Preview released with OCPSTRAT-108 

Goals

• Bring KMS v2 API to beta|stable level
• Create/expose mechanisms for customers to plug in containers/operators which can serve the API server's needs (can it be an operator, something provided via CoreOS layering, vanilla container spec provided to API server operator?)
• Provide similar UX experience for all of self-hosted, hypershift, SNO scenarios
• Provide example container/operator for the mechanism

General Prioritization for the Feature

1. Approved design for detection & actuation for stand-alone OCP clusters.
   1. How to detect a problem like an expired/lost key and no contact with the KMS provider?
   2. How to inform/notify the situation, even at node level, of the situation
2. Tech Preview (Feature gated) enabling Kube-KMS v2 for partners to start working on KMS plugin provider integrations:
   1. Cloud: (priority Azure > AWS > Google)
      1. Azure KMS
      2. Azure Dedicated HSM
      3. AWS KMS
      4. AWS CloudHSM
      5. Google Cloud HSM
   2. On-premise:
      1. HashiCorp Vault
      2. EU FSI & EU Telco KMS/HSM top-2 providers
3. GA after at least one stable KMS plugin provider

Background, and strategic fit

We've had numerous customer requests for allowing external KMS integration to encrypt etcd. The existing etcd encryption mechanism is deemed insufficient for a couple reasons:

1) The encryption algorithm being used is static (and everybody has preferences)

2) The decryption keys are easily discoverable on self-hosted clusters

Upon investigation in API-1021, many current implementation (API, mechanics) were identified as needing improvement which led us to drive this PR we hope overcomes our concerns:

https://github.com/kubernetes/enhancements/pull/3302