As a customer I want to my OSD4 or ROSA clusters to use gp3 volumes that are the best value for money.

Additional Context

References:

• https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-volume-types.html
• SD-ADR-0046 : Move ROSA/OSD to gp3
• New clusters use gp3 storage by default: SDE-1829

Acceptance Criteria

• Investigate live migration of SRE PVs from gp2 to gp3 jaharrin was testing this. Please share outcomes.
• Investigate live migration of master and infra node storage from gp2 to gp3 jaharrin was testing this and implemented POC tooling. Please share outcomes.
• Confirm which versions are able to support migration (likely 4.10+)
• Create a new gp3 storage class and make it the default on all clusters (with above version restrictions)
• gp2 storage class remains on clusters, but is not the default anymore (with above version restrictions)
• SRE PVs (Prometheus) are migrated from gp2 to gp3 (with above version restrictions)
• Master and infra node storage is migrated from gp2 to gp3 (with above version restrictions)
• Document how customers can migrate their volumes and compute nodes

• Scope of the migration still needs to be discussed, e.g. migrate non-ccs only, RH-internal only, ... (cc nraghava-sre)