Complex RHCOS disk layout

Background: 

The business assumptions that most organizations deploy OCP to Cloud providers allowed for RHCOS to use a simple disk setup. bbreard reports that we are seeing significant bare-metal deployments where the assumption of a single root device is fallacious.

Estimate (XS, S, M, L, XL, XXL): XL

Assumptions: 

Bare-metal deployments can contain up to N devices

• Users who deploy to bare-metal may do so in remote, detached environments where re-deployment of a failed node is difficult requiring more durable disk configurations.
• Users have expressed a desire to deploy /var/lib/containers, /var, or the rpm-ostree to different devices.
• RHCOS with 4.3 is only capable of encrypting the root device
• Users have legitimate needs to change the rootfs type

Acceptance Criteria:

The feature is considered done when the following have been met:

4.6 Phase

• Users can deploy /var/, /var/lib/containers/, or the rpm-ostree rootfs to different devices

4.7 Phase

• Encryption is possible on all disk devices, not just root
• RAID-1 is supported to add durability to nodes, including ability to boot from degraded rootfs mirror
• Users can deploy arbitrary non-system paths (e.g. /my-user-directory) to different devices
• Documented support for RAID 1, 5 for user-defined, non-system mount points.