Feature Overview (aka. Goal Summary)

Create a new config option that causes device discovery only once at install time, to keep it static for the remaining lifecycle of the LVMCluster.

Goals (aka. expected user outcomes)

When no deviceSelector/Paths are given, LVMS has a greedy lookup strategy to discovery devices. That can cause problems when devices are added/removed frequently. This happens esp. in test-environments with SAN storage, where disks / LUNs appear / disappear. Esp. the disappearing is a problem, as device removal is not supported by LVMs and leads to a failed/disfunctional LVMCluster.

The goal of this feature is to introduce a new config option to help with this situation to allow either for a dynamic device discovery (reflectting the current behaviour) or a static discovery where device discovery is happening only at LVMCLuster creation.

For all existing LVMClusters, the behaviour keeps unchnaged (dynamic). For all new created LVM Cluster, the default will be static, to avoid unpleasant surprises of the dynamic behaviour during lifetime.  This will reduce the number of support cases for those situations. 

Requirements (aka. Acceptance Criteria):

1. Introduce a new config option in LVMCLuster API at the device Class level to select betwwen dynamic or statuc device discovery
   1. Static: VG will be created with discoverd devices on install time. All later added/discovered devices will be ignored.
   2. devices will be added to the VG if they are present at runtime
2. Change of the default behaviour:
   1. All existing LVMClusters will work in Dynamic Mode
   2. All new LVMCluster default to Static Mode (esp. when created via assisted installer / OLM)
3. Switch of the behaviour

1. 1. It should be possible to switch from dynamic mode to static mode to "lock" a config at a certain point in time.
   2. It should be possible to switch from static mode to dynamic mode to allow adding of new devices to a cluster.
2. Clear status messages: in the status field of the LVM Cluster object, clear and easy to interpret status messages explain the reasons why a a device was included or excluded. Example:
   /dev/loop1 was not part of vg1 at creation (static device discovery enabled)

Questions to Answer (Optional):

none

Out of Scope

tbd

Background

See this bug as an example on the confusion the current behaviour can create:

https://issues.redhat.com/browse/OCPBUGS-30151

Customer Considerations

We need to make sure we dont break existing installation.

Documentation Considerations

Behaviour needs to be document. We should take the opportunity and add a way more detailed explanation on how LVMS can discover and disk devices. Things that section should cover:

• Auto-Discovery vs. Manual config  using deviePath (hint: we recommend static). 
• Manual DevicePath config: explain the different pro/cons of /dev/disk/by-..... and give a recommendation what to use (by-label). See https://github.com/openshift/lvm-operator/pull/599#issuecomment-2031913384 for a good disussion.
• Device Discovery Mode: static and dynamic
• Caution when using multiple disk/devices: Use HW RAID (recommend) or software-raid config (as already documented)

Interoperability Considerations

Assisted Installer should default to the new static discovery mode.