OpenShift supports specifying multiple machineNetworks, but the assisted-service (and thus the agent-based installer) does not.

The fix for OCPBUGS-29975 will allow users to specifiy multiple machineNetworks for clusters where UserManagedNetworking is enabled (i.e. platform: none). However, for clusters using the internal on-prem networking, the belongs-to-majority-group host validation requires all hosts to be in the same L2 network. In baremetal IPI, this restriction does not exist - hosts can be in any of the machineNetworks listed. The agent installer should not be more restrictive than needed to prevent failures.

The control plane hosts need to be in the same L2 network as each other so that the API VIP can move between them. The worker hosts need to be in the same L2 network as each other so that the Ingress VIP can move between them. But there is no need for the control plane and workers to be in the same L2 network as long as all hosts can reach each other at Layer 3. [This was incorrect - according to bnemec@redhat.com there are places in baremetal-runtimecfg that still erroneously assume that both VIPs must be in the same subnet for things to work correctly. As long as this remains the case, the current validation is a good thing.]

Once OCPBUGS-29975 is fixed, users can potentially work around this by specifying platform: baremetal but then generating the cluster-manifests and enabling UserManagedNetworking.  [This was incorrect - user-managed networking is not allowed on platform:baremetal.] It's not clear to me what other side-effects that might have. And moves are underway to remove the UserManagedNetworking flag and tie the networking type exclusively to the platform (MGMT-14934).