1. Proposed title of this feature request
Ability to change Path MTU in cluster while minimizing workload disruption

2. What is the nature and description of the request?
Currently the change of Path MTU is an operation that requires two reboots and is executed on all nodes of the cluster at the same time.

3. Why does the customer need this? (List the business requirements here)

Customer requires an procedure to change Path MTU that attempts to minimizes workload disruption (e.g., potentially using MCP pools). Let's say you have two pools for nodes hosting workloads, MCP A and MCP B. A procedure like this should be doable and documented in order to be supported:

• Pause MCP B.
• Prepare the cluster for MTU migration (setting the mtu migration configuration option)
• MCP A will be updated
• Unpause MCP B, it will be updated
• We have all nodes of the having completed step 1 of the MTU procedure
• Pause MCP B again (if needed)
• Reconfigure the cluster for the new MTU
• MCP A will be updated
• Unpause MCP B, it will be updated
• We have all nodes of the having completed step 2 of the MTU procedure
• Pause pool B again
• Take the cluster out of MTU migration (unsetting the mtu migration configuration option)
• MCP A will be updated
• Unpause MCP B, it will be updated
• We have all nodes of the having completed step 3 of the MTU procedure, migration done

Stretch Goal: A procedure like this would intend to do the MTU setting in only 1 reboot. Would this be doable?

• Pause MCP A and MCP B
• Prepare the cluster for MTU migration (setting the MTU migration configuration option)
• Reconfigure the cluster for the new MTU using machineConfig
• Unpause MCP A so machine config render of step 2 and 3 are combine and applied on MCP A (so only 1 reboot instead of 2 to reduce workload impact)
• Unpause MCP B so machine config render of step 2 and 3 combined and applied on MCP B (so only 1 reboot instead of 2 to reduce Workload impact)
• Pause all MCP A, and MCP B
• Take the cluster out of MTU migration (unsetting the MTU migration configuration option)
• Applied machine config to disabled the service configuring MTU so at the boot time it will not run again
• Unpause MCP A, it will be updated
• Unpause MCP B, it will be updated
• We have all nodes having completed step 3 of the MTU procedure, migration done

In this way only one the nodes of one MCP are rebooted concurrently.

4. List any affected packages or components.

OVN-K, MachineConfig Operator