The stub ignition(aka pointer config) is also saved off in the machine-api namespace by the installer and thereafter unmanaged. They are then referenced in the earlier mentioned machineset. The ignition binary uses stub ignition on first boot to request content from the machine-config-server, it contains the endpoint on the MCS and necessary auth information. (The MCS serves the actual ignition config for the node. The ignition binary now does first boot provisioning based on this, then hands off to the MCD first boot service to take care off the non ignition section of the machine config - which is also a file served via ignition earlier). 

This has caused issues in the past where pre-4.6 born clusters wouldn’t allow new nodes to join due to an out of date ignition stub. The post-4.6 MCS only understood spec 3 ignition, and as the managed ignition stub is only spec 2, it wouldn’t work. 

As of this PR, the MCO does store a copy of the stub ignition. It appends “-managed” to the name of this stub. However, since the machineset still references the old secret({pool}-user-data), it doesn’t get used, despite being managed. The MCO pulls these from the mcs secrets in the MCO namespace. However, we don’t reference them in the machinesets, so they are unused.

Another thing to consider is that the user may have modified their machineset to point to an entirely new user-data stub. Implementing this would require us to translate the ignition stub under use to the latest spec(3.4 as of now, I think). This would be cycling through all the machinesets currently present to find the relevant *-user-data secrets, and then translate them up so the upgraded MCS will always understand them. This can implemented alongside MCO-679, in the proposed subcontroller.

Update(10/13/23):

As I wrote up the card for rollback functionality, it occurred to me that we should be creating a new secret in this process and preserve the old one for rollback purposes

Update(1/19/24):

Based on feedback from the enhancement, as a very low percentage of customers tend to customize their secrets we are going to re-use the "-managed" worker pool secret. This should be mentioned in documentation so it is clear to users and we hope to provide a future API mechanism to support user customizations when this feature is enabled.

Done when:

• MachineSets enrolled for boot image updates will now point towards the "worker-user-data-managed" secret.
• This is managed behind a feature gate.