User or Developer story

As a user I want the CRO to scale pods during UPDATE operations in managed namespaces, so that proper limits are still enforced when the ClusterResourceOverride changes or when pods exist before management is enabled for a namespace.

Engineering Details

In OCPBUGS-15332 we noticed that the CRO was trying to scale during updates, but pre-InPlacePodVerticalScaling, the API didn't allow that, so we were making broken patches that were preventing any pod updates at all from succeeding.

We turned off UPDATE handling to fix that bug, but as InPlacePodVerticalScaling becomes more available we want to come back and add support for that.

The CRO is kind of a "less fancy" version of the VPA (we don't terminate pods, we don't track/manage reactions to changes, etc), and the VPA is in the process of adding support for InPlacePodVerticalScaling, so:

• There probably isn't any reason why we'd do this here for the CRO and then do it again separately for the VPA
• We should just wait until the VPA finishes implementing their enhancement (ideally we would help with that) and marking/handling all the corners, and then we should just come back here and do whatever it is that they do, e.g. https://github.com/kubernetes/autoscaler/issues/4016#issuecomment-1651625943

Acceptance Criteria

• UPDATE is re-enabled in the CRO
• CRO can handle in-place pod resource updates during UPDATE operations (only) when InPlacePodVerticalScaling is available
• We're reasonably sure that we can't generate a patch that will fail (e.g. one that would result in a change to pod QoS)
• selinux relabeling still only happens on a create (it will always fail on UPDATE)