I spent some time looking through the CVO test failures and saw several instances of the apiserver being offline.  You can use this search.ci query to see it's a widespread problem impacting many components on metal.

This situation has been happening a few weeks so please forgive me if this Jira is a dupe of a known problem that is already being worked on.

Component Readiness has found a potential regression in [sig-arch][Feature:ClusterUpgrade] Cluster should remain functional during upgrade [Disruptive] [Serial].

Probability of significant regression: 100.00%

Sample (being evaluated) Release: 4.16
Start Time: 2024-03-19T00:00:00Z
End Time: 2024-03-25T23:59:59Z
Success Rate: 33.33%
Successes: 3
Failures: 6
Flakes: 0

Base (historical) Release: 4.15
Start Time: 2024-02-01T00:00:00Z
End Time: 2024-02-28T23:59:59Z
Success Rate: 97.87%
Successes: 46
Failures: 1
Flakes: 0

View the test details report at https://sippy.dptools.openshift.org/sippy-ng/component_readiness/test_details?arch=amd64&arch=amd64&baseEndTime=2024-02-28%2023%3A59%3A59&baseRelease=4.15&baseStartTime=2024-02-01%2000%3A00%3A00&capability=ClusterUpgrade&component=Cluster%20Version%20Operator&confidence=95&environment=ovn%20upgrade-minor%20amd64%20metal-ipi%20standard&excludeArches=arm64%2Cheterogeneous%2Cppc64le%2Cs390x&excludeClouds=openstack%2Cibmcloud%2Clibvirt%2Covirt%2Cunknown&excludeVariants=hypershift%2Cosd%2Cmicroshift%2Ctechpreview%2Csingle-node%2Cassisted%2Ccompact&groupBy=cloud%2Carch%2Cnetwork&ignoreDisruption=true&ignoreMissing=false&minFail=3&network=ovn&network=ovn&pity=5&platform=metal-ipi&platform=metal-ipi&sampleEndTime=2024-03-25%2023%3A59%3A59&sampleRelease=4.16&sampleStartTime=2024-03-19%2000%3A00%3A00&testId=openshift-tests-upgrade%3Acc7e2ba6166c37b611478c22a74bc749&testName=%5Bsig-arch%5D%5BFeature%3AClusterUpgrade%5D%20Cluster%20should%20remain%20functional%20during%20upgrade%20%5BDisruptive%5D%20%5BSerial%5D&upgrade=upgrade-minor&upgrade=upgrade-minor&variant=standard&variant=standard