4.15 Job is at 14% pass rate over last two weeks.

4.14 Job is at 17% pass rate over last two weeks

There are a lot of aspects to consider here,

1. Longstanding high control plane CPU that got worse in 4.14 (OCPBUGS-19388)
2. Increased ConfigMap and Secret requests during E2E runs
3. cgroupsv2 performance regression in 4.14 and 4.15 (OCPBUGS-18965)
4. Possible increase to vm size for gcp

The actual test failures seems to jump around a bit but we are picking up some of the same worst offenders in both 4.14 and 4.15.

The failed jobs seem to hit 20-30 failed tests in 1-2 batches, they do not seem to correlate overwhelmingly with disruption. The test failures seem to all be failing to make requests to the API. (why would disruption not fail in this time?)

They do seem to overlap with high cpu alerts.

Examples:

https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/periodic-ci-openshift-release-master-ci-4.15-upgrade-from-stable-4.14-e2e-gcp-ovn-rt-upgrade/1705353727109500928

https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/periodic-ci-openshift-release-master-ci-4.15-upgrade-from-stable-4.14-e2e-gcp-ovn-rt-upgrade/1705353728791416832