-
Bug
-
Resolution: Unresolved
-
Normal
-
None
-
4.17.0
-
Moderate
-
None
-
False
-
TRT tooling has detected a bug where we appear to occasionally be briefly losing host-to-pod networking from a worker to all three masters during the time when the network operator is upgrading.
The the below job runs, look for host-to-pod 1s outages to multiple endpoints. Closer examination seems to show that the outage is all originating from the same host and only affects new connections.
Sample job runs:
https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/periodic-ci-openshift-release-master-ci-4.17-e2e-azure-ovn-upgrade/1828143251106828288
Intervals: https://sippy.dptools.openshift.org/sippy-ng/job_runs/1828143251106828288/periodic-ci-openshift-release-master-ci-4.17-e2e-azure-ovn-upgrade/intervals?filterText=&intervalFile=e2e-timelines_spyglass_20240826-195928.json&overrideDisplayFlag=1&selectedSources=OperatorAvailable&selectedSources=OperatorProgressing&selectedSources=OperatorDegraded&selectedSources=KubeletLog&selectedSources=EtcdLog&selectedSources=EtcdLeadership&selectedSources=Alert&selectedSources=Disruption&selectedSources=E2EFailed&selectedSources=APIServerGracefulShutdown&selectedSources=KubeEvent
https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/periodic-ci-openshift-release-master-ci-4.17-e2e-azure-ovn-upgrade/1828143244467245056
Intervals: https://sippy.dptools.openshift.org/sippy-ng/job_runs/1828143244467245056/periodic-ci-openshift-release-master-ci-4.17-e2e-azure-ovn-upgrade/intervals?filterText=&intervalFile=e2e-timelines_spyglass_20240826-200245.json&overrideDisplayFlag=1&selectedSources=OperatorAvailable&selectedSources=OperatorProgressing&selectedSources=OperatorDegraded&selectedSources=KubeletLog&selectedSources=EtcdLog&selectedSources=EtcdLeadership&selectedSources=Alert&selectedSources=Disruption&selectedSources=E2EFailed&selectedSources=APIServerGracefulShutdown&selectedSources=KubeEvent&selectedSources=NodeState
https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/periodic-ci-openshift-release-master-ci-4.17-e2e-azure-ovn-upgrade/1828143239379554304
Intervals: https://sippy.dptools.openshift.org/sippy-ng/job_runs/1828143239379554304/periodic-ci-openshift-release-master-ci-4.17-e2e-azure-ovn-upgrade/intervals?filterText=&intervalFile=e2e-timelines_spyglass_20240826-200958.json&overrideDisplayFlag=1&selectedSources=OperatorAvailable&selectedSources=OperatorProgressing&selectedSources=OperatorDegraded&selectedSources=KubeletLog&selectedSources=EtcdLog&selectedSources=EtcdLeadership&selectedSources=Alert&selectedSources=Disruption&selectedSources=E2EFailed&selectedSources=APIServerGracefulShutdown&selectedSources=KubeEvent&selectedSources=NodeState
https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/periodic-ci-openshift-release-master-ci-4.17-e2e-azure-ovn-upgrade/1826046480729772032
Intervals: https://sippy.dptools.openshift.org/sippy-ng/job_runs/1826046480729772032/periodic-ci-openshift-release-master-ci-4.17-e2e-azure-ovn-upgrade/intervals?filterText=&intervalFile=e2e-timelines_spyglass_20240821-011912.json&overrideDisplayFlag=1&selectedSources=OperatorAvailable&selectedSources=OperatorProgressing&selectedSources=OperatorDegraded&selectedSources=KubeletLog&selectedSources=EtcdLog&selectedSources=EtcdLeadership&selectedSources=Alert&selectedSources=Disruption&selectedSources=E2EFailed&selectedSources=APIServerGracefulShutdown&selectedSources=KubeEvent&selectedSources=NodeState