Uploaded image for project: 'OCP Technical Release Team'
  1. OCP Technical Release Team
  2. TRT-1222

Azure Resource Failures

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Blocker Blocker
    • None
    • None
    • False
    • None
    • False

      Multiple payload failures for azure in both 4.15 and 4.14.

      4.15.0-0.nightly-2023-09-09-023014 failed aggregated-azure-ovn-upgrade-4.15-micro

      Most common failures show multiple master (0,1, etc) nodes

      1700336473544331264

      openshift-e2e-test-build-log

      Sat Sep  9 04:02:53 UTC 2023 [20/20] - 7 Machines - 6 Nodes
      Timed out waiting for node count (6) to equal or exceed machine count (7).
      

      4.15.0-0.ci-2023-09-09-044704 failed aggregated-azure-sdn-upgrade-4.15-minor

       

      dperique edit: a way to see these failures where there are multiple masters is to use this query in search.ci looking for "7 Machines - 6 Nodes" in periodic e2e azure jobs.

      this shows the rough impact:

      $ podman run -it corbinu/alpine-w3m -dump -cols 200 "https://search.ci.openshift.org/?search=7+Machines+-+6+Nodes&maxAge=336h&context=1&type=build-log&name=periodic-ci-openshift-release-master-ci-4.14-e2e-azure&excludeName=&maxMatches=5&maxBytes=20971520&groupBy=job" |grep "failures match"
      
      periodic-ci-openshift-release-master-ci-4.14-e2e-azure-ovn-upgrade (all) - 340 runs, 41% failed, 3% of failures match = 1% impact
      periodic-ci-openshift-release-master-ci-4.14-e2e-azure-ovn (all) - 153 runs, 25% failed, 8% of failures match = 2% impact 

            rh-ee-fbabcock Forrest Babcock
            rh-ee-fbabcock Forrest Babcock
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

              Created:
              Updated:
              Resolved: