Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-27822

4.14 tech preview jobs failing on prometheus tests

    XMLWordPrintable

Details

    • Bug
    • Resolution: Done-Errata
    • Critical
    • 4.14.z
    • 4.14
    • None
    • No
    • False
    • Hide

      None

      Show
      None
    • Release Note Not Required
    • In Progress

    Description

      Several of the 4.14 tech preview jobs have been permfailing since January 20th:
      https://prow.ci.openshift.org/job-history/gs/test-platform-results/logs/periodic-ci-openshift-release-master-ci-4.14-e2e-aws-sdn-techpreview

      example failures:
      https://prow.ci.openshift.org/view/gs/test-platform-results/logs/periodic-ci-openshift-release-master-ci-4.14-e2e-aws-sdn-techpreview/1749740678981619712
      https://prow.ci.openshift.org/view/gs/test-platform-results/logs/periodic-ci-openshift-release-master-ci-4.14-e2e-azure-sdn-techpreview/1749740680076333056
      https://prow.ci.openshift.org/view/gs/test-platform-results/logs/periodic-ci-openshift-release-master-ci-4.14-e2e-gcp-sdn-techpreview/1749740673168314368

      All the jobs fail in the same way:
      fail [github.com/openshift/origin/test/extended/prometheus/prometheus.go:459]: possibly some services didn't register ServiceMonitors to allow metrics collection

      It seems that some resource must not be getting installed in techpreview clusters which is causing this test to fail.

      Unfortunately these jobs failing prevents us from accepting 4.14 payloads, which makes this extremely important to resolve quickly.

      https://amd64.ocp.releases.ci.openshift.org/releasestream/4.14.0-0.nightly/release/4.14.0-0.nightly-2024-01-20-231201 represents the first payload that showed these failures but I did not see anything obviously suspect in the commit changes between it and the last accepted payload.

      The failing tech preview jobs block payload acceptance because this payload acceptance job depends on consuming results from the tech preview jobs: https://prow.ci.openshift.org/view/gs/test-platform-results/logs/periodic-ci-openshift-release-master-nightly-4.14-overall-analysis-all/1749818281608351744

      Attachments

        Issue Links

          Activity

            People

              trking W. Trevor King
              bparees@redhat.com Ben Parees
              Dinesh Kumar S Dinesh Kumar S
              Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: