Uploaded image for project: 'OCP Technical Release Team'
  1. OCP Technical Release Team
  2. TRT-1361

Hypershift failures blocking CI payloads

    XMLWordPrintable

Details

    • Story
    • Resolution: Done
    • Blocker
    • None
    • None
    • False
    • None
    • False

    Description

      Two payloads in a row, first had more failures, second had less but still broken.

      Both exhibit this status on the console operator:

        status:
          conditions:
          - lastTransitionTime: "2023-11-17T06:06:57Z"
            message: 'OAuthClientSyncDegraded: the server is currently unable to handle
              the request (get oauthclients.oauth.openshift.io console)'
            reason: OAuthClientSync_FailedRegister
            status: "True"
            type: Degraded
      

      https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/origin-ci-test/logs/periodic-ci-openshift-hypershift-release-4.15-periodics-e2e-aws-ovn/1725383840110743552/artifacts/e2e-aws-ovn/run-e2e/artifacts/TestUpgradeControlPlane/hostedcluster-example-dcxq4/cluster-scoped-resources/config.openshift.io/clusteroperators.yaml

      https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/origin-ci-test/logs/periodic-ci-openshift-hypershift-release-4.15-periodics-e2e-aws-ovn/1725305112194191360/artifacts/e2e-aws-ovn/run-e2e/artifacts/TestUpgradeControlPlane/hostedcluster-example-c7bz4/cluster-scoped-resources/config.openshift.io/clusteroperators.yaml

      We are suspicious of this PR, however this change was before the payloads started failing, perhaps the issue only surfaces on upgrades once the change was in an accepted payload: https://github.com/openshift/console-operator/pull/808

      There is also a hypershift PR that was only present in second failed payload, possibly a reaction to the problem but didn't fully fix? There were less failures in the second payload than the first: https://github.com/openshift/hypershift/pull/3151 ? If so, this will complicate a revert.

      Discussion: https://redhat-internal.slack.com/archives/C01C8502FMM/p1700226091335339

      Attachments

        Activity

          People

            rhn-engineering-dgoodwin Devan Goodwin
            rhn-engineering-dgoodwin Devan Goodwin
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: