Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-73802

bootstrap-e2e permafailing

XMLWordPrintable

    • None
    • False
    • Hide

      None

      Show
      None
    • 3
    • None
    • None
    • None
    • MCO Sprint 283
    • 1
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Description of problem:

      The bootstrap-e2e test target has been consistently failing since approximately August 12, 2025. Until that point, it was mostly passing.

      Going by the Prow job history, I think the last passing run before this trend started is: https://prow.ci.openshift.org/view/gs/test-platform-results/pr-logs/pull/openshift_machine-config-operator/5242/pull-ci-openshift-machine-config-operator-main-bootstrap-unit/1955281974511800320

       

      How reproducible:

      Always

       

      Steps to Reproduce:

      Either run:

      $ make bootstrap-e2e

       

      Or open a PR to the MCO repo which runs the test-bootstrap-unit CI job.

       

      Actual results

      The test eventually times out after what appears to be numerous retries. A example of this can be found here: https://prow.ci.openshift.org/view/gs/test-platform-results/pr-logs/pull/openshift_machine-config-operator/5551/pull-ci-openshift-machine-config-operator-main-bootstrap-unit/2011447593892057088

       

      Expected results:

      The test suite should pass.

       

      Additional info:

      Using git bisect, I ran the bootstrap-e2e test using the attached shell scripts. The git bisect log contains the following output:

       

      $ git bisect log
git bisect start
# status: waiting for both good and bad commits
# bad: [15a0d2f70fe3274de6a51d8b17dce0b7820b0b08] Merge pull request #5543 from andfasano/fix-iri-load-registry-file
git bisect bad 15a0d2f70fe3274de6a51d8b17dce0b7820b0b08
# status: waiting for good commit(s), bad commit known
# good: [9fe58600f95364de5a029a75ddc00e62a82bc525] Merge pull request #5237 from djoshy/revert-5215-delete-old-mcn-crd
git bisect good 9fe58600f95364de5a029a75ddc00e62a82bc525
# bad: [5ae01fccb85d7d439c788000b0085a210fcfc40b] Merge pull request #5268 from cheesesashimi/zzlotnik/fix-image-registry-error-handling
git bisect bad 5ae01fccb85d7d439c788000b0085a210fcfc40b
# good: [0ae855ee512c7f35a5de5160d9628673904535b2] MCO-1685: Add mco-sanitize output encryption
git bisect good 0ae855ee512c7f35a5de5160d9628673904535b2
# bad: [679b9b5206db33ae9ae3816bdc3f8db61c5d2488] Merge pull request #5351 from djoshy/ami-updater
git bisect bad 679b9b5206db33ae9ae3816bdc3f8db61c5d2488
# bad: [4fe83235d949e834e32cd13aed815b962ca8eba6] Merge pull request #5334 from shiftstack/shiftstack_owners
git bisect bad 4fe83235d949e834e32cd13aed815b962ca8eba6
# bad: [b6edf8e9963693fee23235617b145dcd644471e8] Merge pull request #5303 from pablintino/devex-mco-sanitize-main
git bisect bad b6edf8e9963693fee23235617b145dcd644471e8
# bad: [a3e78f3d6798fcf49896c7fb944d2af5e3d49bca] certcontroller: update unit tests
git bisect bad a3e78f3d6798fcf49896c7fb944d2af5e3d49bca
# bad: [11f0dc609fe062a48faf4fc817085ee5cf3bdb95] certcontroller: resync on infra object
git bisect bad 11f0dc609fe062a48faf4fc817085ee5cf3bdb95
# first bad commit: [11f0dc609fe062a48faf4fc817085ee5cf3bdb95] certcontroller: resync on infra object

       

      It looks like git bisect may have identified https://github.com/openshift/machine-config-operator/commit/11f0dc609fe062a48faf4fc817085ee5cf3bdb95 as the first bad commit. Although I'm not sure how / why.

        1. run-bootstrap-e2e-in-container.sh
          0.5 kB
        2. start-bisect.sh
          0.1 kB

              djoshy David Joshy
              zzlotnik@redhat.com Zack Zlotnik
              None
              None
              Sergio Regidor de la Rosa Sergio Regidor de la Rosa
              None
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated: