Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-7293

RHCOS 9.2 Failing to Bootstrap on Metal, OpenStack, vSphere (all baremetal runtime platforms)

    XMLWordPrintable

Details

    • Bug
    • Resolution: Done
    • Critical
    • None
    • 4.13.0
    • RHCOS
    • Important
    • Approved
    • False
    • Hide

      None

      Show
      None

    Description

      In preparation for landing the RHCOS update to RHEL 9.2, teams are performing /payload testing in a PR, and a problem is sufacing with metal jobs failing to bootstrap.

      The pull request is: https://github.com/openshift/machine-config-operator/pull/3485

      The last two payload attempts have shown a problem such as this on all metal blocking jobs:

      https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/openshift-machine-config-operator-3485-nightly-4.13-e2e-metal-ipi-ovn-ipv6/1623691787119890432

      https://prow.ci.openshift.org/view/gs/origin-ci-test/logs/openshift-machine-config-operator-3485-nightly-4.13-e2e-metal-ipi-sdn/1623691787870670848

      There's not much to go on, just looks like we fail to bootstrap within an hour, and then start gathering data, most of which fails:

      level=debug msg= Loading Platform...
      level=debug msg=Using Install Config loaded from state file
      level=info msg=Waiting up to 1h0m0s (until 4:19PM) for bootstrapping to complete...
      I0209 16:11:04.886200 87679 trace.go:219] Trace[1082817587]: "Reflector ListAndWatch" name:k8s.io/client-go/tools/watch/informerwatcher.go:146 (09-Feb-2023 16:10:51.414) (total time: 13471ms):
      Trace[1082817587]: ---"Objects listed" error:<nil> 13471ms (16:11:04.886)
      Trace[1082817587]: [13.471411761s] [13.471411761s] END

      Attachments

        1. audit.log.gz
          100 kB
          Derek Higgins
        2. image-2023-02-09-14-08-06-539.png
          25 kB
          Stephen Benjamin
        3. image-2023-02-09-14-12-06-860.png
          20 kB
          Stephen Benjamin
        4. image-2023-02-09-14-13-08-116.png
          15 kB
          Stephen Benjamin

        Issue Links

          blocks

          Epic - Created by Jira Software - do not edit or delete. Issue type for a big user story that needs to be broken down. COS-1926 Move RHCOS to RHEL 9.2 in OCP 4.13

          • Blocker - To be worked above all other priorities. This term is chosen when the severity of the issue is very high, has no workaround, or the effort for the change is comparatively low. Issues that may be very publicly visible and could generate significant media attention may also drive a higher priority.
          • Closed
          is blocked by

          Bug - A problem that impairs or prevents the functions of the product. OCPBUGS-7550 Selinux denials on EL8 based pods running atop RHCOS9

          • Critical - To be worked on immediately following BLOCKER issues.
          • Closed

          Bug - A problem that impairs or prevents the functions of the product. OCPBUGS-7707 /etc/NetworkManager/dispatcher.d needs to be relabeled during pivot from 8.6 to 9.2

          • Undefined - Priority not specified.
          • Closed
          is related to

          Story - Created by Jira Software - do not edit or delete. Issue type for a user story. OPNET-226 Investigate resolv-prepender issue

          • Undefined - Priority not specified.
          • Closed
          links to

          Web Link Add tmpfiles.d config to restore dispatcher scripts labels

          Web Link https://bugzilla.redhat.com/show_bug.cgi?id=2158746

          Activity

            People

              bnemec@redhat.com Benjamin Nemec
              rhn-engineering-dgoodwin Devan Goodwin
              Steeve Goveas Steeve Goveas
              Votes:
              0 Vote for this issue
              Watchers:
              22 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: