Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-13532

Host can get stuck on inspecting if the network secret is updated

    XMLWordPrintable

Details

    • Important
    • No
    • Approved
    • False
    • Hide

      None

      Show
      None
    • Hide
      08/08: The fix is verified and will be merged and backported.

      8/1: tracking as telco QE blocking since OCPBUGS-15946 was closed as a dupe of this
      Show
      08/08: The fix is verified and will be merged and backported. 8/1: tracking as telco QE blocking since OCPBUGS-15946 was closed as a dupe of this

    Description

      Description of problem:

      It seems that we don't correctly update the network data secret version in the PreprovisioningImage, resulting in BMO assuming that the image is still stale, while the image-customization-controller assumes it's done. As a result, the host is stuck in inspecting.

      How reproducible:

      What I think I did is to add a network data secret to a host which already has a preprovisioningimage previously created. I need to check if I can repeat it.

      Actual results:

      Host in inspecting, BMO logs show

      {"level":"info","ts":"2023-05-11T11:52:52.348Z","logger":"controllers.BareMetalHost","msg":"network data in pre-provisioning image is out of date","baremetalhost":"openshift-machine-api/oste
st-extraworker-0","provisioningState":"inspecting","latestVersion":"9055823","currentVersion":"9055820"}

      Indeed, the image has the old version:

      status:
  architecture: x86_64
  conditions:
  - lastTransitionTime: "2023-05-11T11:27:51Z"
    message: Generated image
    observedGeneration: 1
    reason: ImageSuccess
    status: "True"
    type: Ready
  - lastTransitionTime: "2023-05-11T11:27:51Z"
    message: ""
    observedGeneration: 1
    reason: ImageSuccess
    status: "False"
    type: Error
  format: iso
  imageUrl: http://metal3-image-customization-service.openshift-machine-api.svc.cluster.local/231b39d5-1b83-484c-9096-aa87c56a222a
  networkData:
    name: ostest-extraworker-0-network-config-secret
    version: "9055820"

      What I find puzzling is that we even have two versions of the secret. I only created it once.

      Attachments

        Issue Links

          depends on

          Bug - A problem that impairs or prevents the functions of the product. OCPBUGS-17363 BMH is not reconciled on Secret change

          • Undefined - Priority not specified.
          • Closed
          is cloned by

          Bug - A problem that impairs or prevents the functions of the product. OCPBUGS-14250 [4.13] Host can get stuck on inspecting if the network secret is updated

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed
          is depended on by

          Bug - A problem that impairs or prevents the functions of the product. OCPBUGS-14250 [4.13] Host can get stuck on inspecting if the network secret is updated

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Closed
          is duplicated by

          Bug - A problem that impairs or prevents the functions of the product. OCPBUGS-16636 Failed to install BM cluster with day1 NMState networking

          • Undefined - Priority not specified.
          • Closed
          links to

          GitHub openshift/image-customization-controller#88: OCPBUGS-13532: Respond to changes in Secrets

          GitHub openshift/image-customization-controller#90: OCPBUGS-13532: Actually respond to changes in Secrets

          GitHub openshift/image-customization-controller#94: OCPBUGS-13532: Reconcile when networkData Secret changes

          Web Link RHSA-2023:5006 OpenShift Container Platform 4.14.z security update

          Activity

            People

              zabitter Zane Bitter
              rhn-engineering-dtantsur Dmitry Tantsur
              Steeve Goveas Steeve Goveas
              Votes:
              0 Vote for this issue
              Watchers:
              13 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: