Uploaded image for project: 'Red Hat Advanced Cluster Management'
  1. Red Hat Advanced Cluster Management
  2. ACM-19146

Assisted image service pod stuck in `CrashLoopBackOff` state with MCE version 2.8.0

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Blocker Blocker
    • MCE 2.8.1
    • ACM 2.13.0, MCE 2.8.0
    • Installer
    • Quality / Stability / Reliability
    • False
    • Hide

      None

      Show
      None
    • False
    • Critical
    • ?
    • None

      Description of the problem:

      When setting up CIM for host inventory in MCE 2.8.0, the assisted image service pod is stuck in `CrashLoopBackOff` state. The assisted image service pod logs suggest the image `https://mirror.openshift.com/pub/openshift-v4/x86_64/dependencies/rhcos/pre-release/dev-4.19/rhcos-dev-4.19-x86_64-live.x86_64.iso` is being pulled and is failing with a 404 error.

      ~~~

      {"file":"/remote-source/app/main.go:132","func":"main.main.func1","level":"fatal","msg":"Failed to populate image store: failed to download https://mirror.openshift.com/pub/openshift-v4/x86_64/dependencies/rhcos/pre-release/dev-4.19/rhcos-dev-4.19-x86_64-live.x86_64.iso: request to https://mirror.openshift.com/pub/openshift-v4/x86_64/dependencies/rhcos/pre-release/dev-4.19/rhcos-dev-4.19-x86_64-live.x86_64.iso returned error code 404\n","time":"2025-03-21T08:23:10Z"}

      ~~~

      The problematic image is not present in your mirror. The image names seem to be mismatching.  

       - The working from mirror.openshift.com - 

      https://mirror.openshift.com/pub/openshift-v4/x86_64/dependencies/rhcos/pre-release/dev-4.19/rhcos-dev-4.19-x86_64-live-iso.x86_64.iso

      • Problematic one from error -

      https://mirror.openshift.com/pub/openshift-v4/x86_64/dependencies/rhcos/pre-release/dev-4.19/rhcos-dev-4.19-x86_64-live.x86_64.iso

      The stateful set for assisted image service pod has the problematic image which lacks `-iso`in `rhcos-dev-4.19-x86_64-live.x86_64.iso`.

      How reproducible:

       In a 4.18.2 cluster, install the MCE 2.8.0 operator. Once installed enable Central Infrastructure Management from UI. After a while the pod/assisted-image-service-0 crashes.

      Steps to reproduce:

      1. Install MCE version 2.8.0 or upgrade the MCE to version 2.8.0 in a 4.18.2 cluster. 

      2. Enable Central Infrastructure Management from UI.

      3. 

      Actual results:

      The assisted image service pod is stuck in `CrashLoopBackOff` state and tries to fetch incorrect image `The assisted image service pod `.

      Expected results:

      The ** assisted image service pod should pull all images and succeed.

      Additional Notes - 

      • The below drive link has the MCE must gather and hub cluster must gather. 

      Link - https://drive.google.com/drive/folders/14dL_80A1Aw6jo0lqlS21HlHgYsoDGN_A?usp=sharing

      • The customer tried with just MCE 2.8.0. I was able to replicate with ACM 2.13 and MCE 2.8.0 versions by upgrading MCE to 2.8.0.

              rh-ee-msmigiel Matthew Smigielski
              rhn-support-adikulka Aditya Kulkarni
              Matthew Smigielski Matthew Smigielski
              Votes:
              2 Vote for this issue
              Watchers:
              10 Start watching this issue

                Created:
                Updated:
                Resolved: