Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-15062

MCO degraded during upgrade from 4.11 to 4.12 for rpm-ostree checksum error

    XMLWordPrintable

Details

    • Important
    • No
    • Rejected
    • False
    • Hide

      None

      Show
      None

    Description

      Description of problem:

      During an OpenShift upgrade from 4.11.41 to 4.12.18 all cluster operators upgraded except the machine-config because reached the degraded status. All the workers node and one master were upgraded successfully. So the upgrade is stuck on the second master to be updated, MCD error is:
      
      Marking Degraded due to: unexpected on-disk state validating against rendered-master-df8668e3a25e8ec50b1a9cc4cfbb16e7: expected target osImageURL "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:127670aaa6073e97bd6d919e1b57bd16b60435410b096aeefce7cd64f6c83d24"
      
      Solution https://access.redhat.com/solutions/5598401 was tried different times with no success.
      
      The expected image is pulled successfully.
      
      But the rpm-otree fails with a checksum error:
      
      Jun 10 16:29:13 master-1 systemd[1]: Started /usr/bin/podman run --env-file /etc/mco/proxy.env --privileged --pid=host --net=host --rm -v /:/run/host quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:127670aaa6073e97bd6d919e1b57bd16b60435410b096aeefce7cd64f6c83d24 rpm-ostree ex deploy-from-self /run/host.
      Jun 10 16:29:13 master-1 podman[43758]: 
      Jun 10 16:29:13 master-1 podman[43758]: NOTICE: Experimental commands are subject to change.
      Jun 10 16:29:16 master-1 podman[43758]: error: Invalid checksum of length 0 expected 32
      

      Version-Release number of selected component (if applicable):

      upgrade from 4.11.41 to 4.12.18

      How reproducible:

      Only at customer environment

      Steps to Reproduce:

      1.
      2.
      3.
      

      Actual results:

      Upgrade stuck.

      Expected results:

      Upgrade to complete.

      Additional info:

      ostree fsck doesn't report any error.

      Attachments

        Activity

          People

            walters@redhat.com Colin Walters
            rhn-support-cpassare Christian Passarelli
            Michael Nguyen Michael Nguyen
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: