Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-16157

Service 'ignition-ostree-transposefs-autosave-xfs' fails while booting the compute nodes with RHCOS image iso

XMLWordPrintable

    • Critical
    • No
    • Sprint 239 - Update&Remoting, Sprint 240 - Update&Remoting
    • 2
    • Proposed
    • ppc64le
    • False
    • Hide

      None

      Show
      None

      Description of problem:

      I am trying to deploy Hypershift Agent-based Installation using MCE 2.4.0. When I try to boot the compute nodes with generated RHCOS image iso, if fails with the following error:

      [   16.364578] ignition-ostree-growfs[1953]: data blocks changed from 843003 to 31357435
[   16.365213] XFS (dm-4): Unmounting Filesystem
[   16.368682] systemd[1]: Finished Ignition OSTree: Grow Root Filesystem.
[  OK  ] Finished Ignition OSTree: Grow Root Filesystem.
[   16.370378] systemd[1]: Mounting /sysroot...
         Mounting /sysroot...
[   16.371925] systemd[1]: Starting Ignition OSTree: Autosave XFS Rootfs Partition...
         Starting Ignition OSTree: ...tosave XFS Rootfs Partition...
[   16.381431] XFS (dm-4): Mounting V5 Filesystem
[   16.685770] XFS (dm-4): Ending clean mount
[  OK  ] Mounted /sysroot.
[   16.795604] systemd[1]: Mounted /sysroot.
[   16.974819] zram: Added device: zram0
[   16.975392] zram0: detected capacity change from 0 to 20971520
[   17.009481] XFS (zram0): Mounting V5 Filesystem
[   17.012547] XFS (zram0): Ending clean mount
[   17.015224] ignition-ostree-transposefs[1987]: Moving rootfs to RAM...
[   17.016256] ignition-ostree-transposefs[1987]: Mounting /dev/disk/by-label/root ro (/dev/dm-4) to /var/tmp/mnt
[   17.020762] dm-4: Can't mount, would change RO state
[   17.021151] ignition-ostree-transposefs[2103]: mount: /var/tmp/mnt: /dev/mapper/mpatha4 already mounted on /sysroot.
[   17.021483] systemd[1]: ignition-ostree-transposefs-autosave-xfs.service: Main process exited, code=exited, status=32/n/a
[   17.021684] systemd[1]: ignition-ostree-transposefs-autosave-xfs.service: Failed with result 'exit-code'.
[   17.021923] systemd[1]: Failed to start Ignition OSTree: Autosave XFS Rootfs Partition.
[FAILED] Failed to start Ignition O...Autosave XFS Rootfs Partition.
See 'systemctl status ignition-ostree-t...sefs-autosave-xfs.service' for details.
[   17.022390] systemd[1]: Dependency failed for Ignition Complete.
[DEPEND] Dependency failed for Ignition Complete.
[   17.022556] systemd[1]: Dependency failed for Initrd Default Target.
[DEPEND] Dependency failed for Initrd Default Target.
[   17.022710] systemd[1]: initrd.target: Job initrd.target/start failed with result 'dependency'.
[   17.022749] systemd[1]: initrd.target: Triggering OnFailure= dependencies.
[   17.023073] systemd[1]: ignition-complete.target: Job ignition-complete.target/start failed with result 'dependency'.
[   17.023111] systemd[1]: ignition-ostree-transposefs-autosave-xfs.service: Triggering OnFailure= dependencies.
[   17.032743] systemd[1]: Stopped target Initrd /usr File System.

      Version-Release number of selected component (if applicable):

      OCP 4.14.0-ec.3

      RHCOS: rhcos-414.92.202307070025-0-live.ppc64le.iso
      https://releases-rhcos-art.apps.ocp-virt.prod.psi.redhat.com/?stream=prod/streams/4.14-9.2&arch=ppc64le

      How reproducible:

      Always

      Steps to Reproduce:

      1. Installed multicluster-engine.v2.4.0-56 operator and created an instance of it.
      2. Created an AgentServiceConfig with the following details: 
        export DB_VOLUME_SIZE="10Gi"
export FS_VOLUME_SIZE="100Gi"
export OCP_VERSION="4.14.0"
export ARCH="ppc64le"
export OCP_RELEASE_VERSION="414.92.202307070025-0"
export ISO_URL="http://141.125.106.38/rhcos-414.92.202307070025-0-live.ppc64le.iso"
export ROOT_FS_URL="http://141.125.106.38/rhcos-414.92.202307070025-0-live-rootfs.ppc64le.img"

        3.  Created an OCP 4.14.0-ec.3 Hosted cluster

        hypershift create cluster agent \
	--name="hypershift" \
	--pull-secret=/root/pull-secret \
	--agent-namespace="clusters-hypershift" \
	--base-domain="ocp-powervs-ppc64le.com" \
	--api-server-address=api.hypershift.ocp-powervs-ppc64le.com \
	--ssh-key /root/id_rsa.pub \
	--release-image=quay.io/openshift-release-dev/ocp-release:4.14.0-ec.3-multi --render > render.yaml

               4. Created InfraEnv which generates ISO

      Actual results:
      Worker nodes fail to boot.
        

      Expected results:
      All nodes should boot up using the generated iso image, and cluster deployment should be successful.

      Additional info:

      This particular service got introduced via https://github.com/coreos/fedora-coreos-config/pull/2320 upstream

      Logs:
      worker-0 boot logs: https://drive.google.com/file/d/1l-pcFGqkvyPsTfFlaRnMypqyBLjwK_sz/view?usp=drive_link

            rhn-support-jmarrero Joseph Marrero Corchado
            aishwarya_kamat Aishwarya Kamat (Inactive)
            Aashish Radhakrishnan Aashish Radhakrishnan
            Votes:
            0 Vote for this issue
            Watchers:
            17 Start watching this issue

              Created:
              Updated:
              Resolved: