Loading...

XML

Word

Printable

Type: Bug
Resolution: Unresolved
Priority: Major
Fix Version/s: 4.17.0
Affects Version/s: 4.16, 4.17
Component/s: RHCOS
Labels:

Severity:
Moderate
Regression:
No
Story Points:
5
Sprint:
255 - Integration & Delivery
sprint_count:
1
Release Blocker:
Rejected
Blocked:
False
Blocked Reason:

Hide

None

Show
None
Release Note Text:

Hide
Cause: Enabling LUKS encryption on a system using 512 emulation disks (aka 512e disks) fails at the "ignition-ostree-growfs" step with "Device size is not aligned to requested sector size". This happens because of an alignment bug in sfdisk when growing a partition.

Consequence: The system fails provisioning and enters the emergency shell in the initramfs.

Fix: A workaround was added in ignition-ostree-growfs to detect this situation and fix the alignment.

Result: The system no longer fails provisioning.

Show
Cause: Enabling LUKS encryption on a system using 512 emulation disks (aka 512e disks) fails at the "ignition-ostree-growfs" step with "Device size is not aligned to requested sector size". This happens because of an alignment bug in sfdisk when growing a partition. Consequence: The system fails provisioning and enters the emergency shell in the initramfs. Fix: A workaround was added in ignition-ostree-growfs to detect this situation and fix the alignment. Result: The system no longer fails provisioning.
Release Note Type:
Bug Fix
Release Note Status:
In Progress
Target Version:

4.17.0
Target Backport Versions:

4.16.z

SFDC Cases Counter:
SFDC Cases Open:
SFDC Cases Links:

Description of problem:

When installing 4.16 UPI BM with TANG or TPM disk_encryption, the ignition fails with the following errors:

~~~
Entering emergency mode. Exit the shell to continue.
Type "journalctl" to view system logs.
You might want to save "/run/initramfs/rdsosreport.txt" to a USB stick or /boot
after mounting them and attach it to a bug report.

:/# systemctl status ignition-ostree-growfs.service --no-pager -l
× ignition-ostree-growfs.service - Ignition OSTree: Grow Root Filesystem
     Loaded: loaded (/usr/lib/systemd/system/ignition-ostree-growfs.service; enabled; preset: enabled)
     Active: failed (Result: exit-code) since Wed 2024-06-12 19:52:32 UTC; 1min 31s ago
    Process: 3725 ExecStart=/usr/sbin/ignition-ostree-growfs (code=exited, status=1/FAILURE)
   Main PID: 3725 (code=exited, status=1/FAILURE)
        CPU: 242msJun 12 19:52:29 localhost systemd[1]: Starting Ignition OSTree: Grow Root Filesystem...
Jun 12 19:52:30 localhost ignition-ostree-growfs[3753]: CHANGED: partition=4 start=1050624 old: size=6037504 end=7088127 new: size=936652431 end=937703054
Jun 12 19:52:32 localhost ignition-ostree-growfs[3889]: Device size is not aligned to requested sector size.
Jun 12 19:52:32 localhost systemd[1]: ignition-ostree-growfs.service: Main process exited, code=exited, status=1/FAILURE
Jun 12 19:52:32 localhost systemd[1]: ignition-ostree-growfs.service: Failed with result 'exit-code'.
Jun 12 19:52:32 localhost systemd[1]: Failed to start Ignition OSTree: Grow Root Filesystem.:/# systemctl status ignition-ostree-transposefs-save.service --no-pager -l
× ignition-ostree-transposefs-save.service - Ignition OSTree: Save Partitions
     Loaded: loaded (/usr/lib/systemd/system/ignition-ostree-transposefs-save.service; enabled; preset: enabled)
     Active: failed (Result: exit-code) since Wed 2024-06-12 19:52:47 UTC; 2min 48s ago
   Duration: 32.567s
    Process: 4654 ExecStart=/usr/libexec/ignition-ostree-transposefs save (code=exited, status=32)
   Main PID: 4654 (code=exited, status=32)
        CPU: 4msJun 12 19:52:46 localhost systemd[1]: Starting Ignition OSTree: Save Partitions...
Jun 12 19:52:46 localhost ignition-ostree-transposefs[4654]: Moving rootfs to RAM...
Jun 12 19:52:46 localhost ignition-ostree-transposefs[4654]: Mounting /dev/disk/by-label/root ro (/dev/dm-0) to /var/tmp/mnt
Jun 12 19:52:46 localhost ignition-ostree-transposefs[4657]: mount: /var/tmp/mnt: /dev/mapper/root already mounted on /sysroot.
Jun 12 19:52:47 localhost systemd[1]: ignition-ostree-transposefs-save.service: Main process exited, code=exited, status=32/n/a
Jun 12 19:52:47 localhost systemd[1]: ignition-ostree-transposefs-save.service: Failed with result 'exit-code'.
Jun 12 19:52:47 localhost systemd[1]: Failed to start Ignition OSTree: Save Partitions.
Jun 12 19:52:47 localhost systemd[1]: ignition-ostree-transposefs-save.service: Triggering OnFailure= dependencies.:/# lsblk 
NAME     MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINTS
sda        8:0    0 447.1G  0 disk  
sdb        8:16   0 447.1G  0 disk  
|-sdb1     8:17   0     1M  0 part  
|-sdb2     8:18   0   127M  0 part  
|-sdb3     8:19   0   384M  0 part  
`-sdb4     8:20   0 446.6G  0 part  
  `-root 253:0    0   2.9G  0 crypt /sysroot/sysroot/ostree/deploy/rhcos/var
                                    /sysroot/usr
                                    /sysroot/etc
                                    /sysroot
                                    /sysroot/sysroot
                                    
:/# lsblk -t /dev/sdb
NAME   ALIGNMENT MIN-IO OPT-IO PHY-SEC LOG-SEC ROTA SCHED      RQ-SIZE  RA WSAME
sdb            0   4096      0    4096     512    0 mq-deadline
|                                                                   64 128    0B
|-sdb1         0   4096      0    4096     512    0 mq-deadline
|                                                                   64 128    0B
|-sdb2         0   4096      0    4096     512    0 mq-deadline
|                                                                   64 128    0B
|-sdb3         0   4096      0    4096     512    0 mq-deadline
|                                                                   64 128    0B
`-sdb4         0   4096      0    4096     512    0 mq-deadline
                                                                    64 128    0B
  `-root
               0   4096      0    4096    4096    0                    128    0B
~~~

Latest "4.15.0-0.nightly-2024-06-12-223752" is not showing this problem and the error was confirmed with SNO and Ha topologies, please look into this ASAP.

Version-Release number of selected component (if applicable):

4.16.0-0.nightly-2024-06-13-031814

How reproducible:

Always

Steps to Reproduce:

1. Create butane template to enable TPM or TANG disk encryption as per official doc (https://docs.openshift.com/container-platform/4.15/installing/install_config/installing-customizing.html#installation-special-config-storage-procedure_installing-customizing)
2. Do not enable disk_mirroring, only disk_encryption
3. Deploy UPI BM

Actual results:

Ignition failure (ignition-ostree-growfs.service + ignition-ostree-transposefs-save.service)

Expected results:

Successful ignition & installation

Additional info:

Possible upstream related bug https://github.com/coreos/fedora-coreos-tracker/issues/1384

rdsosreport files from SNO deploy available here: https://drive.google.com/drive/folders/1Qj3XP7O79QoWOEq8YjqopqWKINRoDd5c?usp=sharing

is cloned by

OCPBUGS-36147 [4.16] Resizing LUKS on 512e disk causes ignition-ostree-growfs to fail with "Device size is not aligned to requested sector size."