-
Bug
-
Resolution: Unresolved
-
Undefined
-
None
-
rhel-9.7
-
None
-
None
-
Low
-
1
-
rhel-image-mode
-
2
-
False
-
False
-
-
None
-
Bootc Sprint #21
-
None
-
None
-
Unspecified
-
Unspecified
-
Unspecified
-
-
All
-
None
What were you trying to do that didn't work?
Our customer uses the last bootc image, but the issue can be reproduced with R4E as well. He cannot start the ostree-state-overlay@opt.service that fails, and using `ostree admin state-overlay opt /opt` does not work either. The issue is easily reproducible and I'm still wondering whether a pre-requisite ostree command has not been forgotten here, or whether this method is considered as deprecated. Bootc upstream doc suggests the use of state overlays:
https://bootc-dev.github.io/bootc/filesystem.html#enabling-state-overlays
But it also recommends to simply symlink the parts of the /opt needed into /var in the subsequent section.
What is the impact of this issue to you?
Quoting the customer: "Not having a state overlay for `/opt` renders Image mode unusable without costly workarounds as it hampers the operation of our vulnarability scanner fleet."
Please provide the package NVR for which the bug is seen:
https://registry.redhat.io/rhel9/rhel-bootc:9.7-1762991756
ostree-2025.6-1.el9_7.x86_64
How reproducible is this bug?:
Always
Steps to reproduce
- Use the latest bootc image
- `systemctl enable --now ostree-state-overlay@opt.service`
- OR `ostree admin state-overlay opt /opt`
Expected results
# mount | grep -e ^overlay -e /opt overlay on /opt type overlay (rw,relatime,seclabel,lowerdir=/opt,upperdir=/var/ostree/state-overlays/opt/upper,workdir=/var/ostree/state-overlays/opt/work)
Actual results
# systemctl status ostree-state-overlay@opt.service
× ostree-state-overlay@opt.service - OSTree State Overlay On /opt
Loaded: loaded (/usr/lib/systemd/system/ostree-state-overlay@.service; enabled; preset: disabled)
Active: failed (Result: exit-code) since Wed 2025-11-26 09:59:17 UTC; 7s ago
Docs: man:ostree(1)
Process: 1244 ExecStart=/usr/bin/ostree admin state-overlay opt /opt (code=exited, status=1/FAILURE)
Main PID: 1244 (code=exited, status=1/FAILURE)
CPU: 24ms
Nov 26 09:59:17 localhost.localdomain systemd[1]: Starting OSTree State Overlay On /opt...
Nov 26 09:59:17 localhost.localdomain ostree[1244]: error: lgetxattr(user.ostree.deploymentcsum): No data available
Nov 26 09:59:17 localhost.localdomain systemd[1]: ostree-state-overlay@opt.service: Main process exited, code=exited, status=1/FAILURE
Nov 26 09:59:17 localhost.localdomain systemd[1]: ostree-state-overlay@opt.service: Failed with result 'exit-code'.
Nov 26 09:59:17 localhost.localdomain systemd[1]: Failed to start OSTree State Overlay On /opt.
Possible workaround
attr -s ostree.deploymentcsum -V $(bootc status --format json | jq -r .status.booted.ostree.checksum) /var/ostree/state-overlays/opt/upper
Note: no idea if the above workaround is suitable for production.
Additional notes
Breakpoint 1, ot_admin_builtin_state_overlay (argc=3, argv=0x7ffe3f5b2ad8, invocation=0x7ffe3f5b2890, cancellable=0x0, error=0x7ffe3f5b2908)
at src/ostree/ot-admin-builtin-state-overlay.c:239
239 {
(gdb) list
234
235 /* Called by ostree-state-overlay@.service. */
236 gboolean
237 ot_admin_builtin_state_overlay (int argc, char **argv, OstreeCommandInvocation *invocation,
238 GCancellable *cancellable, GError **error)
239 {
240 g_autoptr (GOptionContext) context = g_option_context_new ("NAME MOUNTPATH");
241 g_autoptr (OstreeSysroot) sysroot = NULL;
242
243 /* First parse the args without loading the sysroot to see what options are
[...]
(gdb) list
62 return TRUE;
63 }
64
65 /* XXX: upstream to libglnx */
66 static gboolean
67 lgetxattrat_allow_noent (int dfd, const char *path, const char *attribute, GBytes **out_bytes,
68 GError **error)
69 {
70 g_autoptr (GError) local_error = NULL;
71 *out_bytes = glnx_lgetxattrat (dfd, path, attribute, &local_error);
(gdb) n
70 g_autoptr (GError) local_error = NULL;
(gdb) n
71 *out_bytes = glnx_lgetxattrat (dfd, path, attribute, &local_error);
(gdb) p path
$9 = 0x55ddca4b43e8 "upper"
(gdb) p attribute
$10 = 0x55ddca4b43ee "user.ostree.deploymentcsum"
(gdb) n
72 if (!*out_bytes)
(gdb) p *out_bytes
$11 = (GBytes *) 0x0
(gdb) p local_error->message
$13 = (gchar *) 0x55ddcbb7b2f0 "lgetxattr(user.ostree.deploymentcsum): No data available"
As per my understanding, by default this user xattr being not set on my OS after the new deploy, the code from libostree returns the error after getting the ENODATA errno.
