What were you trying to do that didn't work?

Installing httpd on a machine already deployed in image-mode.

More details about the root cause and possible solutions: https://bootc-dev.github.io/bootc/filesystem.html#var

What is the impact of this issue to you?

httpd does not start

Please provide the package NVR for which the bug is seen:

httpd-2.4.63-1.el10.x86_64
or any recent httpd

How reproducible is this bug?:

always

Steps to reproduce

1. Run a machine in Image Mode. For example there are images used by testing-farm.

$ tmt run provision -h virtual --image http://images.osci.redhat.com/RHEL-10.1-image-mode-x86_64.qcow2 login

.... snip

bash-5.2# bootc status 
● Booted image: images.paas.redhat.com/testingfarm/rhel-bootc:10.1
        Digest: sha256:007ea1cbc9f4899e554f4c422ee6be40c6fc6a54a633fae6e3abf771820b2029 (amd64)
       Version: 10.1 (2025-05-15T21:45:31Z)
bash-5.2# 

2 .Prepare a Containerfile and build it

bash-5.2# cat Containerfile 
FROM images.paas.redhat.com/testingfarm/rhel-bootc:10.1
RUN dnf -y install httpd && dnf -y clean allbash-5.2# 
bash-5.2# podman build -t localhost/test .
STEP 1/2: FROM images.paas.redhat.com/testingfarm/rhel-bootc:10.1...Running transaction
  Preparing        :                                                        1/1 
  Installing       : apr-1.7.5-2.el10.x86_64                               1/12 
  Installing       : apr-util-lmdb-1.6.3-21.el10.x86_64                    2/12 
  Installing       : apr-util-openssl-1.6.3-21.el10.x86_64                 3/12 
  Installing       : apr-util-1.6.3-21.el10.x86_64                         4/12 
  Installing       : httpd-tools-2.4.63-1.el10.x86_64                      5/12 
  Installing       : redhat-logos-httpd-100.0-4.el10.noarch                6/12 
  Running scriptlet: httpd-filesystem-2.4.63-1.el10.noarch                 7/12 
  Installing       : httpd-filesystem-2.4.63-1.el10.noarch                 7/12 
  Installing       : mailcap-2.1.54-8.el10.noarch                          8/12 
  Installing       : httpd-core-2.4.63-1.el10.x86_64                       9/12 
  Installing       : mod_http2-2.0.29-2.el10.x86_64                       10/12 
  Installing       : mod_lua-2.4.63-1.el10.x86_64                         11/12 
  Installing       : httpd-2.4.63-1.el10.x86_64                           12/12 
  Running scriptlet: httpd-2.4.63-1.el10.x86_64                           12/12 
Installed products updated.Installed:
  apr-1.7.5-2.el10.x86_64                apr-util-1.6.3-21.el10.x86_64          
  apr-util-lmdb-1.6.3-21.el10.x86_64     apr-util-openssl-1.6.3-21.el10.x86_64  
  httpd-2.4.63-1.el10.x86_64             httpd-core-2.4.63-1.el10.x86_64        
  httpd-filesystem-2.4.63-1.el10.noarch  httpd-tools-2.4.63-1.el10.x86_64       
  mailcap-2.1.54-8.el10.noarch           mod_http2-2.0.29-2.el10.x86_64         
  mod_lua-2.4.63-1.el10.x86_64           redhat-logos-httpd-100.0-4.el10.noarch Complete!
41 files removed
COMMIT localhost/test
--> ddae77e2492e
Successfully tagged localhost/test:latest
ddae77e2492e4766e296c75f79d02febd366ed05fc93abc92aa94594fff93bab
bash-5.2# 

3. Switch to the new version and reboot so the new image is applied

bash-5.2# bootc switch --transport containers-storage localhost/test
layers already present: 0; layers needed: 68 (1.7 GB)
Fetched layers: 1.56 GiB in 10 seconds (156.95 MiB/s)                                                                                         Deploying: done (2 seconds)                                                                                                               Queued for next boot: ostree-unverified-image:containers-storage:localhost/test
  Version: 10.1
  Digest: sha256:dc7058a860fe10aad4310489d6c2042d203ea4f4c376cfc4e5b7e3dda20fbfaf
bash-5.2# reboot

...

4. Check the httpd package installation

bash-5.2# bootc status
● Booted image: containers-storage:localhost/test
        Digest: sha256:dc7058a860fe10aad4310489d6c2042d203ea4f4c376cfc4e5b7e3dda20fbfaf (amd64)
       Version: 10.1 (2025-05-20T09:42:24Z)  Rollback image: images.paas.redhat.com/testingfarm/rhel-bootc:10.1
          Digest: sha256:007ea1cbc9f4899e554f4c422ee6be40c6fc6a54a633fae6e3abf771820b2029 (amd64)
         Version: 10.1 (2025-05-15T21:45:31Z)
bash-5.2# 
bash-5.2# rpm -qa httpd*
httpd-tools-2.4.63-1.el10.x86_64
httpd-filesystem-2.4.63-1.el10.noarch
httpd-core-2.4.63-1.el10.x86_64
httpd-2.4.63-1.el10.x86_64
bash-5.2# 
bash-5.2# for i in `rpm -ql httpd-filesystem`; do ls -d $i >/dev/null; done
ls: cannot access '/var/www': No such file or directory
ls: cannot access '/var/www/cgi-bin': No such file or directory
ls: cannot access '/var/www/html': No such file or directory
bash-5.2#

Expected results

/var/lib/ dirs and files are created and the service can be started

Note

I have not tested but I expect that when the httpd package is created during the initial image creation all files and dirs are created as expected. The issue is present only when the existing image is updated.