-
Bug
-
Resolution: Unresolved
-
Undefined
-
None
-
4.21
-
None
-
False
-
-
None
-
Moderate
-
None
-
None
-
None
-
None
-
None
-
None
-
None
-
None
-
None
-
None
-
None
-
None
Description of problem:
In preperation to sanity check LCA operator deployed via OLMv1 (which currently requires the TechPreviewNoUpgrade feature gate enabled).
First was to enable the feature gate, but install LCA the traditional way first in order to find if the feature gate affects LCA seedgen.
The result is seed gen fails due to container error on spoke cluster
[core@helix80 ~]$ sudo podman logs lca_image_builder | grep -C5 -i error
time="2025-12-02 22:47:08" level=info msg="Unauthenticated etcd server for recert is up and running"
time="2025-12-02 22:47:08" level=info msg="Start running recert"
time="2025-12-02 22:47:08" level=info msg="Executing nsenter with args [--target 1 --cgroup --mount --ipc --pid -- podman run --rm --network=host --privileged --replace --name recert -v /etc:/host-etc -v /etc/ssh:/ssh -v /etc/kubernetes:/kubernetes -v /var/lib/kubelet:/kubelet -v /var/tmp:/var/tmp -v /etc/machine-config-daemon:/machine-config-daemon -v /etc/pki:/pki -e RECERT_CONFIG=/var/tmp/backup/recert_config.json --authfile /var/lib/lca/ibu-seedgen-orch/auth.json registry.redhat.io/openshift4/recert-rhel9@sha256:73c87fcc1a20bfec8aef234bb2d12bd69ba24072256548f003f245e2f560618f]"
time="2025-12-02 22:47:10" level=info msg="Killing the unauthenticated etcd server"
time="2025-12-02 22:47:10" level=info msg="Executing nsenter with args [--target 1 --cgroup --mount --ipc --pid -- podman stop recert_etcd]"
time="2025-12-02 22:47:10" level=error msg="failed to create seed image: failed to run once recert: failed to run recert tool container: 2025-12-02 22:47:09 - INFO - src/cluster_crypto/crypto_utils.rs:272: using openssl: OpenSSL 3.5.1 1 Jul 2025 (Library: OpenSSL 3.5.1 1 Jul 2025)\n\n2025-12-02 22:47:09 - INFO - src/cluster_crypto/crypto_utils.rs:300: OpenSSL version is compatible\nError: etcd to utf-8\n\nCaused by:\n invalid utf-8 sequence of 1 bytes from index 0\n: exit status 1"
time="2025-12-02 22:47:10" level=info msg="Cleaning up seed cluster"
time="2025-12-02 22:47:10" level=info msg="Removing seed image"
time="2025-12-02 22:47:10" level=info msg="Executing nsenter with args [--target 1 --cgroup --mount --ipc --pid -- podman rmi registry.kni-qe-84.telcoqe.eng.rdu2.dc.redhat.com:5000/ibu/seed:4.21.0-0.nightly-2025-11-22-193140]"
time="2025-12-02 22:47:10" level=error msg="failed to remove seed image: failed to run \"podman\" in host namespace with args [rmi registry.kni-qe-84.telcoqe.eng.rdu2.dc.redhat.com:5000/ibu/seed:4.21.0-0.nightly-2025-11-22-193140]: Error: registry.kni-qe-84.telcoqe.eng.rdu2.dc.redhat.com:5000/ibu/seed:4.21.0-0.nightly-2025-11-22-193140: image not known\n: exit status 1"
time="2025-12-02 22:47:10" level=info msg="Cleaning up systemd service units"
time="2025-12-02 22:47:10" level=info msg="Disabling service unit installation-configuration.service"
time="2025-12-02 22:47:10" level=info msg="Running systemctl disable [installation-configuration.service --now]"
time="2025-12-02 22:47:10" level=info msg="Executing nsenter with args [--target 1 --cgroup --mount --ipc --pid -- systemctl disable installation-configuration.service --now]"
time="2025-12-02 22:47:11" level=info msg="Removing service unit installation-configuration.service"
--
time="2025-12-02 22:47:11" level=info msg="Removing /etc/cni/multus/certs folder"
time="2025-12-02 22:47:11" level=info msg="Removing /var/seed_data folder"
time="2025-12-02 22:47:11" level=info msg="Restoring cluster services (i.e. kubelet.service unit)"
time="2025-12-02 22:47:11" level=info msg="Running systemctl enable [kubelet.service --now]"
time="2025-12-02 22:47:11" level=info msg="Executing nsenter with args [--target 1 --cgroup --mount --ipc --pid -- systemctl enable kubelet.service --now]"
time="2025-12-02 22:47:13" level=fatal msg="Failed to restore seed cluster: encountered 1 errors during restoration"
Version-Release number of selected component (if applicable):
OCP 4.21.0-0.nightly-2025-11-22-193140
LCA
How reproducible:
Always
Steps to Reproduce:
1. Deploy SNO OCP 4.21 spoke for seed gen via ZTP applying RAN DU profile from RDS with the feature gate enabled with:
installConfigOverrides: |
{
"featureSet": "TechPreviewNoUpgrade"
}
2. Run seed gen job after seed deployment and monitor progress of the seedgen CR
3.
Actual results:
seedgen fails due to container exit 1 status
Expected results:
seedgen completes successfully.
Additional info:
This issue does not happen if the TPNU feature gate is not enabled with deployment config the same otherwise.