if we create a MC to install MCO extensions, if we try to install new packages in the nodes later the installation fails


    

4.18.0-0.test-2024-10-30-172532-ci-ln-n12mphb-latest

We can see this behaviour in 4.15 too (we haven't checked 4.14 or older)

Always
    

    1. Enable centos repos in node 

$ cat ~/copy_centos_repo_2_node.sh 
function copy2node() {
oc cp $2 openshift-machine-config-operator/$(oc get pods -n openshift-machine-config-operator -l "k8s-app=machine-config-daemon" --field-selector "spec.nodeName=$1" -ojsonpath="{.items[0].metadata.name}"):/rootfs/$3; 
}

mkdir -p /tmp/centos-cp/

podman create --name my-centos quay.io/centos/centos:stream9
podman cp my-centos:/etc/yum.repos.d/. /tmp/centos-cp/
podman cp my-centos:/etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial /tmp/centos-cp/
podman cp my-centos:/etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-SIG-Extras-SHA512 /tmp/centos-cp/
podman rm my-centos


copy2node $1 /tmp/centos-cp/centos.repo /etc/yum.repos.d/
copy2node $1 /tmp/centos-cp/RPM-GPG-KEY-centosofficial  /etc/pki/rpm-gpg/
oc debug node/$1 -- chroot /host sh -c "echo '9-stream' > /etc/dnf/vars/stream"



$  ~/copy_centos_repo_2_node.sh ip-10-0-72-151.us-east-2.compute.internal


    2. Install the wget  package in the node

$ oc debug node/ip-10-0-72-151.us-east-2.compute.internal
Starting pod/ip-10-0-72-151us-east-2computeinternal-debug-r74qk ...
To use host binaries, run `chroot /host`
chroPod IP: 10.0.72.151
If you don't see a command prompt, try pressing enter.
sh-5.1# chroot /host
sh-5.1# rpm-ostree install wget
⠁ Checking out tree 3786809...         
....
Added:
  wget-1.21.1-8.el9.x86_64
Changes queued for next boot. Run "systemctl reboot" to start a reboot

sh-5.1# reboot



    3. Install the extensions

$ oc create -f << EOF
apiVersion: machineconfiguration.openshift.io/v1
kind: MachineConfig
metadata:
  labels:
    machineconfiguration.openshift.io/role: worker
  name: tc-56131-all-extensions
spec:
  config:
    ignition:
      version: 3.1.0
  extensions:
  - usbguard
  - kerberos
  - kernel-devel
  - sandboxed-containers
  - wasm
  - ipsec
  - sysstat 
EOF

sh-5.1# rpm-ostree status
State: idle
Deployments:
* ostree-unverified-registry:registry.build09.ci.openshift.org/ci-ln-n12mphb/stable@sha256:b5ba1df86a070575eeaab2309870f3136831b940c00ba5a084f04c664ef494e9
                   Digest: sha256:b5ba1df86a070575eeaab2309870f3136831b940c00ba5a084f04c664ef494e9
                  Version: 418.94.202410300104-0 (2024-10-30T01:08:28Z)
          LayeredPackages: crun-wasm kata-containers kernel-devel kernel-headers krb5-workstation libkadm5 libreswan NetworkManager-libreswan usbguard wget

    4. After the extensions are installed, try to install a new package in the node

sh-5.1# rpm-ostree install zsh
⠁ Checking out tree 3786809...          
....
rpm-md repo 'baseos' (cached); generated: 2024-10-28T14:40:00Z solvables: 4479
rpm-md repo 'appstream' (cached); generated: 2024-10-28T14:42:53Z solvables: 17185
error: Packages not found: crun-wasm, kata-containers


    

After installing the MCO extensions new packages installations fail.
    

We should be able to add packages to the nodes the same way we were able to do it before installing the extensions.