After a non-clean reboot, during the boot process and subsequent trigger of crio-wipe, crio was unable to clean the storage:

~~~
Nov 06 04:21:02 XXXXXXXXXXXXXXXXX crio[2987]: time="2024-11-06 04:21:02.567534481Z" level=info msg="Starting CRI-O, version: 1.29.9-2.rhaos4.16.git933bdd2.el9, git: unknown(clean)"
Nov 06 04:21:02 XXXXXXXXXXXXXXXXX crio[2987]: time="2024-11-06 04:21:02.738954676Z" level=info msg="Checking whether cri-o should wipe containers: open /var/run/crio/version: no such file or directory"
Nov 06 04:21:02 XXXXXXXXXXXXXXXXX crio[2987]: time="2024-11-06 04:21:02.739010978Z" level=info msg="File /var/lib/crio/clean.shutdown not found. Wiping storage directory /var/lib/containers/storage because of suspected dirty shutdown"
...
Nov 06 04:21:45 XXXXXXXXXXXXXXXXX crio[2987]: time="2024-11-06 04:21:45.575787172Z" level=fatal msg="failed to remove storage directory: unlinkat /var/lib/containers/storage: directory not empty"
Nov 06 04:21:45 XXXXXXXXXXXXXXXXX systemd[1]: crio-wipe.service: Main process exited, code=exited, status=1/FAILURE
Nov 06 04:21:45 XXXXXXXXXXXXXXXXX systemd[1]: crio-wipe.service: Failed with result 'exit-code'.
Nov 06 04:21:45 XXXXXXXXXXXXXXXXX systemd[1]: Failed to start CRI-O Auto Update Script.
Nov 06 04:21:45 XXXXXXXXXXXXXXXXX systemd[1]: crio-wipe.service: Consumed 18.020s CPU time.
~~~

Openshift 4.16.16
cri-o 1.29.9-2.rhaos4.16.git933bdd2.el9

N/A

    1.
    2.
    3.
    

crio-wipe failed to clear storage

crio-wipe clear the storage

This was related with a situation in which one pod has incorrect permissions on his storage volume and we want to clarify this issue as a possible root cause, also try to understand, if possible, why it happened.