-
Bug
-
Resolution: Unresolved
-
Normal
-
None
-
rhel-8.6.0
-
None
-
No
-
Important
-
rhel-sst-container-tools
-
3
-
False
-
-
None
-
Red Hat Enterprise Linux
-
None
-
None
-
None
-
None
What were you trying to do that didn't work?
Podman seems to leak a lot of stuff on the filesystem, and Customer can see a lot of containers left over when they do `buildah containers`. However `podman system prune --external` doesn't work either.
`podman system reset` would indeed clear out the storage, but Cu would have to bring the system to a grinding halt which is not possible everytime and affect any podman builds in progress.
github issue - https://github.com/containers/podman/issues/23683
What is the impact of this issue to you?
Causing more downtime, and eating up more disk space.
Steps to reproduce
- Dockerfile:
FROM ubi8/ubi
RUN truncate -s 10G out
RUN sleep infinity - Script run.sh
#!/usr/bin/env bash
podman build -f Dockerfile -t podmanleaker &
sleep 10 && kill -9 $! - Repeatedly running `run.sh` will increase the size of this folder and `podman system prune --external` doesn't help here.
- $ podman --version
podman version 4.2.0
$ podman system df
TYPE TOTAL ACTIVE SIZE RECLAIMABLE
Images 10 1 5.349GB 5.247GB (98%)
Containers 0 0 0B 0B (0%)
Local Volumes 0 0 0B 0B (0%)
$ podman unshare du -sh /tmp/containers/rootless_user/
1.1G /tmp/containers/rootless_user/
$ ./run.sh &>/dev/null
$ podman system df
TYPE TOTAL ACTIVE SIZE RECLAIMABLE
Images 10 1 5.349GB 5.247GB (98%)
Containers 0 0 0B 0B (0%)
Local Volumes 0 0 0B 0B (0%)
$ podman unshare du -sh /tmp/containers/rootless_user/overlay
1.2G /tmp/containers/rootless_user/
Expected results
Delete these leaked layers, without nuking everything.
Actual results
Space is not released even after `podman system prune --external`
$ podman system df
TYPE TOTAL ACTIVE SIZE RECLAIMABLE
Images 5 3 13.57GB 3.845GB (28%)
Containers 0 0 0B 0B (0%)
Local Volumes 0 0 0B 0B (0%)
$ podman unshare du -sh /tmp/containers/rootless_user
82G /tmp/containers/rootless_user
$ podman system prune --external
$ podman unshare du -sh /tmp/containers/rootless_user
82G /tmp/containers/rootless_user
- buildah containers does display a bunch of things (even after the prune):
$ buildah --version
buildah version 1.29.1 (image-spec 1.0.2-dev, runtime-spec 1.0.2-dev)
$ buildah containers | wc -l
23