-
Bug
-
Resolution: Done-Errata
-
Major
-
None
-
4.15.z
crun won't reparent and reap zombie process because ptm2 buffer is full
Our customer requested pod deletion, but the pod is stuck in terminating. Upon inspection of the node logs, we can see that the container will never shut down, triggering known crio bug `level=warning msg="Stopping container (...) with stop signal timed out. Killing"` https://issues.redhat.com/browse/OCPBUGS-28981. OCPBUGS-28981 just makes the issue more apparent, but is more than likely not related to it. We then look at the crio container, and can see that it's not being shut down. We can see that the thread group leader (`rsyslogd` in this specific case) with PID 1 received a kill -9 and is trying to shut down as hard as it can, but it's stuck in `zap_pid_ns_processes`. The apparent reason is that we see a bash process in `ZO` (zombie) state which belongs to `crun`. However, `crun` never reaps the zombie process as it's stuck sleeping, waiting to be able to write to tty `ptm2`. It cannot do so, because the tty is full. The tty belongs to `crio`.
This is the same issue as https://github.com/cri-o/cri-o/issues/6699, see https://github.com/cri-o/cri-o/issues/6699#issuecomment-1452796427
More in a private comment
This is a clone of https://issues.redhat.com/browse/RHEL-30102 (the crun bug) as I don't know which component should handle this
- is cloned by
-
OCPBUGS-32482 [release-4.14] crun won't reap zombie process because ptm2 buffer is full [CRI-O]
- Closed
- links to
-
RHBA-2024:2865 OpenShift Container Platform 4.15.z bug fix update