-
Bug
-
Resolution: Not a Bug
-
Minor
-
None
-
4.13.z
-
Quality / Stability / Reliability
-
False
-
-
None
-
None
-
No
-
None
-
None
-
None
-
None
-
Customer Facing
-
Red Hat OpenShift Container Platform
-
None
-
None
-
None
-
None
-
None
-
None
-
None
Description of problem:
A customer reported that when a node on ESXi is rebooted with an ESXi machine, the images that existed before the reboot are lost after the reboot. The specific procedure is to restart ESXi and check with podman images after the node is booted. I would like to ask if this is the expected cleaning up of the developer.
Version-Release number of selected component (if applicable):
Customer's environment is 4.11 on vSphere. My environment is 4.13 on AWS and I reproduced the following steps
How reproducible:
I did not reproduce this by simply stopping the instance in my AWS verification environment. However, I was able to reproduce it by running echo c > /proc/sysrq-trigger to intentionally cause kernel panic and then reboot.
Steps to Reproduce:
1.Deploy an appropriate pod to a node, and then reduce the number of replicas or otherwise make the pod unscheduled. In this test, httpd was deployed. 2.Check the images using podman images. Obtain other ls results as well. #podman images --all REPOSITORY TAG IMAGE ID CREATED SIZE image-registry.openshift-image-registry.svc:5000/openshift/httpd <none> 4c2e3e6ed803 3 days ago 454 MB quay.io/openshift-release-dev/ocp-v4.0-art-dev <none> 781a37f1fe2f 2 weeks ago 453 MB quay.io/openshift-release-dev/ocp-v4.0-art-dev <none> 54541c943861 2 weeks ago 455 MB quay.io/openshift-release-dev/ocp-v4.0-art-dev <none> 3976fb9d3385 2 weeks ago 549 MB quay.io/openshift-release-dev/ocp-v4.0-art-dev <none> deda63aac062 3 weeks ago 622 MB quay.io/openshift-release-dev/ocp-v4.0-art-dev <none> 06c3508474ed 3 weeks ago 806 MB quay.io/openshift-release-dev/ocp-v4.0-art-dev <none> 2e93e1fa02de 3 weeks ago 365 MB quay.io/openshift-release-dev/ocp-v4.0-art-dev <none> 2bdce6613e65 3 weeks ago 508 MB quay.io/openshift-release-dev/ocp-v4.0-art-dev <none> 5fd99b169b27 3 weeks ago 767 MB quay.io/openshift-release-dev/ocp-v4.0-art-dev <none> 530aaab5f11b 3 weeks ago 916 MB quay.io/openshift-release-dev/ocp-v4.0-art-dev <none> e7d9a027abca 3 weeks ago 456 MB quay.io/openshift-release-dev/ocp-v4.0-art-dev <none> f17170cae8db 3 weeks ago 454 MB quay.io/openshift-release-dev/ocp-v4.0-art-dev <none> 65df008b154e 3 weeks ago 484 MB quay.io/openshift-release-dev/ocp-v4.0-art-dev <none> 43f894469415 3 weeks ago 384 MB quay.io/openshift-release-dev/ocp-v4.0-art-dev <none> c76f55db8fbb 3 weeks ago 347 MB quay.io/openshift-release-dev/ocp-v4.0-art-dev <none> 8d66687d9781 3 weeks ago 404 MB quay.io/openshift-release-dev/ocp-v4.0-art-dev <none> 11e33901d661 3 weeks ago 374 MB quay.io/openshift-release-dev/ocp-v4.0-art-dev <none> c5c53d84e7fe 3 weeks ago 454 MB quay.io/openshift-release-dev/ocp-v4.0-art-dev <none> 1e91c9731ae4 3 weeks ago 417 MB quay.io/openshift-release-dev/ocp-v4.0-art-dev <none> 5c5a5dd41dc1 3 weeks ago 401 MB quay.io/openshift-release-dev/ocp-v4.0-art-dev <none> 06a2691e725c 3 weeks ago 446 MB quay.io/openshift-release-dev/ocp-v4.0-art-dev <none> d7841824be9b 3 weeks ago 380 MB quay.io/openshift-release-dev/ocp-v4.0-art-dev <none> d298c30a25d0 3 weeks ago 366 MB quay.io/openshift-release-dev/ocp-v4.0-art-dev <none> 838ebd6278bd 3 weeks ago 431 MB quay.io/openshift-release-dev/ocp-v4.0-art-dev <none> cb9e81fdf506 3 weeks ago 437 MB quay.io/openshift-release-dev/ocp-v4.0-art-dev <none> 2a2a90c629d8 3 weeks ago 342 MB quay.io/openshift-release-dev/ocp-v4.0-art-dev <none> 9dede6d5713e 3 weeks ago 617 MB quay.io/openshift-release-dev/ocp-v4.0-art-dev <none> 39f285a6e873 3 weeks ago 415 MB quay.io/openshift-release-dev/ocp-v4.0-art-dev <none> 599e0f50ede7 3 weeks ago 590 MB quay.io/openshift-release-dev/ocp-v4.0-art-dev <none> 7edeff8495a6 3 weeks ago 382 MB quay.io/openshift-release-dev/ocp-v4.0-art-dev <none> 3657b7793308 3 weeks ago 368 MB quay.io/openshift-release-dev/ocp-v4.0-art-dev <none> f5189c73bf7e 3 weeks ago 331 MB 3. Stopping after letting os panic. #echo c > /proc/sysrq-triggerStop The node instance in the AWS console after executing the above. 4. Booting up the node Confirm that the node has become NotReady and is also stopped in the AWS console article, after that start it up.5. Confirm information as in step 2. #podman images --all REPOSITORY TAG IMAGE ID CREATED SIZE quay.io/openshift-release-dev/ocp-v4.0-art-dev <none> deda63aac062 3 weeks ago 622 MB quay.io/openshift-release-dev/ocp-v4.0-art-dev <none> 06c3508474ed 3 weeks ago 806 MB quay.io/openshift-release-dev/ocp-v4.0-art-dev <none> 2e93e1fa02de 3 weeks ago 365 MB quay.io/openshift-release-dev/ocp-v4.0-art-dev <none> 2bdce6613e65 3 weeks ago 508 MB quay.io/openshift-release-dev/ocp-v4.0-art-dev <none> 5fd99b169b27 3 weeks ago 767 MB quay.io/openshift-release-dev/ocp-v4.0-art-dev <none> 530aaab5f11b 3 weeks ago 916 MB quay.io/openshift-release-dev/ocp-v4.0-art-dev <none> e7d9a027abca 3 weeks ago 456 MB quay.io/openshift-release-dev/ocp-v4.0-art-dev <none> f17170cae8db 3 weeks ago 454 MB quay.io/openshift-release-dev/ocp-v4.0-art-dev <none> c76f55db8fbb 3 weeks ago 347 MB quay.io/openshift-release-dev/ocp-v4.0-art-dev <none> 1e91c9731ae4 3 weeks ago 417 MB quay.io/openshift-release-dev/ocp-v4.0-art-dev <none> 06a2691e725c 3 weeks ago 446 MB quay.io/openshift-release-dev/ocp-v4.0-art-dev <none> d7841824be9b 3 weeks ago 380 MB quay.io/openshift-release-dev/ocp-v4.0-art-dev <none> d298c30a25d0 3 weeks ago 366 MB quay.io/openshift-release-dev/ocp-v4.0-art-dev <none> 9dede6d5713e 3 weeks ago 617 MB quay.io/openshift-release-dev/ocp-v4.0-art-dev <none> 599e0f50ede7 3 weeks ago 590 MB quay.io/openshift-release-dev/ocp-v4.0-art-dev <none> 7edeff8495a6 3 weeks ago 382 MB quay.io/openshift-release-dev/ocp-v4.0-art-dev <none> 3657b7793308 3 weeks ago 368 MB quay.io/openshift-release-dev/ocp-v4.0-art-dev <none> f5189c73bf7e 3 weeks ago 331 MB
Actual results:
I attach on test1.tar including results. The following output is recorded before and after the reboot. # podman images --all # ls -lR /var/lib/containers/ --ignore="overlay"
Expected results:
I would like to know if this is the expected behavior, as it appears to be a non-gracefull shutdown that cleans up all images. If it is intended, it should output some logs meaning clean up.
Additional info:
- relates to
-
RFE-7117 [RFE] Share images between OCP nodes for network bandwidth reduction and installation/upgrade/node creation speedup
-
- Waiting
-
- links to