-
Bug
-
Resolution: Won't Do
-
Normal
-
None
-
4.16, 4.17
-
Low
-
None
-
Rejected
-
False
-
Description of problem:
When set imagepruner with "keepTagRevisions":0,"keepYoungerThanDuration":"0s" , the pruner pods will fail after 5 reties with error "error deleting repository openshift/** layer link sha256:8694db102e5bd27fa30106f87d5a0f0c5ccccac0e5cc38ba56080d7559377096 from the registry: 500 Internal Server Error error deleting repository openshift/** layer link sha256:7027f4e4058bde8aaa497e47562e962c293039ba16f5fbfd07ff43a0d1dbd5a2 from the registry: 500 Internal Server Error" That will cause image registry to Degraded finially.
Version-Release number of selected component (if applicable):
4.17.0-0.nightly-2024-08-01-213905
How reproducible:
always
Steps to Reproduce:
1. Clean up cluster images by setting
$oc patch imagepruner/cluster -p '{"spec":{"keepTagRevisions":0,"keepYoungerThanDuration":"0s","schedule": "* * * * *"}}' --type=merge
2. for i in {1..7}; do oc new-project test$i ; oc new-app httpd~https://github.com/openshift/httpd-ex.git -n test$i ; sleep 30 ; oc delete istag httpd-ex:latest ; done
3. Check the image pruner pod
Actual results:
'ImagePrunerDegraded: Job has reached the specified backoff limit'
Expected results:
Should clean up the metadata of all unused images
Additional info:
The must-gather log
https://drive.google.com/drive/folders/1DY1dkvWOCNBV8cRnjRaWLa7KPTxgVR7t?usp=drive_link
This only happens on 4.17 recently. and I found some error in apiserver pods 'E0806 02:48:05.974562 1 strategy.go:60] unable to parse manifest for "sha256:****": unexpected end of JSON input' , but these similar errors existing in 4.16 apiserver pods when same setting imagepruner passed
UPDATE: this also happens on 4.16.6 https://issues.redhat.com/browse/OCPBUGS-38020?focusedId=25291852&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-25291852
