-
Bug
-
Resolution: Unresolved
-
Normal
-
None
-
4.19.0
-
Quality / Stability / Reliability
-
False
-
-
None
-
Moderate
-
None
-
None
-
None
-
None
-
None
-
None
-
None
-
None
-
None
-
None
-
None
-
None
Description of problem:
When multiple invalid PinnedImageSets are applied and fail, the errors reported in the MachineConfigNode object do not clear until all PISs are corrected.
Version-Release number of selected component (if applicable):
Found in 4.19, but likely in all versions with PIS (both tech preview & GA).
How reproducible:
Always.
Steps to Reproduce:
1. Apply 2 invalid PISs.
$ oc create -f - << EOF
apiVersion: machineconfiguration.openshift.io/v1alpha1
kind: PinnedImageSet
metadata:
name: test-pinned
labels:
machineconfiguration.openshift.io/role: "worker"
spec:
pinnedImages:
- name: quay.io/openshift-release-dev/ocp-release@sha256:65d3a308767b1773b6e3ead2ec1bcae499dde6ef085753d7e20e685f78841079
EOF
$ oc create -f - << EOF
apiVersion: machineconfiguration.openshift.io/v1alpha1
kind: PinnedImageSet
metadata:
name: test-pinned-2
labels:
machineconfiguration.openshift.io/role: "worker"
spec:
pinnedImages:
- name: quay.io/openshift-release-dev/ocp-release@sha256:fd3692eff21338e900a244dfe62152c959b84d73f2dd4503893de0f3fae61b0b
EOF
2. Observe an MCN for a node targeted by the invalid PISs.
$ oc describe machineconfignode/<node-name> … Pinned Image Sets: Desired Generation: 2 Last Failed Generation: 2 Last Failed Generation Errors: failed to execute podman manifest inspect for "quay.io/openshift-release-dev/ocp-release@sha256:fd3692eff21338e900a244dfe62152c959b84d73f2dd4503893de0f3fae61b0b": exit status 125 Name: test-pinned Desired Generation: 1 Last Failed Generation: 1 Last Failed Generation Errors: failed to execute podman manifest inspect for "quay.io/openshift-release-dev/ocp-release@sha256:fd3692eff21338e900a244dfe62152c959b84d73f2dd4503893de0f3fae61b0b": exit status 125 Name: test-pinned-2
3. Correct one of the PISs.
$ oc edit pinnedimageset/test-pinned <change PIS.Spec.PinnedImages.Name to `quay.io/openshift-release-dev/ocp-release@sha256:513cf1028aa1a021fa73d0601427a0fbcf6d212b88aaf9d76d4e4841a061e44e`>
4. See that MCN has not cleared the error for the corrected PIS.
$ oc describe machineconfignode/<node-name> … Pinned Image Sets: Desired Generation: 3 Last Failed Generation: 3 Last Failed Generation Errors: failed to execute podman manifest inspect for "quay.io/openshift-release-dev/ocp-release@sha256:fd3692eff21338e900a244dfe62152c959b84d73f2dd4503893de0f3fae61b0b": exit status 125 Name: test-pinned Desired Generation: 1 Last Failed Generation: 1 Last Failed Generation Errors: failed to execute podman manifest inspect for "quay.io/openshift-release-dev/ocp-release@sha256:fd3692eff21338e900a244dfe62152c959b84d73f2dd4503893de0f3fae61b0b": exit status 125 Name: test-pinned-2
5. Correct the second PIS.
$ oc edit pinnedimageset/test-pinned-2 <change PIS.Spec.PinnedImages.Name to `quay.io/openshift-release-dev/ocp-release@sha256:61eae2d261e54d1b8a0e05f6b5326228b00468364563745eed88460af04f909b`>
6. See that the MCN object is no longer reporting errors for the PISs.
$ oc describe machineconfignode/<node-name> … Pinned Image Sets: Desired Generation: 3 Name: test-pinned Desired Generation: 2 Name: test-pinned-2
Actual results:
When there are multiple invalid PISs applied, the failures in the MCN object are not getting cleared when one invalid PIS is corrected. Rather, the PIS errors are cleared from the MCN object when all invalid PISs are corrected.
Expected results:
PIS errors should clear as they are corrected and should not be dependent on one another.
Additional info:
See relevant conversation here.
