-
Bug
-
Resolution: Won't Do
-
Major
-
None
-
4.11.z
-
None
-
Rejected
-
False
-
Description of problem:
Imagestream trigger annonation doesn't work for daemonset
Version-Release number of selected component (if applicable):
Server Version: 4.11.9 Kubernetes Version: v1.24.0+dc5a2fd
How reproducible:
Always
Steps to Reproduce:
1.Create a imagestream to v1 image oc import-image trigger-example:v1 --from quay.io/mwasher/trigger-example:v1 --confirm oc import-image trigger-example:v2 --from quay.io/mwasher/trigger-example:v2 --confirm oc tag trigger-example:v1 trigger-example:latest 2. Apply the DaemonSet and Deployment Configuration $ oc apply -f deployment.yaml $ oc apply -f daemonset.yaml $ sleep 10 3. Retag the `trigger-example:latest` ImageStream oc tag trigger-example:v2 trigger-example:latest 4. The Deployment starts rolling out, but the DaemonSet does not
Actual results:
Pods are not rolled out
Expected results:
The DaemonSet image is updated and Pods rolledout
Additional info:
Confirmed that this issue is seen in `Server Version: 4.11.9`. Reviewing the ControllerManager code, DaemonSet, Deployment, StatefulSet are all part of the update triggers. Debugging the failures in cluster, increase the OpenShiftControllerManager logLevel to TraceAll: ~~~ |⇒ kl -n openshift-controller-manager controller-manager-bw7z5 | grep -ie "DaemonSet" -e "daemonset-example" | less I1128 05:48:32.540957 1 image_trigger_controller.go:332] Error syncing resource daemonsets.apps/trigger-example/daemonset-example: daemonsets.apps "daemonset-example" is forbidden: User "system:serviceaccoun t:openshift-infra:image-trigger-controller" cannot update resource "daemonsets" in API group "apps" in the namespace "trigger-example" I1128 05:48:33.541988 1 image_trigger_controller.go:375] Started syncing resource "daemonsets.apps/trigger-example/daemonset-example" I1128 05:48:33.542080 1 annotations.go:126] *v1.DaemonSet/daemonset-example has triggers: []trigger.ObjectFieldTrigger{trigger.ObjectFieldTrigger{From:trigger.ObjectReference{Kind:"ImageStreamTag", Name:"tri gger-example:latest", Namespace:"", APIVersion:""}, FieldPath:"spec.template.spec.containers[?(@.name==\"daemonset-example\")].image", Paused:false}} I1128 05:48:33.542139 1 annotations.go:164] *v1.DaemonSet/daemonset-example detected change on spec.template.spec.containers[?(@.name=="daemonset-example")].image = quay.io/mwasher/trigger-example@sha256:a5f 9005dc27a5da42d051d3f1724fcd87598f8a888943ad9b4375b7e8a047a62 00000010 09 44 61 65 6d 6f 6e 53 65 74 12 86 18 0a b2 14 |.DaemonSet......| 00000020 0a 11 64 61 65 6d 6f 6e 73 65 74 2d 65 78 61 6d |..daemonset-exam| I1128 05:48:33.542421 1 round_trippers.go:463] PUT https://172.30.0.1:443/apis/apps/v1/namespaces/trigger-example/daemonsets/daemonset-example I1128 05:48:33.544316 1 image_trigger_controller.go:377] Finished syncing resource "daemonsets.apps/trigger-example/daemonset-example" (2.346928ms) ~~~ Reviewing the `system:serviceaccount:openshift-infra:image-trigger-controller` ClusterRoleBindings+ClusterRole for the SA, it looks like the RBAC policies are in place for all the required types but I believe the APIGroup is wrong. The ClusterRole appears to be managed by the OpenShift API and not the Operator, but this looks like it might be in the process of changing.[0] I believe the bug is here: [1] ~~~ rbacv1helpers.NewRule("get", "update").Groups(extensionsGroup).Resources("daemonsets").RuleOrDie(), ~~~ I believe that it should read as follows, adding the appsGroup, otherwise the resulting RBAC doesn't include the right ApiGroups ~~~ rbacv1helpers.NewRule("get", "update").Groups(extensionsGroup, appsGroup).Resources("daemonsets").RuleOrDie(), ~~~ A workaround for fixing the issue is manually adding the permissions to the clusterrole and re-tagging the Image to trigger a rollout. Resources: [0] https://github.com/openshift/openshift-apiserver/blob/master/pkg/bootstrappolicy/controller_policy.go/#L216-L217 [1] https://github.com/openshift/openshift-apiserver/blame/0f9edf3f0e2eca1998ae15fca71e0844aae8641c/pkg/bootstrappolicy/controller_policy.go#L212-L234