-
Bug
-
Resolution: Not a Bug
-
Critical
-
None
-
4.8
-
None
-
Rejected
-
False
-
Description of problem:
Dynatrace pods in ImagePullBackOff when installing Dynatrace OneAgent.
$ oc get all NAME READY STATUS RESTARTS AGE pod/dynatrace-oneagent-operator-5bcd879579-j2mcp 0/1 ImagePullBackOff 0 15m pod/dynatrace-oneagent-webhook-85dcf747f6-fhrnx 0/2 ImagePullBackOff 0 15m NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/dynatrace-oneagent-webhook ClusterIP 172.30.229.58 <none> 443/TCP 15m NAME READY UP-TO-DATE AVAILABLE AGE deployment.apps/dynatrace-oneagent-operator 0/1 1 0 15m deployment.apps/dynatrace-oneagent-webhook 0/1 1 0 15m NAME DESIRED CURRENT READY AGE replicaset.apps/dynatrace-oneagent-operator-5bcd879579 1 1 0 15m replicaset.apps/dynatrace-oneagent-webhook-85dcf747f6 1 1 0 15m $ oc describe pod dynatrace-oneagent-operator-5bcd879579-j2mcp Name: dynatrace-oneagent-operator-5bcd879579-j2mcp Namespace: openshift-dynatrace Priority: 0 Node: worker-0.lucluster.lab.psi.pnq2.redhat.com/10.74.210.82 Start Time: Thu, 03 Nov 2022 09:54:50 +0100 Labels: dynatrace=operator name=dynatrace-oneagent-operator operator=oneagent pod-template-hash=5bcd879579 Annotations: alm-examples: [ { "apiVersion": "dynatrace.com/v1alpha1", "kind": "OneAgent", "metadata": { "name": "oneagent", "namespace": "dynatrace" }, "spec": { "apiUrl": "https://ENVIRONMENTID.live.dynatrace.com/api", "tolerations": [ { "effect": "NoSchedule", "key": "node-role.kubernetes.io/master", "operator": "Exists" } ], "useUnprivilegedMode": true } }, { "apiVersion": "dynatrace.com/v1alpha1", "kind": "OneAgentAPM", "metadata": { "name": "oneagentapm", "namespace": "dynatrace" }, "spec": { "apiUrl": "https://ENVIRONMENTID.live.dynatrace.com/api" } } ] capabilities: Deep Insights categories: Monitoring,Logging & Tracing containerImage: registry.marketplace.redhat.com/rhm/dynatrace/dynatrace-oneagent-operator@sha256:0c91309827f6b765068ac4f1e9b0e77c188e2aa4f127fb1463c5b0fc5... createdAt: 2021-03-19T15:00:00Z k8s.v1.cni.cncf.io/network-status: [{ "name": "openshift-sdn", "interface": "eth0", "ips": [ "10.128.2.91" ], "default": true, "dns": {} }] k8s.v1.cni.cncf.io/networks-status: [{ "name": "openshift-sdn", "interface": "eth0", "ips": [ "10.128.2.91" ], "default": true, "dns": {} }] marketplace.openshift.io/remote-workflow: https://marketplace.redhat.com/en-us/operators/oneagent-certified-rhmp/pricing?utm_source=openshift_console marketplace.openshift.io/support-workflow: https://marketplace.redhat.com/en-us/operators/oneagent-certified-rhmp/support?utm_source=openshift_console olm.operatorGroup: openshift-dynatrace-cm87j olm.operatorNamespace: openshift-dynatrace olm.targetNamespaces: openshift-dynatrace openshift.io/scc: restricted operatorframework.io/properties: {"properties":[{"type":"olm.gvk","value":{"group":"dynatrace.com","kind":"OneAgent","version":"v1alpha1"}},{"type":"olm.gvk","value":{"gro... operators.operatorframework.io/builder: operator-sdk-v1.5.0 operators.operatorframework.io/project_layout: go.kubebuilder.io/v3 repository: https://github.com/Dynatrace/dynatrace-oneagent-operator support: Dynatrace Status: Pending IP: 10.128.2.91 IPs: IP: 10.128.2.91 Controlled By: ReplicaSet/dynatrace-oneagent-operator-5bcd879579 Containers: dynatrace-oneagent-operator: Container ID: Image: registry.marketplace.redhat.com/rhm/dynatrace/dynatrace-oneagent-operator@sha256:0c91309827f6b765068ac4f1e9b0e77c188e2aa4f127fb1463c5b0fc551826e2 Image ID: Ports: 8080/TCP, 10080/TCP Host Ports: 0/TCP, 0/TCP Args: operator State: Waiting Reason: ImagePullBackOff Ready: False Restart Count: 0 Limits: cpu: 100m memory: 256Mi Requests: cpu: 10m memory: 64Mi Liveness: http-get http://:server-port/healthz delay=15s timeout=1s period=10s #success=1 #failure=3 Readiness: http-get http://:server-port/healthz delay=15s timeout=1s period=10s #success=1 #failure=3 Environment: POD_NAMESPACE: openshift-dynatrace (v1:metadata.namespace) POD_NAME: dynatrace-oneagent-operator-5bcd879579-j2mcp (v1:metadata.name) RELATED_IMAGE_DYNATRACE_ONEAGENT: registry.marketplace.redhat.com/rhm/dynatrace/oneagent@sha256:09c03345575f03325c4000d7eb98e1fc55502d7bb29c6fdb71c8b7e224f1ad7a OPERATOR_CONDITION_NAME: dynatrace-monitoring.v0.10.1 Mounts: /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-4xwj5 (ro) Conditions: Type Status Initialized True Ready False ContainersReady False PodScheduled True Volumes: kube-api-access-4xwj5: Type: Projected (a volume that contains injected data from multiple sources) TokenExpirationSeconds: 3607 ConfigMapName: kube-root-ca.crt ConfigMapOptional: <nil> DownwardAPI: true ConfigMapName: openshift-service-ca.crt ConfigMapOptional: <nil> QoS Class: Burstable Node-Selectors: <none> Tolerations: node.kubernetes.io/memory-pressure:NoSchedule op=Exists node.kubernetes.io/not-ready:NoExecute op=Exists for 300s node.kubernetes.io/unreachable:NoExecute op=Exists for 300s Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 5m42s default-scheduler Successfully assigned openshift-dynatrace/dynatrace-oneagent-operator-5bcd879579-j2mcp to worker-0.lucluster.lab.psi.pnq2.redhat.com Normal AddedInterface 5m40s multus Add eth0 [10.128.2.91/23] from openshift-sdn Normal Pulling 4m4s (x4 over 5m40s) kubelet Pulling image "registry.marketplace.redhat.com/rhm/dynatrace/dynatrace-oneagent-operator@sha256:0c91309827f6b765068ac4f1e9b0e77c188e2aa4f127fb1463c5b0fc551826e2" Warning Failed 4m2s (x4 over 5m38s) kubelet Failed to pull image "registry.marketplace.redhat.com/rhm/dynatrace/dynatrace-oneagent-operator@sha256:0c91309827f6b765068ac4f1e9b0e77c188e2aa4f127fb1463c5b0fc551826e2": rpc error: code = Unknown desc = unable to retrieve auth token: invalid username/password: unauthorized: The login credentials are not valid, or your IBM Cloud account is not active. Warning Failed 4m2s (x4 over 5m38s) kubelet Error: ErrImagePull Warning Failed 3m51s (x6 over 5m37s) kubelet Error: ImagePullBackOff Normal BackOff 38s (x20 over 5m37s) kubelet Back-off pulling image "registry.marketplace.redhat.com/rhm/dynatrace/dynatrace-oneagent-operator@sha256:0c91309827f6b765068ac4f1e9b0e77c188e2aa4f127fb1463c5b0fc551826e2" $ oc describe pod dynatrace-oneagent-webhook-85dcf747f6-fhrnx Name: dynatrace-oneagent-webhook-85dcf747f6-fhrnx Namespace: openshift-dynatrace Priority: 0 Node: worker-0.lucluster.lab.psi.pnq2.redhat.com/10.74.210.82 Start Time: Thu, 03 Nov 2022 09:54:50 +0100 Labels: dynatrace.com/operator=oneagent internal.oneagent.dynatrace.com/app=webhook internal.oneagent.dynatrace.com/component=webhook pod-template-hash=85dcf747f6 Annotations: alm-examples: [ { "apiVersion": "dynatrace.com/v1alpha1", "kind": "OneAgent", "metadata": { "name": "oneagent", "namespace": "dynatrace" }, "spec": { "apiUrl": "https://ENVIRONMENTID.live.dynatrace.com/api", "tolerations": [ { "effect": "NoSchedule", "key": "node-role.kubernetes.io/master", "operator": "Exists" } ], "useUnprivilegedMode": true } }, { "apiVersion": "dynatrace.com/v1alpha1", "kind": "OneAgentAPM", "metadata": { "name": "oneagentapm", "namespace": "dynatrace" }, "spec": { "apiUrl": "https://ENVIRONMENTID.live.dynatrace.com/api" } } ] capabilities: Deep Insights categories: Monitoring,Logging & Tracing containerImage: registry.marketplace.redhat.com/rhm/dynatrace/dynatrace-oneagent-operator@sha256:0c91309827f6b765068ac4f1e9b0e77c188e2aa4f127fb1463c5b0fc5... createdAt: 2021-03-19T15:00:00Z k8s.v1.cni.cncf.io/network-status: [{ "name": "openshift-sdn", "interface": "eth0", "ips": [ "10.128.2.92" ], "default": true, "dns": {} }] k8s.v1.cni.cncf.io/networks-status: [{ "name": "openshift-sdn", "interface": "eth0", "ips": [ "10.128.2.92" ], "default": true, "dns": {} }] marketplace.openshift.io/remote-workflow: https://marketplace.redhat.com/en-us/operators/oneagent-certified-rhmp/pricing?utm_source=openshift_console marketplace.openshift.io/support-workflow: https://marketplace.redhat.com/en-us/operators/oneagent-certified-rhmp/support?utm_source=openshift_console olm.operatorGroup: openshift-dynatrace-cm87j olm.operatorNamespace: openshift-dynatrace olm.targetNamespaces: openshift-dynatrace openshift.io/scc: restricted operatorframework.io/properties: {"properties":[{"type":"olm.gvk","value":{"group":"dynatrace.com","kind":"OneAgent","version":"v1alpha1"}},{"type":"olm.gvk","value":{"gro... operators.operatorframework.io/builder: operator-sdk-v1.5.0 operators.operatorframework.io/project_layout: go.kubebuilder.io/v3 repository: https://github.com/Dynatrace/dynatrace-oneagent-operator support: Dynatrace Status: Pending IP: 10.128.2.92 IPs: IP: 10.128.2.92 Controlled By: ReplicaSet/dynatrace-oneagent-webhook-85dcf747f6 Containers: webhook: Container ID: Image: registry.marketplace.redhat.com/rhm/dynatrace/dynatrace-oneagent-operator@sha256:0c91309827f6b765068ac4f1e9b0e77c188e2aa4f127fb1463c5b0fc551826e2 Image ID: Ports: 8383/TCP, 8443/TCP Host Ports: 0/TCP, 0/TCP Args: webhook-server State: Waiting Reason: ImagePullBackOff Ready: False Restart Count: 0 Limits: cpu: 100m memory: 256Mi Requests: cpu: 10m memory: 64Mi Liveness: http-get https://:server-port/healthz delay=60s timeout=1s period=10s #success=1 #failure=3 Readiness: http-get https://:server-port/healthz delay=60s timeout=1s period=10s #success=1 #failure=3 Environment: POD_NAMESPACE: openshift-dynatrace (v1:metadata.namespace) POD_NAME: dynatrace-oneagent-webhook-85dcf747f6-fhrnx (v1:metadata.name) OPERATOR_CONDITION_NAME: dynatrace-monitoring.v0.10.1 Mounts: /mnt/webhook-certs from certs-volume (rw) /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-7mnlf (ro) bootstrapper: Container ID: Image: registry.marketplace.redhat.com/rhm/dynatrace/dynatrace-oneagent-operator@sha256:0c91309827f6b765068ac4f1e9b0e77c188e2aa4f127fb1463c5b0fc551826e2 Image ID: Ports: 8484/TCP, 9080/TCP Host Ports: 0/TCP, 0/TCP Args: webhook-bootstrapper State: Waiting Reason: ImagePullBackOff Ready: False Restart Count: 0 Limits: cpu: 100m memory: 256Mi Requests: cpu: 10m memory: 64Mi Liveness: http-get http://:server-port/healthz delay=60s timeout=1s period=10s #success=1 #failure=3 Readiness: http-get http://:server-port/healthz delay=60s timeout=1s period=10s #success=1 #failure=3 Environment: POD_NAMESPACE: openshift-dynatrace (v1:metadata.namespace) POD_NAME: dynatrace-oneagent-webhook-85dcf747f6-fhrnx (v1:metadata.name) OPERATOR_CONDITION_NAME: dynatrace-monitoring.v0.10.1 Mounts: /mnt/webhook-certs from certs-volume (rw) /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-7mnlf (ro) Conditions: Type Status Initialized True Ready False ContainersReady False PodScheduled True Volumes: certs-volume: Type: EmptyDir (a temporary directory that shares a pod's lifetime) Medium: SizeLimit: <unset> kube-api-access-7mnlf: Type: Projected (a volume that contains injected data from multiple sources) TokenExpirationSeconds: 3607 ConfigMapName: kube-root-ca.crt ConfigMapOptional: <nil> DownwardAPI: true ConfigMapName: openshift-service-ca.crt ConfigMapOptional: <nil> QoS Class: Burstable Node-Selectors: <none> Tolerations: node.kubernetes.io/memory-pressure:NoSchedule op=Exists node.kubernetes.io/not-ready:NoExecute op=Exists for 300s node.kubernetes.io/unreachable:NoExecute op=Exists for 300s Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 13m default-scheduler Successfully assigned openshift-dynatrace/dynatrace-oneagent-webhook-85dcf747f6-fhrnx to worker-0.lucluster.lab.psi.pnq2.redhat.com Normal AddedInterface 13m multus Add eth0 [10.128.2.92/23] from openshift-sdn Normal Pulling 12m (x3 over 13m) kubelet Pulling image "registry.marketplace.redhat.com/rhm/dynatrace/dynatrace-oneagent-operator@sha256:0c91309827f6b765068ac4f1e9b0e77c188e2aa4f127fb1463c5b0fc551826e2" Warning Failed 12m (x3 over 13m) kubelet Failed to pull image "registry.marketplace.redhat.com/rhm/dynatrace/dynatrace-oneagent-operator@sha256:0c91309827f6b765068ac4f1e9b0e77c188e2aa4f127fb1463c5b0fc551826e2": rpc error: code = Unknown desc = unable to retrieve auth token: invalid username/password: unauthorized: The login credentials are not valid, or your IBM Cloud account is not active. Warning Failed 12m (x3 over 13m) kubelet Error: ErrImagePull Normal BackOff 12m (x5 over 13m) kubelet Back-off pulling image "registry.marketplace.redhat.com/rhm/dynatrace/dynatrace-oneagent-operator@sha256:0c91309827f6b765068ac4f1e9b0e77c188e2aa4f127fb1463c5b0fc551826e2" Warning Failed 12m (x5 over 13m) kubelet Error: ImagePullBackOff Warning Failed 12m (x3 over 13m) kubelet Error: ImagePullBackOff Normal BackOff 3m40s (x41 over 13m) kubelet Back-off pulling image "registry.marketplace.redhat.com/rhm/dynatrace/dynatrace-oneagent-operator@sha256:0c91309827f6b765068ac4f1e9b0e77c188e2aa4f127fb1463c5b0fc551826e2"
Version-Release number of selected component (if applicable):
OpenShift 4.8.48 and Dynatrace OneAgent 0.10.1.
Steps to Reproduce:
1. Install the operator from the Console.
2. Check the pods created by it and find that they are in ImagePullBackOff status.
Actual results:
Image registry.marketplace.redhat.com/rhm/dynatrace/dynatrace-oneagent-operator@sha256:0c91309827f6b765068ac4f1e9b0e77c188e2aa4f127fb1463c5b0fc551826e2 missing.
Expected results:
The image mentioned can be pulled.
