Uploaded image for project: 'AMQ Streams'
  1. AMQ Streams
  2. ENTMQST-2317

[DOC OCP] Prometheus server instance doesn't start in OCP 4.5

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Major Major
    • 1.7.0.GA
    • 1.5.0.GA
    • documentation
    • None
    • False
    • False
    • Undefined

      • The Prometheus server instance is not coming up. I am installing in OpenShift 4.5. The Prometheus-operator pod is running.
      • Firstly I had to remove ""securityContext" from prometheus operator file so that operator can be installed first.
              securityContext:
                runAsNonRoot: true
                runAsUser: 65534
        
      • With above change, prometheus operator is up and running. Now I execute following steps to run prometheus server pod.
      [cpandey@cpandey amq-streams-1.5.0]$ oc apply -f strimzi-service-monitor.yaml
      servicemonitor.monitoring.coreos.com/kafka-service-monitor created
      [cpandey@cpandey amq-streams-1.5.0]$ oc apply -f prometheus-rules.yaml
      prometheusrule.monitoring.coreos.com/prometheus-k8s-rules created
      [cpandey@cpandey amq-streams-1.5.0]$ oc apply -f prometheus.yaml
      clusterrole.rbac.authorization.k8s.io/prometheus-server created
      serviceaccount/prometheus-server created
      clusterrolebinding.rbac.authorization.k8s.io/prometheus-server created
      prometheus.monitoring.coreos.com/prometheus created
      
      • In prometheus operator, I see following error:
        E1008 11:35:37.716784       1 reflector.go:178] pkg/mod/k8s.io/client-go@v0.18.8/tools/cache/reflector.go:125: Failed to list *v1.PrometheusRule: prometheusrules.monitoring.coreos.com is forbidden: User "system:serviceaccount:rdsl-amq-streams:prometheus-operator" cannot list resource "prometheusrules" in API group "monitoring.coreos.com" at the cluster scope
        E1008 11:35:38.904304       1 reflector.go:178] pkg/mod/k8s.io/client-go@v0.18.8/tools/cache/reflector.go:125: Failed to list *v1.ServiceMonitor: servicemonitors.monitoring.coreos.com is forbidden: User "system:serviceaccount:rdsl-amq-streams:prometheus-operator" cannot list resource "servicemonitors" in API group "monitoring.coreos.com" at the cluster scope
        E1008 11:35:43.443510       1 reflector.go:178] pkg/mod/k8s.io/client-go@v0.18.8/tools/cache/reflector.go:125: Failed to list *v1.ConfigMap: configmaps is forbidden: User "system:serviceaccount:rdsl-amq-streams:prometheus-operator" cannot list resource "configmaps" in API group "" at the cluster scope
        
      • why v1.PrometheusRule: prometheusrules.monitoring.coreos.com is forbidden: User "system:serviceaccount:rdsl-amq-streams:prometheus-operator" cannot list resource "prometheusrules" in API group "monitoring.coreos.com" at the cluster scope ?
      [cpandey@cpandey amq-streams-1.5.0]$ oc get pod
      NAME                                                   READY   STATUS    RESTARTS   AGE
      amq-streams-cluster-operator-v1.5.3-58d9544874-48t2b   1/1     Running   0          13h
      my-cluster-entity-operator-76c6fdd84f-nns59            3/3     Running   2          13h
      my-cluster-kafka-0                                     2/2     Running   0          13h
      my-cluster-kafka-1                                     2/2     Running   0          13h
      my-cluster-kafka-2                                     2/2     Running   0          13h
      my-cluster-kafka-exporter-7b5cddf8d5-xxxxk             1/1     Running   3          13h
      my-cluster-zookeeper-0                                 1/1     Running   0          13h
      my-cluster-zookeeper-1                                 1/1     Running   0          13h
      my-cluster-zookeeper-2                                 1/1     Running   0          13h
      prometheus-operator-747b66899f-l5llv                   1/1     Running   0          13h
      [cpandey@cpandey amq-streams-1.5.0]$ oc get events
      No resources found in rdsl-amq-streams namespace.
      [cpandey@cpandey amq-streams-1.5.0]$
      

            dlaing@redhat.com Daniel Laing (Inactive)
            rhn-support-cpandey Chandra Shekhar Pandey (Inactive)
            Michal Toth Michal Toth
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: