Uploaded image for project: 'OpenShift Logging'
  1. OpenShift Logging
  2. LOG-7338

Fine grained access issue with Lokistack alertingrule

XMLWordPrintable

    • Incidents & Support
    • False
    • Hide

      None

      Show
      None
    • False
    • NEW
    • NEW
    • Bug Fix
    • Hide

      1. Install Red Hat OpenShift Logging Operator v6.3
      2. Install Red Hat Loki Operator v6.3
      3. Create a service account for the logCollector:

      $ oc create sa collector -n openshift-logging 

      4. Bind that serviceAccount to the following roles allowing the collection and writing of logs:

      $ oc adm policy add-cluster-role-to-user collect-application-logs system:serviceaccount:openshift-logging:collector
      $ oc adm policy add-cluster-role-to-user collect-infrastructure-logs system:serviceaccount:openshift-logging:collector
      $ oc adm policy add-cluster-role-to-user cluster-logging-write-application-logs system:serviceaccount:openshift-logging:collector 
      $ oc adm policy add-cluster-role-to-user cluster-logging-write-infrastructure-logs system:serviceaccount:openshift-logging:collector 

      5. Create a secret, LokiStack, UIPlugin, and ClusterlogForwarder Objects:

      apiVersion: v1
      kind: Secret
      metadata:
        name: logging-loki-s3
        namespace: openshift-logging
      stringData:
        access_key_id: <>
        access_key_secret: <>
        bucketnames: lokistoragebucket
        endpoint: https://s3.eu-west-3.amazonaws.com
        region: eu-west-3
      ---
      apiVersion: loki.grafana.com/v1
      kind: LokiStack
      metadata:
        name: logging-loki
        namespace: openshift-logging
      spec:
        managementState: Managed
        rules:
          enabled: true
          namespaceSelector:
            matchLabels:
              openshift.io/log-alerting: 'true'
          selector:
            matchLabels:
              openshift.io/log-alerting: 'true'
        size: 1x.extra-small
        storage:
          schemas:
          - version: v13
            effectiveDate: "2024-03-01"
          secret:
            name: logging-loki-s3
            type: s3
        storageClassName: gp2-csi
        tenants:
          mode: openshift-logging
      ---
      apiVersion: observability.openshift.io/v1alpha1
      kind: UIPlugin
      metadata:
        name: logging
      spec:
        type: Logging
        logging:
          lokiStack:
            name: logging-loki
      ---
      apiVersion: observability.openshift.io/v1
      kind: ClusterLogForwarder
      metadata:
        annotations:
          observability.openshift.io/log-level: debug
        name: instance
        namespace: openshift-logging
      spec:
        serviceAccount:
          name: collector
        outputs:
        - name: default-lokistack
          type: lokiStack
          lokiStack:
            target:
              name: logging-loki
              namespace: openshift-logging
            authentication:
              token:
                from: serviceAccount
          tls:
            ca:
              key: service-ca.crt
              configMapName: openshift-service-ca.crt
        pipelines:
        - name: default-logstore
          inputRefs:
          - application
          - infrastructure
          #- audit
          outputRefs:
          - default-lokistack 
      

      6. Create a user with the name `omega`.
      7. Create a namespace with the name `omegans` with `omega` user.
      8. Add below label to the `omegans` namespace:

      $ oc label namespaces omegans openshift.io/log-alerting=true 

      9. Create below deployment in `omegans` namespace:

      apiVersion: apps/v1
      kind: Deployment
      metadata:
        labels:
          app: testapp-logalerting
          openshift.io/log-alerting: 'true'
        name: omegans-logalerting
        namespace: omegans
      spec:
        replicas: 1
        selector:
          matchLabels:
            app: testapp-logalerting
        template:
          metadata:
            labels:
              app: testapp-logalerting
          spec:
            containers:
            - image: quay.io/rhobs/testapp-logalerting:latest
              imagePullPolicy: IfNotPresent
              name: testapp-logalerting 

      10. Create alertingrule in the `omegans` namespace:

      apiVersion: loki.grafana.com/v1
      kind: AlertingRule
      metadata:
        labels:
          openshift.io/log-alerting: 'true'
        name: always-firing-alert
        namespace: omegans
      spec:
        groups:
          - interval: 1m
            name: AlwaysFiringGroup
            rules:
              - alert: OmegaalwaysFiringTestAlert
                annotations:
                  description: This alert always fires for testing alert pipeline in omegans
                  summary: Always firing test alert
                expr: |
                  count_over_time({kubernetes_namespace_name="omegans"} |= "" [1m]) >= 0
                for: 0s
                labels:
                  severity: critical
        tenantID: application 

      11. Create below RoleBinding, so `omega` user can see logs under Observe => Logs

      kind: RoleBinding
      apiVersion: rbac.authorization.k8s.io/v1
      metadata:
        name: view-application-logs2
        namespace: omegans
      subjects:
        - kind: User
          apiGroup: rbac.authorization.k8s.io
          name: omega
      roleRef:
        apiGroup: rbac.authorization.k8s.io
        kind: ClusterRole
        name: cluster-logging-application-view

      12. Create below RoleBinding, so `omega` user can see Alerts under Observe => Alerts

      kind: RoleBinding
      apiVersion: rbac.authorization.k8s.io/v1
      metadata:
        name: omega-view
        namespace: omegans
      subjects:
        - kind: User
          apiGroup: rbac.authorization.k8s.io
          name: omega
      roleRef:
        apiGroup: rbac.authorization.k8s.io
        kind: ClusterRole
        name: monitoring-rules-view 

      13. Login to the webconsole with the `omega` user.
      14. Navigate to Observe => Logs and check if Logs are visible.
      15. Navigate to Observe => Alerts and check if Alerts are visible.

      Show
      1. Install Red Hat OpenShift Logging Operator v6.3 2. Install Red Hat Loki Operator v6.3 3. Create a service account for the logCollector: $ oc create sa collector -n openshift-logging 4. Bind that serviceAccount to the following roles allowing the collection and writing of logs: $ oc adm policy add-cluster-role-to-user collect-application-logs system:serviceaccount:openshift-logging:collector $ oc adm policy add-cluster-role-to-user collect-infrastructure-logs system:serviceaccount:openshift-logging:collector $ oc adm policy add-cluster-role-to-user cluster-logging-write-application-logs system:serviceaccount:openshift-logging:collector $ oc adm policy add-cluster-role-to-user cluster-logging-write-infrastructure-logs system:serviceaccount:openshift-logging:collector 5. Create a secret, LokiStack, UIPlugin, and ClusterlogForwarder Objects: apiVersion: v1 kind: Secret metadata: name: logging-loki-s3 namespace: openshift-logging stringData: access_key_id: <> access_key_secret: <> bucketnames: lokistoragebucket endpoint: https: //s3.eu-west-3.amazonaws.com region: eu-west-3 --- apiVersion: loki.grafana.com/v1 kind: LokiStack metadata: name: logging-loki namespace: openshift-logging spec: managementState: Managed rules: enabled: true namespaceSelector: matchLabels: openshift.io/log-alerting: ' true ' selector: matchLabels: openshift.io/log-alerting: ' true ' size: 1x.extra-small storage: schemas: - version: v13 effectiveDate: "2024-03-01" secret: name: logging-loki-s3 type: s3 storageClassName: gp2-csi tenants: mode: openshift-logging --- apiVersion: observability.openshift.io/v1alpha1 kind: UIPlugin metadata: name: logging spec: type: Logging logging: lokiStack: name: logging-loki --- apiVersion: observability.openshift.io/v1 kind: ClusterLogForwarder metadata: annotations: observability.openshift.io/log-level: debug name: instance namespace: openshift-logging spec: serviceAccount: name: collector outputs: - name: default -lokistack type: lokiStack lokiStack: target: name: logging-loki namespace: openshift-logging authentication: token: from: serviceAccount tls: ca: key: service-ca.crt configMapName: openshift-service-ca.crt pipelines: - name: default -logstore inputRefs: - application - infrastructure #- audit outputRefs: - default -lokistack 6. Create a user with the name `omega`. 7. Create a namespace with the name `omegans` with `omega` user. 8. Add below label to the `omegans` namespace: $ oc label namespaces omegans openshift.io/log-alerting= true 9. Create below deployment in `omegans` namespace: apiVersion: apps/v1 kind: Deployment metadata: labels: app: testapp-logalerting openshift.io/log-alerting: ' true ' name: omegans-logalerting namespace: omegans spec: replicas: 1 selector: matchLabels: app: testapp-logalerting template: metadata: labels: app: testapp-logalerting spec: containers: - image: quay.io/rhobs/testapp-logalerting:latest imagePullPolicy: IfNotPresent name: testapp-logalerting 10. Create alertingrule in the `omegans` namespace: apiVersion: loki.grafana.com/v1 kind: AlertingRule metadata:   labels:     openshift.io/log-alerting: ' true '   name: always-firing-alert   namespace: omegans spec:   groups:     - interval: 1m       name: AlwaysFiringGroup       rules:         - alert: OmegaalwaysFiringTestAlert           annotations:             description: This alert always fires for testing alert pipeline in omegans             summary: Always firing test alert           expr: |             count_over_time({kubernetes_namespace_name= "omegans" } |= "" [1m]) >= 0           for : 0s           labels:             severity: critical   tenantID: application 11. Create below RoleBinding, so `omega` user can see logs under Observe => Logs kind: RoleBinding apiVersion: rbac.authorization.k8s.io/v1 metadata:   name: view-application-logs2   namespace: omegans subjects:   - kind: User     apiGroup: rbac.authorization.k8s.io     name: omega roleRef:   apiGroup: rbac.authorization.k8s.io   kind: ClusterRole   name: cluster-logging-application-view 12. Create below RoleBinding, so `omega` user can see Alerts under Observe => Alerts kind: RoleBinding apiVersion: rbac.authorization.k8s.io/v1 metadata:   name: omega-view   namespace: omegans subjects:   - kind: User     apiGroup: rbac.authorization.k8s.io     name: omega roleRef:   apiGroup: rbac.authorization.k8s.io   kind: ClusterRole   name: monitoring-rules-view 13. Login to the webconsole with the `omega` user. 14. Navigate to Observe => Logs and check if Logs are visible. 15. Navigate to Observe => Alerts and check if Alerts are visible.

      • Non-admin users are unable to see their own namespace alerts in the Observe> Alerts section under the Developer web console.
      • Getting `No alerts found` error.

      • Non-admin user can see logs in => Observe > Logs section under the Developer web console.

      • In the above screenshots, the username is `omega`, which is a non-admin user.
      • `omega` user having access of `omegans` namespace.
      • Below alert is firing in the `omegans` namespace:

      • But `omega` user can not see this alert in the Developer > Observe > Alerts section.
      • Here is the operator version:
        $ oc get csv -n openshift-loggingNAME                                    DISPLAY                          VERSION   REPLACES                                PHASE
        cluster-logging.v6.3.0                  Red Hat OpenShift Logging        6.3.0     cluster-logging.v6.2.3                  Succeeded
        cluster-observability-operator.v1.2.0   Cluster Observability Operator   1.2.0     cluster-observability-operator.v1.1.1   Succeeded
        loki-operator.v6.3.0                    Loki Operator                    6.3.0     loki-operator.v6.2.3                    Succeeded 
      • Additional information:
        $ oc get rolebindings -A -o json | jq '.items[] | select(.subjects[0].name=="omega")' | jq '[.roleRef.name,.metadata.namespace]'
        
        [
          "admin",
          "omegans"
        ]
        [
          "monitoring-rules-view",
          "omegans"
        ]
        [
          "cluster-logging-application-view",
          "omegans"
        ]
        [
          "user-settings-09f8d780-2880-4265-8aa7-c6159a4c317b-role",
          "openshift-console-user-settings"
        ]
      • The same scenario is working fine in previous logging/lokistack versions.

       

      Current Results:

      • `omega` user not able to see own namespace alerts in the Observe> Alerts section under the Developer web console.

      Expected Results:

      • `omega` user should be able to see their own namespace alerts in the Observe> Alerts section under the Developer web console.

              Unassigned Unassigned
              rhn-support-pripatil Prithviraj Patil
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated:
                Resolved: