Uploaded image for project: 'OpenShift Logging'
  1. OpenShift Logging
  2. LOG-7339

Non-admin user fine grained access issue with loki alertingrule

XMLWordPrintable

    • Incidents & Support
    • 0
    • False
    • Hide

      None

      Show
      None
    • False
    • NEW
    • NEW
    • Release Note Not Required
    • Hide

      1. Install Red Hat OpenShift Logging Operator
      2. Install Red Hat Loki Operator
      3. Create a service account for the logCollector:

      $ oc create sa collector -n openshift-logging 

      4. Bind that serviceAccount to the following roles allowing the collection and writing of logs:

      $ oc adm policy add-cluster-role-to-user collect-application-logs system:serviceaccount:openshift-logging:collector
      $ oc adm policy add-cluster-role-to-user collect-infrastructure-logs system:serviceaccount:openshift-logging:collector
      $ oc adm policy add-cluster-role-to-user cluster-logging-write-application-logs system:serviceaccount:openshift-logging:collector 
      $ oc adm policy add-cluster-role-to-user cluster-logging-write-infrastructure-logs system:serviceaccount:openshift-logging:collector 

      5. Create a secret, LokiStack, UIPlugin, and ClusterlogForwarder Objects:

      apiVersion: v1
      kind: Secret
      metadata:
        name: logging-loki-s3
        namespace: openshift-logging
      stringData:
        access_key_id: <>
        access_key_secret: <>
        bucketnames: lokistoragebucket
        endpoint: https://s3.eu-west-3.amazonaws.com
        region: eu-west-3
      ---
      apiVersion: loki.grafana.com/v1
      kind: LokiStack
      metadata:
        name: logging-loki
        namespace: openshift-logging
      spec:
        managementState: Managed
        rules:
          enabled: true
          namespaceSelector:
            matchLabels:
              openshift.io/log-alerting: 'true'
          selector:
            matchLabels:
              openshift.io/log-alerting: 'true'
        size: 1x.extra-small
        storage:
          schemas:
          - version: v13
            effectiveDate: "2024-03-01"
          secret:
            name: logging-loki-s3
            type: s3
        storageClassName: gp2-csi
        tenants:
          mode: openshift-logging
      ---
      apiVersion: observability.openshift.io/v1alpha1
      kind: UIPlugin
      metadata:
        name: logging
      spec:
        type: Logging
        logging:
          lokiStack:
            name: logging-loki
      ---
      apiVersion: observability.openshift.io/v1
      kind: ClusterLogForwarder
      metadata:
        annotations:
          observability.openshift.io/log-level: debug
        name: instance
        namespace: openshift-logging
      spec:
        serviceAccount:
          name: collector
        outputs:
        - name: default-lokistack
          type: lokiStack
          lokiStack:
            target:
              name: logging-loki
              namespace: openshift-logging
            authentication:
              token:
                from: serviceAccount
          tls:
            ca:
              key: service-ca.crt
              configMapName: openshift-service-ca.crt
        pipelines:
        - name: default-logstore
          inputRefs:
          - application
          - infrastructure
          #- audit
          outputRefs:
          - default-lokistack 
      

      6. Create a user with the names `omega` and `titan`
      7. Create a namespace with the name `omegans` with the `omega` user and `ns2` with the user `titan`.
      8. Add below label:

      $ oc label namespaces omegans openshift.io/log-alerting=true 
      $ oc label namespaces ns2 openshift.io/log-alerting=true 

      9. Create below deployment in `omegans` and `ns2` namespace:

      apiVersion: apps/v1
      kind: Deployment
      metadata:
        labels:
          app: testapp-logalerting
          openshift.io/log-alerting: 'true'
        name: omegans-logalerting
        namespace: omegans
      spec:
        replicas: 1
        selector:
          matchLabels:
            app: testapp-logalerting
        template:
          metadata:
            labels:
              app: testapp-logalerting
          spec:
            containers:
            - image: quay.io/rhobs/testapp-logalerting:latest
              imagePullPolicy: IfNotPresent
              name: testapp-logalerting 
      
      =========
      
      apiVersion: apps/v1
      kind: Deployment
      metadata:
        labels:
          app: testapp-logalerting
          openshift.io/log-alerting: 'true'
        name: ns2-logalerting
        namespace: ns2
      spec:
        replicas: 1
        selector:
          matchLabels:
            app: testapp-logalerting
        template:
          metadata:
            labels:
              app: testapp-logalerting
          spec:
            containers:
            - image: quay.io/rhobs/testapp-logalerting:latest
              imagePullPolicy: IfNotPresent
              name: testapp-logalerting

      10. Create alertingrule in the `omegans`  and `ns2` namespace:

      apiVersion: loki.grafana.com/v1
      kind: AlertingRule
      metadata:
        labels:
          openshift.io/log-alerting: 'true'
        name: always-firing-alert
        namespace: omegans
      spec:
        groups:
          - interval: 1m
            name: AlwaysFiringGroup
            rules:
              - alert: OmegaalwaysFiringTestAlert
                annotations:
                  description: This alert always fires for testing alert pipeline in omegans
                  summary: Always firing test alert
                expr: |
                  count_over_time({kubernetes_namespace_name="omegans"} |= "" [1m]) >= 0
                for: 0s
                labels:
                  severity: critical
        tenantID: application 
      
      ============
      
      apiVersion: loki.grafana.com/v1
      kind: AlertingRule
      metadata:
        labels:
          openshift.io/log-alerting: 'true'
        name: titan-always-firing-alert
        namespace: ns2
      spec:
        groups:
          - interval: 1m
            name: AlwaysFiringGroup
            rules:
              - alert: titanalwaysFiringTestAlert
                annotations:
                  description: This alert always fires for testing alert pipeline in omegans
                  summary: Always firing test alert
                expr: |
                  count_over_time({kubernetes_namespace_name="ns2"} |= "" [1m]) >= 0
                for: 0s
                labels:
                  severity: critical
        tenantID: application 

       

      11. Create below RoleBinding for both users, so they can see logs under Observe => Logs

      kind: RoleBinding
      apiVersion: rbac.authorization.k8s.io/v1
      metadata:
        name: view-application-logs2
        namespace: omegans
      subjects:
        - kind: User
          apiGroup: rbac.authorization.k8s.io
          name: omega
      roleRef:
        apiGroup: rbac.authorization.k8s.io
        kind: ClusterRole
        name: cluster-logging-application-view
      
      =========
      
      kind: RoleBinding
      apiVersion: rbac.authorization.k8s.io/v1
      metadata:
        name: view-logs
        namespace: ns2
      subjects:
        - kind: User
          apiGroup: rbac.authorization.k8s.io
          name: titan
      roleRef:
        apiGroup: rbac.authorization.k8s.io
        kind: ClusterRole
        name: cluster-logging-application-view

       

      12. Create below RoleBinding for both users, so they can see Alerts under Observe => Alerts

      kind: RoleBinding
      apiVersion: rbac.authorization.k8s.io/v1
      metadata:
        name: omega-view
        namespace: omegans
      subjects:
        - kind: User
          apiGroup: rbac.authorization.k8s.io
          name: omega
      roleRef:
        apiGroup: rbac.authorization.k8s.io
        kind: ClusterRole
        name: monitoring-rules-view 
      
      ========
      
      kind: RoleBinding
      apiVersion: rbac.authorization.k8s.io/v1
      metadata:
        name: titan-view
        namespace: ns2
      subjects:
        - kind: User
          apiGroup: rbac.authorization.k8s.io
          name: titan
      roleRef:
        apiGroup: rbac.authorization.k8s.io
        kind: ClusterRole
        name: monitoring-rules-view 

      13. Login to the webconsole with the `omega` user.
      14. Navigate to Observe => Alerts (omega user can see alerts from other namespaces)

      15. Login to the webconsole with the `titan` user.
      16. Navigate to Observe => Alerts (titan user can see alerts from other namespaces)

      Expectation:

      • omega user should only see Alerts from omegans namespace.
      • titan user should only see Alerts from ns2 namespace.
      Show
      1. Install Red Hat OpenShift Logging Operator 2. Install Red Hat Loki Operator 3. Create a service account for the logCollector: $ oc create sa collector -n openshift-logging 4. Bind that serviceAccount to the following roles allowing the collection and writing of logs: $ oc adm policy add-cluster-role-to-user collect-application-logs system:serviceaccount:openshift-logging:collector $ oc adm policy add-cluster-role-to-user collect-infrastructure-logs system:serviceaccount:openshift-logging:collector $ oc adm policy add-cluster-role-to-user cluster-logging-write-application-logs system:serviceaccount:openshift-logging:collector $ oc adm policy add-cluster-role-to-user cluster-logging-write-infrastructure-logs system:serviceaccount:openshift-logging:collector 5. Create a secret, LokiStack, UIPlugin, and ClusterlogForwarder Objects: apiVersion: v1 kind: Secret metadata: name: logging-loki-s3 namespace: openshift-logging stringData: access_key_id: <> access_key_secret: <> bucketnames: lokistoragebucket endpoint: https: //s3.eu-west-3.amazonaws.com region: eu-west-3 --- apiVersion: loki.grafana.com/v1 kind: LokiStack metadata: name: logging-loki namespace: openshift-logging spec: managementState: Managed rules: enabled: true namespaceSelector: matchLabels: openshift.io/log-alerting: ' true ' selector: matchLabels: openshift.io/log-alerting: ' true ' size: 1x.extra-small storage: schemas: - version: v13 effectiveDate: "2024-03-01" secret: name: logging-loki-s3 type: s3 storageClassName: gp2-csi tenants: mode: openshift-logging --- apiVersion: observability.openshift.io/v1alpha1 kind: UIPlugin metadata: name: logging spec: type: Logging logging: lokiStack: name: logging-loki --- apiVersion: observability.openshift.io/v1 kind: ClusterLogForwarder metadata: annotations: observability.openshift.io/log-level: debug name: instance namespace: openshift-logging spec: serviceAccount: name: collector outputs: - name: default -lokistack type: lokiStack lokiStack: target: name: logging-loki namespace: openshift-logging authentication: token: from: serviceAccount tls: ca: key: service-ca.crt configMapName: openshift-service-ca.crt pipelines: - name: default -logstore inputRefs: - application - infrastructure #- audit outputRefs: - default -lokistack 6. Create a user with the names `omega` and `titan` 7. Create a namespace with the name `omegans` with the `omega` user and `ns2` with the user `titan`. 8. Add below label: $ oc label namespaces omegans openshift.io/log-alerting= true $ oc label namespaces ns2 openshift.io/log-alerting= true 9. Create below deployment in `omegans` and `ns2` namespace: apiVersion: apps/v1 kind: Deployment metadata: labels: app: testapp-logalerting openshift.io/log-alerting: ' true ' name: omegans-logalerting namespace: omegans spec: replicas: 1 selector: matchLabels: app: testapp-logalerting template: metadata: labels: app: testapp-logalerting spec: containers: - image: quay.io/rhobs/testapp-logalerting:latest imagePullPolicy: IfNotPresent name: testapp-logalerting ========= apiVersion: apps/v1 kind: Deployment metadata: labels: app: testapp-logalerting openshift.io/log-alerting: ' true ' name: ns2-logalerting namespace: ns2 spec: replicas: 1 selector: matchLabels: app: testapp-logalerting template: metadata: labels: app: testapp-logalerting spec: containers: - image: quay.io/rhobs/testapp-logalerting:latest imagePullPolicy: IfNotPresent name: testapp-logalerting 10. Create alertingrule in the `omegans`  and `ns2` namespace: apiVersion: loki.grafana.com/v1 kind: AlertingRule metadata:   labels:     openshift.io/log-alerting: ' true '   name: always-firing-alert   namespace: omegans spec:   groups:     - interval: 1m       name: AlwaysFiringGroup       rules:         - alert: OmegaalwaysFiringTestAlert           annotations:             description: This alert always fires for testing alert pipeline in omegans             summary: Always firing test alert           expr: |             count_over_time({kubernetes_namespace_name= "omegans" } |= "" [1m]) >= 0           for : 0s           labels:             severity: critical   tenantID: application ============ apiVersion: loki.grafana.com/v1 kind: AlertingRule metadata:   labels:     openshift.io/log-alerting: ' true '   name: titan-always-firing-alert   namespace: ns2 spec:   groups:     - interval: 1m       name: AlwaysFiringGroup       rules:         - alert: titanalwaysFiringTestAlert           annotations:             description: This alert always fires for testing alert pipeline in omegans             summary: Always firing test alert           expr: |             count_over_time({kubernetes_namespace_name= "ns2" } |= "" [1m]) >= 0           for : 0s           labels:             severity: critical   tenantID: application   11. Create below RoleBinding for both users, so they can see logs under Observe => Logs kind: RoleBinding apiVersion: rbac.authorization.k8s.io/v1 metadata:   name: view-application-logs2   namespace: omegans subjects:   - kind: User     apiGroup: rbac.authorization.k8s.io     name: omega roleRef:   apiGroup: rbac.authorization.k8s.io   kind: ClusterRole   name: cluster-logging-application-view ========= kind: RoleBinding apiVersion: rbac.authorization.k8s.io/v1 metadata:   name: view-logs   namespace: ns2 subjects:   - kind: User     apiGroup: rbac.authorization.k8s.io     name: titan roleRef:   apiGroup: rbac.authorization.k8s.io   kind: ClusterRole   name: cluster-logging-application-view   12. Create below RoleBinding for both users, so they can see Alerts under Observe => Alerts kind: RoleBinding apiVersion: rbac.authorization.k8s.io/v1 metadata:   name: omega-view   namespace: omegans subjects:   - kind: User     apiGroup: rbac.authorization.k8s.io     name: omega roleRef:   apiGroup: rbac.authorization.k8s.io   kind: ClusterRole   name: monitoring-rules-view ======== kind: RoleBinding apiVersion: rbac.authorization.k8s.io/v1 metadata:   name: titan-view   namespace: ns2 subjects:   - kind: User     apiGroup: rbac.authorization.k8s.io     name: titan roleRef:   apiGroup: rbac.authorization.k8s.io   kind: ClusterRole   name: monitoring-rules-view 13. Login to the webconsole with the `omega` user. 14. Navigate to Observe => Alerts (omega user can see alerts from other namespaces) 15. Login to the webconsole with the `titan` user. 16. Navigate to Observe => Alerts (titan user can see alerts from other namespaces) Expectation: omega user should only see Alerts from omegans namespace. titan user should only see Alerts from ns2 namespace.
    • Log Storage - Sprint 273
    • Important

      • Non-admin users can see Alerts from different namespaces in the Observe> Alerts section under the Developer web console.

      Current Results:

      • Non-admin users can see Alerts from different namespaces in the Observe> Alerts section under the Developer web console.

      Expected Results:

      • Non-admin users should only see Alerts from their own namespaces in the Observe> Alerts section under the Developer web console.
      • Operator version:
        $ oc get csv
        
        NAME                                    DISPLAY                          VERSION   REPLACES                                PHASE
        cluster-logging.v6.2.3                  Red Hat OpenShift Logging        6.2.3     cluster-logging.v6.2.2                  Succeeded
        cluster-observability-operator.v1.2.0   Cluster Observability Operator   1.2.0     cluster-observability-operator.v1.1.1   Succeeded
        loki-operator.v6.2.3                    Loki Operator                    6.2.3     loki-operator.v6.2.2                    Succeeded
         
      • Non-admin usernames: "titan" and "omega"
      • Non-admin user "titan" is able to view alerts from other namespaces, including those to which he does not have access:

      • Non-admin user "omega" is able to view alerts from other namespaces, including those to which he does not have access:

      • Alertingrules:
        $ oc get alertingrule -n omegans
        
        NAME                  AGE
        always-firing-alert   4h26m
        -------
        
         $ oc get alertingrule -n ns2
        
        NAME                        AGE
        titan-always-firing-alert   15m
      • Additional Information:
        $ oc get rolebindings -A -o json | jq '.items[] | select(.subjects[0].name=="omega")' | jq '[.roleRef.name,.metadata.namespace]'
        
        [
          "admin",
          "omegans"
        ]
        [
          "monitoring-rules-view",
          "omegans"
        ]
        [
          "cluster-logging-application-view",
          "omegans"
        ]
        [
          "user-settings-09f8d780-2880-4265-8aa7-c6159a4c317b-role",
          "openshift-console-user-settings"
        ]
         
        
        ========
        
        $ oc get rolebindings -A -o json | jq '.items[] | select(.subjects[0].name=="titan")' | jq '[.roleRef.name,.metadata.namespace]'
        
        [
          "admin",
          "ns2"
        ]
        [
          "monitoring-rules-view",
          "ns2"
        ]
        [
          "cluster-logging-application-view",
          "ns2"
        ]
        [
          "user-settings-a4ff45d6-7975-4a03-9707-a98f0dba67a2-role",
          "openshift-console-user-settings"
        ]
      • Screenshot of the firing alerts from Kube:Admin:

       

       

       

              rojacob@redhat.com Robert Jacob
              rhn-support-pripatil Prithviraj Patil
              Votes:
              1 Vote for this issue
              Watchers:
              5 Start watching this issue

                Created:
                Updated:
                Resolved: