Uploaded image for project: 'OCP Technical Release Team'
  1. OCP Technical Release Team
  2. TRT-1892

Add audit log check for audit-violations

XMLWordPrintable

    • Icon: Story Story
    • Resolution: Done
    • Icon: Normal Normal
    • None
    • None
    • None
    • None
    • False
    • Hide

      None

      Show
      None
    • False
    • None
    • None
    • None
    • None

      Recent PodSecurityViolation failures were hard to pin down where the failure occurred

      The tests showed:

       [bz-apiserver-auth][invariant] alert/PodSecurityViolation should not be at or above info expand_less 	0s
      {  PodSecurityViolation was at or above info for at least 40m36s on platformidentification.JobType{Release:"4.18", FromRelease:"4.18", Platform:"gcp", Architecture:"amd64", Network:"ovn", Topology:"ha"} (maxAllowed=0s): pending for 0s, firing for 40m36s:
      
      Nov 21 22:34:16.134 - 1218s I namespace/openshift-kube-apiserver alert/PodSecurityViolation alertstate/firing severity/info ALERTS{alertname="PodSecurityViolation", alertstate="firing", namespace="openshift-kube-apiserver", ocp_namespace="openshift-catalogd", policy_level="restricted", prometheus="openshift-monitoring/k8s", severity="info"}
      Nov 21 22:34:16.134 - 1218s I namespace/openshift-kube-apiserver alert/PodSecurityViolation alertstate/firing severity/info ALERTS{alertname="PodSecurityViolation", alertstate="firing", namespace="openshift-kube-apiserver", ocp_namespace="openshift-operator-controller", policy_level="restricted", prometheus="openshift-monitoring/k8s", severity="info"}}
      

      loki showed insights-operator

      Documentation indicates we should review the audit logs

      Which contains entries like:

      {
          "kind": "Event",
          "apiVersion": "audit.k8s.io/v1",
          "level": "Metadata",
          "auditID": "6d8f3eda-2ad6-41d4-a158-c0474bc2eba5",
          "stage": "ResponseComplete",
          "requestURI": "/apis/apps/v1/namespaces/openshift-operator-controller/deployments",
          "verb": "create",
          "user": {
              "username": "system:serviceaccount:openshift-cluster-olm-operator:cluster-olm-operator",
              "uid": "ebda9d89-6c84-4e46-8919-c622af7922f3",
              "groups": [
                  "system:serviceaccounts",
                  "system:serviceaccounts:openshift-cluster-olm-operator",
                  "system:authenticated"
              ],
              "extra": {
                  "authentication.kubernetes.io/credential-id": [
                      "JTI=4813932a-d08d-4735-bca1-2a2bea50b6c4"
                  ],
                  "authentication.kubernetes.io/node-name": [
                      "ci-op-hs76pwwz-1e160-l6xvj-master-0"
                  ],
                  "authentication.kubernetes.io/node-uid": [
                      "45dd8e53-80fc-499c-bdad-0cc9b24a06ba"
                  ],
                  "authentication.kubernetes.io/pod-name": [
                      "cluster-olm-operator-5b7c88f447-54phc"
                  ],
                  "authentication.kubernetes.io/pod-uid": [
                      "86600c7f-612f-492b-83b3-498648da225e"
                  ]
              }
          },
          "sourceIPs": [
              "10.0.0.3"
          ],
          "userAgent": "cluster-olm-operator/v0.0.0 (linux/amd64) kubernetes/$Format",
          "objectRef": {
              "resource": "deployments",
              "namespace": "openshift-operator-controller",
              "name": "operator-controller-controller-manager",
              "apiGroup": "apps",
              "apiVersion": "v1"
          },
          "responseStatus": {
              "metadata": {},
              "code": 201
          },
          "requestReceivedTimestamp": "2024-11-21T22:26:32.795959Z",
          "stageTimestamp": "2024-11-21T22:26:33.021455Z",
          "annotations": {
              "authorization.k8s.io/decision": "allow",
              "authorization.k8s.io/reason": "RBAC: allowed by ClusterRoleBinding \"cluster-olm-operator-role\" of ClusterRole \"cluster-olm-operator\" to ServiceAccount \"cluster-olm-operator/openshift-cluster-olm-operator\"",
              "pod-security.kubernetes.io/audit-violations": "would violate PodSecurity \"restricted:latest\": restricted volume types (volume \"etc-containers\" uses restricted volume type \"hostPath\")"
          }
      }
      
      

      Adding auditloganalyzer test to extract more detail on the cause

              rh-ee-fbabcock Forrest Babcock
              rh-ee-fbabcock Forrest Babcock
              None
              None
              None
              None
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated:
                Resolved: