Uploaded image for project: 'OpenShift Logging'
  1. OpenShift Logging
  2. LOG-2923

Audit logs missing after upgrade to Logging 5.4

XMLWordPrintable

    • False
    • None
    • False
    • NEW
    • VERIFIED
    • Before this update, a refactoring of the Fluentd collector plugins removed the timestamp field for events. This update restores the timestamp field, sourced from the event's received time.
    • Log Collection - Sprint 223, Log Collection - Sprint 224

      Description of problem:

      When upgrading from OpenShift Logging 5.3 to OpenShift Logging 5.4.4, audit logs are no longer visible in Kibana. This behaviour was reported by a customer in a Support Case and I was able to reproduce the issue internally.

      Some audit logs still seem to be shipped ("SERVICE_START" and "SERVICE_STOP" audit entries) but no other audit logs are displayed despite audit logging being set up correctly.

      Version-Release number of selected component (if applicable):

      • OpenShift Container Platform 4.10.26
      • OpenShift Logging 5.4.4 upgraded from OpenShift Logging 5.3.10

      How reproducible:

      Always

      Steps to Reproduce:
      1. Install OpenShift Logging 5.3 on an OpenShift Container Platform cluster.
      2. Create a ClusterLogging as described in the documentation. Configure audit logs to be shipped by creating the following ClusterLogForwarder:

      apiVersion: logging.openshift.io/v1
      kind: ClusterLogForwarder
      metadata:
        name: instance
        namespace: openshift-logging
      spec:
        pipelines:
        - inputRefs:
          - infrastructure
          - application
          - audit
          name: enable-default-log-store
          outputRefs:
          - default

      3. Go to Kibana, create a "audit*" index and observe that there are hundreds of audit log entries being shipped constantly
      4. Upgrade to OpenShift Logging 5.4.4 ("stable-5.4")

      Actual results:

      Go to Kibana and observe that there are only a few audit log entries being shown

      Expected results:

      Go to Kibana and observe that all audit log entries are shown

      Additional info:

      • Will append additional "oc adm inspect" output for both versions to this Bug

        1. inspect-both-logging-versions.tar.bz2
          658 kB
          Simon Krenger
        2. kibana-audit-logs.png
          290 kB
          Simon Krenger
        3. must-gather.local.4391419834030085349-logging-53.tar.bz2
          5.80 MB
          Simon Krenger
        4. must-gather.local.62588704368112526-logging-54.tar.bz2
          6.52 MB
          Simon Krenger
        5. noname
          3 kB
          Alan Conway

              jcantril@redhat.com Jeffrey Cantrill
              rhn-support-skrenger Simon Krenger
              Ishwar Kanse Ishwar Kanse
              Votes:
              3 Vote for this issue
              Watchers:
              10 Start watching this issue

                Created:
                Updated:
                Resolved: