Uploaded image for project: 'Observability and Data Analysis Program'
  1. Observability and Data Analysis Program
  2. OBSDA-866

Improve vector collector regex to set log level of the logs more accurately

XMLWordPrintable

    • False
    • Hide

      None

      Show
      None
    • False
    • Not Selected
    • 0

      Improve vector collector regex to set log level of the logs.

      vector sets the log level to "DEBUG" if anywhere in the log "debug" is present. Actual log level is "INFO". 
      example:

      INFO [l-12-thread-194] c.x.x.impl.ProdLogging                 c.x.x.x.i.ProdLogging.infoLogforPRODDebugging(ProdLogging.java:71) - GET... 

      As per the below vector config spinet, the regex searches for "debug" anywhere in the log and if it sees it, marks it as debug. It checks for debug before it checks for info and because it's a simple "if then else" statement, it stops when it finds a match.:

        if !exists(.level) {
          .level = "default"
          if match!(.message, r'Warning|WARN|^W[0-9]+|level=warn|Value:warn|"level":"warn"|<warn>') {
            .level = "warn"
          } else if match!(.message, r'Error|ERROR|^E[0-9]+|level=error|Value:error|"level":"error"|<error>') {
            .level = "error"
          } else if match!(.message, r'Critical|CRITICAL|^C[0-9]+|level=critical|Value:critical|"level":"critical"|<critical>') {
            .level = "critical"
          } else if match!(.message, r'Debug|DEBUG|^D[0-9]+|level=debug|Value:debug|"level":"debug"|<debug>') {
            .level = "debug"
          } else if match!(.message, r'Notice|NOTICE|^N[0-9]+|level=notice|Value:notice|"level":"notice"|<notice>') {
            .level = "notice"
          } else if match!(.message, r'Alert|ALERT|^A[0-9]+|level=alert|Value:alert|"level":"alert"|<alert>') {
            .level = "alert"
          } else if match!(.message, r'Emergency|EMERGENCY|^EM[0-9]+|level=emergency|Value:emergency|"level":"emergency"|<emergency>') {
            .level = "emergency"
          } else if match!(.message, r'(?i)\b(?:info)\b|^I[0-9]+|level=info|Value:info|"level":"info"|<info>') {
            .level = "info"
              }
        } 

       

       Why does the customer need this? (List the business requirements here)

      => For the requirement to specifically have these log level keywords in any application
      logs. It sets the incorrect log level.
      
      => log forwarder - log level should not based on any matching keyword within a message but an explicit loglevel field. 

      How reproducible:

      Always

      Steps to Reproduce:

      1. Generate an info level log message that contains the word error     
      2. View the logs in the main console UI (not the console pod logs)     
      3. Observe that the log entry is highlighted in red as if it were an error 

      List any affected packages or components:

      Vector/ collector

              jamparke@redhat.com Jamie Parker
              rhn-support-amanverm Aman Dev Verma
              Votes:
              1 Vote for this issue
              Watchers:
              6 Start watching this issue

                Created:
                Updated: