Uploaded image for project: 'Project Quay'
  1. Project Quay
  2. PROJQUAY-10604

Splunk log producer: log_action failed error message loses all context due to dict.update() returning None

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Normal Normal
    • None
    • quay-v3.17.0
    • quay
    • False
    • Hide

      None

      Show
      None
    • False

      Description

      When Splunk(HEC) is unreachable and ALLOW_WITHOUT_STRICT_LOGGING: true is set, the error logging in splunk_logs_model.py line 197 loses all context (kind, account, performer, repository) from the log message, making it impossible to know which action failed.

      Version-Release number

      Quay 3.17

      How reproducible

      Always

      Steps to Reproduce

      1. Configure Quay with LOGS_MODEL: splunk, producer: splunk_hec, and ALLOW_WITHOUT_STRICT_LOGGING: true
      2. Make the Splunk(HEC) endpoint unreachable (e.g. stop Splunk or block port 8088)
      3. Perform any action that generates an audit log (login, create org, push image, etc.)
      4. Check the Quay app pod logs

      Actual results

      The pod log shows:

      gunicorn-web stdout | [ERROR] [data.logs_model.splunk_logs_model] log_action failed
      

      No context is included - you cannot tell which action failed, which user triggered it, or which namespace/repository was involved.

      The root cause is at data/logs_model/splunk_logs_model.py line 197:

      logger.exception("log_action failed", extra=({"exception": lse}).update(log_data))
      

      dict.update() modifies the dict in-place and returns None in Python. So extra=None is passed to logger.exception(), and all the context fields (kind, account, performer, repository, ip) are lost.

      Expected results

      The error log should include the full context of the failed action, e.g.:

      [ERROR] log_action failed | kind=create_repo account=testorg performer=quay repository=testrepo
      

      Fix

      extra = {"exception": lse}
      extra.update(log_data)
      logger.exception("log_action failed", extra=extra)
      

      Additional info

      Introduced by PR #4941 (PROJQUAY-10317).

      Verified on a live Quay 3.17 instance with Splunk HEC at 18.116.40.103:8088 down. Both quay-quay-app pods show the bare log_action failed message with no useful details.

        1. app1.log
          503 kB
        2. app2.log
          1.25 MB

              rh-ee-shossain Shaon Hossain
              szhao@redhat.com Sean Zhao
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated: