Uploaded image for project: 'Project Quay'
  1. Project Quay
  2. PROJQUAY-7124

Allow Splunk HTTP Event collectors as a Splunk producer

XMLWordPrintable

    • False
    • None
    • False

      Story: As a Quay admin I want to be able to forward Quay action logs to a Splunk HTTP Event Collector so I can use Splunk cloud which does not support Splunks Simple Receiver.

      Acceptance criteria:

      • support for Splunk HEC is available
      • Splunk properties like index, hostname and sourcetype are configurable
      • custom SSL certs and hostnames remain available (as currently in the regular Splunk producer)
      • Splunk HEC token can be configured

      Implementation:

      This adds a new LOGS_MODEL_CONFIG schema called splunk_hec_config

      "splunk_hec_config": {
                          "type": "object",
                          "description": "Logs model config for splunk HTTP event collector action logs configuration",
                          "x-reference": "https://docs.splunk.com/Documentation/SplunkCloud/latest/Data/UsetheHTTPEventCollector#More_information_on_HEC_for_developers",
                          "properties": {
                              "host": {
                                  "type": "string",
                                  "description": "Splunk cluster endpoint",
                                  "x-example": "host.splunk.example",
                              },
                              "port": {
                                  "type": "number",
                                  "description": "Splunk management cluster endpoint port",
                                  "x-example": 8080,
                                  "default": 443,
                              },
                              "hec_token": {
                                  "type": "string",
                                  "description": "HEC token for splunk.",
                                  "x-example": "1ad4d7bb-eed9-443a-897d-29e3b27df7a8",
                              },
                              "url_scheme": {
                                  "type": "string",
                                  "description": "The url scheme for accessing the splunk service. If Splunk is behind SSL"
                                  "*at all*, this *must* be `https`",
                                  "enum": ["http", "https"],
                                  "x-example": "https",
                                  "default": "https",
                              },
                              "verify_ssl": {
                                  "type": "boolean",
                                  "description": "Enable (True) or disable (False) SSL verification for https connections."
                                  "Defaults to True",
                                  "x-example": True,
                                  "default": True,
                              },
                              "ssl_ca_path": {
                                  "type": "string",
                                  "description": "*Relative container path* to a single .pem file containing a CA "
                                  "certificate for SSL verification",
                                  "x-example": "conf/stack/ssl-ca-cert.pem",
                              },
                              "index": {
                                  "type": "string",
                                  "description": "The splunk index to use (overrides the token's default index).",
                                  "x-example": "main",
                              },
                              "splunk_host": {
                                  "type": "string",
                                  "description": "The host name to log this event with (Defaults to the configured server hostname).",
                                  "x-example": "quay.dev",
                                  "default": "configured server hostname",
                              },
                              "splunk_sourcetype": {
                                  "type": "string",
                                  "description": "The name of the Splunk sourcetype to use.",
                                  "x-example": "quay-sourcetype",
                                  "default": "access_combined",
                              },
                          },
                          "required": ["host", "hec_token"],
                      },
      

      For example:

      LOGS_MODEL: "splunk"
      LOGS_MODEL_CONFIG: {
        "producer": "splunk_hec",
        "splunk_hec_config": {
          "host": "prd-p-aaaaaq.splunkcloud.com",
          "port": "8088",
          "hec_token": "12345678-1234-1234-1234-1234567890ab",
          "url_scheme": "https",
          "verify_ssl": False,
          "index": "quay",
          "splunk_host": "quay-dev",
          "splunk_sourcetype": "quay",
        }
      }
      

      Important: This complements and does not replace the existing splunk_config variant which can still be used with Splunk Simple Receivers. For Splunk HTTP Event Collectors, the new config model should be used, which triggers Quay to talk to Splunk via a different API.

              DanielMesser Daniel Messer
              DanielMesser Daniel Messer
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated:
                Resolved: