Uploaded image for project: 'OpenShift Logging'
  1. OpenShift Logging
  2. LOG-3965

Collector to act as http server - preview

    XMLWordPrintable

Details

    • http-server-preview
    • False
    • None
    • False
    • Green
    • NEW
    • To Do
    • NEW
    • 100
    • 100% 100%
    • Hide
      Preview: A new field `ClustserLogForwarder.input.receiver.http` allows the log forwarder to receive logs as a HTTP server.
      The initial preview supports receiving audit logs from a kube-apiserver web-hook. Future releases will add support for other HTTP log sources.
      Show
      Preview: A new field `ClustserLogForwarder.input.receiver.http` allows the log forwarder to receive logs as a HTTP server. The initial preview supports receiving audit logs from a kube-apiserver web-hook. Future releases will add support for other HTTP log sources.
    • Feature
    • Proposed
    • Administer, API, Instructions

    Description

      Goals

      This epic has been restricted to cover the preview version of this feature, minimal features to act as webhook to a kube-apiserver. Completing the feature to be configurable for other use cases is in a separate epic LOG-4562.

      Collector can be configured to listen for HTTP connections and receive logs as an HTTP server, also referred to as a "webhook"

      Logs from inbound connection can be normalized, filtered and forwarded using existing features of the collector.

      At a minimum must be able to receive logs in the kube-apiserver webhook format, but should be (simply) configurable to receive logs from other popular HTTP protocols and REST APIs (e.g. vector, loki, OTLP etc.)

      Non Goals

      Not a universal HTTP server. Aim to be flexible enough for mainstream HTTP logging use-cases without complex configuration.

      Motivation

      Collect logs from Kube-APIserver in hypershift configuration, where the log files are inaccessible so must use the webhook logging option.

      Feeds into a more general requirement for server-side features, begins an extentable "input" framework much like the existing "output" framework.

      Alternatives

      None good.

      Acceptance Criteria

      • Add "receiver" abstraction to CLF, implement HTTP receiver.
      • Auto-create a named service for each source
        • Service delegates each incoming connection to an arbitrary collector instance (kube-proxy load-balancing)
        • Kube-apiserver webhook logs received by the collector
        • HTTP input logs can be forwarded in all ways that file-based logs can.

      Risk and Assumptions

      Acting as server is a new concept for the collector.

      Documentation Considerations

      Doc for input types will be similar in effort  to documenting output types

      Open Questions

      Minimum acceptance criteria

      Consider other HTTP log protocols for basic acceptance criteria, e.g. vector, fluentd, loki, OTLP, others?

      Additional Notes

      Attachments

        Issue Links

          Activity

            People

              rhn-engineering-aconway Alan Conway
              rhn-engineering-aconway Alan Conway
              Anping Li Anping Li
              Votes:
              1 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: