Goals

This epic has been restricted to cover the preview version of this feature, minimal features to act as webhook to a kube-apiserver. Completing the feature to be configurable for other use cases is in a separate epic LOG-4562.

Collector can be configured to listen for HTTP connections and receive logs as an HTTP server, also referred to as a "webhook"

Logs from inbound connection can be normalized, filtered and forwarded using existing features of the collector.

At a minimum must be able to receive logs in the kube-apiserver webhook format, but should be (simply) configurable to receive logs from other popular HTTP protocols and REST APIs (e.g. vector, loki, OTLP etc.)

Non Goals

Not a universal HTTP server. Aim to be flexible enough for mainstream HTTP logging use-cases without complex configuration.

Motivation

Collect logs from Kube-APIserver in hypershift configuration, where the log files are inaccessible so must use the webhook logging option.

Feeds into a more general requirement for server-side features, begins an extentable "input" framework much like the existing "output" framework.

Alternatives

None good.

Acceptance Criteria

• Add "receiver" abstraction to CLF, implement HTTP receiver.
• Auto-create a named service for each source
  • Service delegates each incoming connection to an arbitrary collector instance (kube-proxy load-balancing)
  • Kube-apiserver webhook logs received by the collector
  • HTTP input logs can be forwarded in all ways that file-based logs can.

Risk and Assumptions

Acting as server is a new concept for the collector.

Documentation Considerations

Doc for input types will be similar in effort  to documenting output types

Open Questions

Minimum acceptance criteria

Consider other HTTP log protocols for basic acceptance criteria, e.g. vector, fluentd, loki, OTLP, others?

Additional Notes