Uploaded image for project: 'OpenShift Logging'
  1. OpenShift Logging
  2. LOG-2016

Allow use of multiple Elasticsearch nodes as outputs in ClusterLogForwarder

XMLWordPrintable

    • Icon: Story Story
    • Resolution: Obsolete
    • Icon: Undefined Undefined
    • None
    • None
    • Log Collection
    • None
    • False
    • False
    • NEW
    • NEW

      Story

      Extend the Elasticsearch output configuration to include a field:

          // Nodes is a list of additional URLs of Elasticsearch nodes
    // to use for load-balancing and fail-over.
    //
    // +optional
    Nodes []string `json:"nodes,omitempty"`

      If specified, the URLs listed, plus the URL from the Output.URL field, should be configured as the "hosts" in the fluentd elasticsearch plug-in.

      Note this is the same pattern as "Brokers" field in the Kafka output config, so that code can be used as a guide.

      Acceptance Criteria

      • Unit tests to verify the correct fluentd configuration is generated
      • Functional test to verify the "Nodes" URLs are used if the main URL is invalid.

      Notes

      Kibana using "elasticsearch.hosts":
      https://www.elastic.co/guide/en/kibana/current/production.html#high-availability
      Fluentd using "hosts":
      https://docs.fluentd.org/output/elasticsearch#hosts-optional
      Vector v0.28-1 is using "endpoints"
      https://vector.dev/docs/reference/configuration/sinks/elasticsearch/#endpoints

            [LOG-2016] Allow use of multiple Elasticsearch nodes as outputs in ClusterLogForwarder

            Sergey Yedrikov added a comment -

            In vector v0.28-1, the elasticsearch sink supports multiple endpoints: https://vector.dev/docs/reference/configuration/sinks/elasticsearch/#endpoints

            Sergey Yedrikov added a comment - In vector v0.28-1, the elasticsearch sink supports multiple endpoints: https://vector.dev/docs/reference/configuration/sinks/elasticsearch/#endpoints

            Theodor Li (Inactive) added a comment -

            rhn-engineering-aconway This looks good to me, thank you very much.

            Theodor Li (Inactive) added a comment - rhn-engineering-aconway This looks good to me, thank you very much.

            Alan Conway added a comment -

            Originally raised as github issue https://github.com/openshift/cluster-logging-operator/issues/1238

            Alan Conway added a comment - Originally raised as github issue https://github.com/openshift/cluster-logging-operator/issues/1238

              Unassigned Unassigned
              rhn-engineering-aconway Alan Conway
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated:
                Resolved: