Uploaded image for project: 'OpenShift Logging'
  1. OpenShift Logging
  2. LOG-5920

'maxWrite' and 'delivery' tuning options are missing under collector config for kafka

XMLWordPrintable

    • False
    • None
    • False
    • NEW
    • OBSDA-550 - Updated APIs for Logging 6.0
    • VERIFIED
    • Release Note Not Required
    • Log Collection - Sprint 257

      Description:
      When tuning options are configured under the CLF, only compression is reflected under the collector config. maxWrite and delivery type is missing under config.

      ...
      [sinks.output_kafka_app]
      type = "kafka"
      inputs = ["output_kafka_app_topic"]
      bootstrap_servers = "my-cluster-kafka-bootstrap.amq-aosqe.svc:9092"
      topic = "{{ _internal.output_kafka_app_topic }}"
      compression = "zstd"
      
      [sinks.output_kafka_app.encoding]
      codec = "json"
      timestamp_format = "rfc3339"
      except_fields = .["_internal"]
      ...
      

      Logging version: 6.0

      How reproducible: Always

      Expected Result:
      Config should have below config when maxWrite and delivery type is added under tuning config

      [sinks.output_kafka_app.batch]
      max_bytes = 10000000
      [sinks.output_kafka_app.buffer]
      when_full = "drop_newest"
      

      Actual Result:
      [sinks.output_kafka_app.batch] and [sinks.output_kafka_app.buffer] configs are missing

      CLF spec:

      spec:
        managementState: Managed
        outputs:
          - kafka:
              topic: topic-logging-app
              tuning:
                compression: zstd
                delivery: atMostOnce
                maxWrite: 10M
              url: 'tls://my-cluster-kafka-bootstrap.amq-aosqe.svc:9092/topic-logging-app'
            name: kafka-app
            type: kafka
        pipelines:
          - inputRefs:
              - application
            name: my-app-output
            outputRefs:
              - kafka-app
        serviceAccount:
          name: my-kafka-sa

      Additional Info: vector.toml attached

              rh-ee-calee Calvin Lee
              rhn-support-kbharti Kabir Bharti
              Kabir Bharti Kabir Bharti
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated:
                Resolved: