Uploaded image for project: 'OpenShift Logging'
  1. OpenShift Logging
  2. LOG-5247

can not forward logs to kafka when compression=gzip,zlib

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Not a Bug
    • Icon: Undefined Undefined
    • None
    • Logging 5.9.0
    • Log Collection
    • False
    • None
    • False
    • NEW
    • OBSDA-549 - Reliability and performance tuning for log collection
    • NEW
    • Bug Fix

      Description of problem:

      When compression=gzip, vector can not forward logs to kafka.

      oc logs clf-logs-djv92
Creating the directory used for persisting Vector state /var/lib/vector/amq-aosqe/clf-logs
Starting Vector process...
2024-03-14T15:20:59.950615Z ERROR vector::topology::builder: Configuration error. error=Sink "output_kafka_app": creating kafka producer failed: Client config error: Unsupported value "gzip" for configuration property "compression.codec": zlib not available at build time compression.codec gzip

      When compression=zlib, vector can not forward logs to kafka.

      oc logs clf-logs-6n4hm
Creating the directory used for persisting Vector state /var/lib/vector/amq-aosqe/clf-logs
Starting Vector process...
2024-03-14T15:21:52.924203Z ERROR vector::cli: Configuration error. error=unknown variant `zlib`, expected one of `none`, `gzip`, `snappy`, `lz4`, `zstd`

      Steps to Reproduce:

      1. Deploy AMQ Stream(kafka 2.6)
      2. Forward logs to kafka using compression=gzip

      Expected results:

      Fix the gzip error in upstream.
      Address the supported compression `none`, `snappy`, `lz4`, `zstd`

      Additional info:

              Unassigned Unassigned
              rhn-support-anli Anping Li
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated:
                Resolved: