Uploaded image for project: 'OpenShift Logging'
  1. OpenShift Logging
  2. LOG-5921

'Could not generate topic for healthcheck' warnings on collector when forwarding to kafka topics

XMLWordPrintable

    • False
    • None
    • False
    • NEW
    • VERIFIED
    • Release Note Not Required
    • Log Collection - Sprint 257, Log Collection - Sprint 258

      Description of problem:

      Collector shows health check warnings when forwarding to kafka topics.

      collector logs: 

      Creating the directory used for persisting Vector state /var/lib/vector/openshift-logging/my-custom-clf
      Checking for buffer lock files
      /var/lib/vector/openshift-logging/my-custom-clf /usr/bin
      found lock files: ''
      /usr/bin
      Starting Vector process...
      2024-08-01T14:08:47.119614Z  WARN vector::sinks::kafka::sink: Could not generate topic for healthcheck. error=Missing fields on event: ["_internal.output_kafka_audit_topic"]
      2024-08-01T14:08:47.124083Z  WARN vector::sinks::kafka::sink: Could not generate topic for healthcheck. error=Missing fields on event: ["_internal.output_kafka_infra_topic"]
      2024-08-01T14:08:47.131341Z  WARN vector::sinks::kafka::sink: Could not generate topic for healthcheck. error=Missing fields on event: ["_internal.output_kafka_app_topic"]
      2024-08-01T14:08:47.134932Z  WARN vector::internal_events::file::source: Currently ignoring file too small to fingerprint. file=/var/log/ovn/acl-audit-log.log
      2024-08-01T14:09:03.540548Z  WARN vector::internal_events::file::source: Currently ignoring file too small to fingerprint. file=/var/log/pods/openshift-monitoring_node-exporter-shzsv_1600416d-1795-440a-b88f-22ad5c078529/init-textfile/0.log
      2024-08-01T14:09:03.540780Z  WARN vector::internal_events::file::source: Currently ignoring file too small to fingerprint. file=/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-xqcm9_89a660ff-32bb-4d8f-8701-839653e1faf4/kubecfg-setup/0.log
      2024-08-01T14:09:03.542123Z  WARN vector::internal_events::file::source: Currently ignoring file too small to fingerprint. file=/var/log/pods/openshift-multus_multus-additional-cni-plugins-wjrhr_b1734482-dc61-484c-bf68-3b93746e99d3/kube-multus-additional-cni-plugins/0.log

      Version-Release number of selected component (if applicable):

      Logging v6.0

      How reproducible:

      Always

      Steps to Reproduce:

      1. Deploy AMQ operator, Kafka cluster and create kafka topics.
      2. Create CLF to forward logs to kafka topics.
      3. Check collector logs.

      Actual results:

      'Could not generate topic for healthcheck' warnings on collector

      Expected results:

      No warnings/errors on collector during forwarding to kafka

      Additional info: 

      No such warnings when using CLO v5.9.z

      CLF and vector.toml attached.

        1. collector-config-for-kafka.txt
          20 kB
          Kabir Bharti
        2. kafka-topic-clf.txt
          6 kB
          Kabir Bharti

              rh-ee-calee Calvin Lee
              rhn-support-kbharti Kabir Bharti
              Kabir Bharti Kabir Bharti
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

                Created:
                Updated:
                Resolved: