Uploaded image for project: 'OpenShift Logging'
  1. OpenShift Logging
  2. LOG-1623

Metric`log_collected_bytes_total` is not exposed

    XMLWordPrintable

Details

    • Logging (Core) - Sprint 205
    • Passed
    • NEW
    • NEW
    • Hide
      Before this update, not having `@type collected_tail_monitor`as one of the `<source>` elements in the `fluent.conf` file caused the new fluent plugin, https://github.com/openshift/origin-aggregated-logging/tree/release-5.2/fluentd/lib/fluent-plugin-collected, to not load. This update fixes the issue by adding the following source section to `fluent.conf`:
      ----
      <source>
        @type collected_tail_monitor
        <labels>
          host ${hostname}
        </labels>
      </source>
      ----
      Show
      Before this update, not having `@type collected_tail_monitor`as one of the `<source>` elements in the `fluent.conf` file caused the new fluent plugin, https://github.com/openshift/origin-aggregated-logging/tree/release-5.2/fluentd/lib/fluent-plugin-collected, to not load. This update fixes the issue by adding the following source section to `fluent.conf`: ---- <source>   @type collected_tail_monitor   <labels>     host ${hostname}   </labels> </source> ----

    Description

      When testing latest  logging 5.2, the metric`log_collected_bytes_total` is not exposed:

      $ oc exec fluentd-wpmtm -- curl -k -H "Authorization: Bearer `oc sa get-token prometheus-k8s -n openshift-monitoring`" -s -H "Content-type: application/json" https://172.30.205.232:2112/metrics
      Defaulted container "fluentd" out of: fluentd, logfilesmetricexporter, fluentd-init (init)
      # HELP go_gc_duration_seconds A summary of the pause duration of garbage collection cycles.
      # TYPE go_gc_duration_seconds summary
      go_gc_duration_seconds{quantile="0"} 3.4822e-05
      go_gc_duration_seconds{quantile="0.25"} 3.7027e-05
      go_gc_duration_seconds{quantile="0.5"} 4.5873e-05
      go_gc_duration_seconds{quantile="0.75"} 6.9477e-05
      go_gc_duration_seconds{quantile="1"} 0.000100475
      go_gc_duration_seconds_sum 0.000388743
      go_gc_duration_seconds_count 7
      # HELP go_goroutines Number of goroutines that currently exist.
      # TYPE go_goroutines gauge
      go_goroutines 12
      # HELP go_info Information about the Go environment.
      # TYPE go_info gauge
      go_info{version="go1.15.13"} 1
      # HELP go_memstats_alloc_bytes Number of bytes allocated and still in use.
      # TYPE go_memstats_alloc_bytes gauge
      go_memstats_alloc_bytes 7.257008e+06
      # HELP go_memstats_alloc_bytes_total Total number of bytes allocated, even if freed.
      # TYPE go_memstats_alloc_bytes_total counter
      go_memstats_alloc_bytes_total 2.147036e+07
      # HELP go_memstats_buck_hash_sys_bytes Number of bytes used by the profiling bucket hash table.
      # TYPE go_memstats_buck_hash_sys_bytes gauge
      go_memstats_buck_hash_sys_bytes 1.447888e+06
      # HELP go_memstats_frees_total Total number of frees.
      # TYPE go_memstats_frees_total counter
      go_memstats_frees_total 49607
      # HELP go_memstats_gc_cpu_fraction The fraction of this program's available CPU time used by the GC since the program started.
      # TYPE go_memstats_gc_cpu_fraction gauge
      go_memstats_gc_cpu_fraction 3.3753944316300868e-06
      # HELP go_memstats_gc_sys_bytes Number of bytes used for garbage collection system metadata.
      # TYPE go_memstats_gc_sys_bytes gauge
      go_memstats_gc_sys_bytes 4.876304e+06
      # HELP go_memstats_heap_alloc_bytes Number of heap bytes allocated and still in use.
      # TYPE go_memstats_heap_alloc_bytes gauge
      go_memstats_heap_alloc_bytes 7.257008e+06
      # HELP go_memstats_heap_idle_bytes Number of heap bytes waiting to be used.
      # TYPE go_memstats_heap_idle_bytes gauge
      go_memstats_heap_idle_bytes 5.8179584e+07
      # HELP go_memstats_heap_inuse_bytes Number of heap bytes that are in use.
      # TYPE go_memstats_heap_inuse_bytes gauge
      go_memstats_heap_inuse_bytes 8.241152e+06
      # HELP go_memstats_heap_objects Number of allocated objects.
      # TYPE go_memstats_heap_objects gauge
      go_memstats_heap_objects 11740
      # HELP go_memstats_heap_released_bytes Number of heap bytes released to OS.
      # TYPE go_memstats_heap_released_bytes gauge
      go_memstats_heap_released_bytes 5.7573376e+07
      # HELP go_memstats_heap_sys_bytes Number of heap bytes obtained from system.
      # TYPE go_memstats_heap_sys_bytes gauge
      go_memstats_heap_sys_bytes 6.6420736e+07
      # HELP go_memstats_last_gc_time_seconds Number of seconds since 1970 of last garbage collection.
      # TYPE go_memstats_last_gc_time_seconds gauge
      go_memstats_last_gc_time_seconds 1.6268504019521108e+09
      # HELP go_memstats_lookups_total Total number of pointer lookups.
      # TYPE go_memstats_lookups_total counter
      go_memstats_lookups_total 0
      # HELP go_memstats_mallocs_total Total number of mallocs.
      # TYPE go_memstats_mallocs_total counter
      go_memstats_mallocs_total 61347
      # HELP go_memstats_mcache_inuse_bytes Number of bytes in use by mcache structures.
      # TYPE go_memstats_mcache_inuse_bytes gauge
      go_memstats_mcache_inuse_bytes 6944
      # HELP go_memstats_mcache_sys_bytes Number of bytes used for mcache structures obtained from system.
      # TYPE go_memstats_mcache_sys_bytes gauge
      go_memstats_mcache_sys_bytes 16384
      # HELP go_memstats_mspan_inuse_bytes Number of bytes in use by mspan structures.
      # TYPE go_memstats_mspan_inuse_bytes gauge
      go_memstats_mspan_inuse_bytes 82960
      # HELP go_memstats_mspan_sys_bytes Number of bytes used for mspan structures obtained from system.
      # TYPE go_memstats_mspan_sys_bytes gauge
      go_memstats_mspan_sys_bytes 98304
      # HELP go_memstats_next_gc_bytes Number of heap bytes when next garbage collection will take place.
      # TYPE go_memstats_next_gc_bytes gauge
      go_memstats_next_gc_bytes 7.976368e+06
      # HELP go_memstats_other_sys_bytes Number of bytes used for other system allocations.
      # TYPE go_memstats_other_sys_bytes gauge
      go_memstats_other_sys_bytes 721952
      # HELP go_memstats_stack_inuse_bytes Number of bytes in use by the stack allocator.
      # TYPE go_memstats_stack_inuse_bytes gauge
      go_memstats_stack_inuse_bytes 688128
      # HELP go_memstats_stack_sys_bytes Number of bytes obtained from system for stack allocator.
      # TYPE go_memstats_stack_sys_bytes gauge
      go_memstats_stack_sys_bytes 688128
      # HELP go_memstats_sys_bytes Number of bytes obtained from system.
      # TYPE go_memstats_sys_bytes gauge
      go_memstats_sys_bytes 7.4269696e+07
      # HELP go_threads Number of OS threads created.
      # TYPE go_threads gauge
      go_threads 9
      # HELP log_logged_bytes_total Total number of bytes written to a single log file path, accounting for rotations
      # TYPE log_logged_bytes_total counter
      log_logged_bytes_total{containername="csi-liveness-probe",namespace="openshift-cluster-csi-drivers",path="/var/log/containers/aws-ebs-csi-driver-node-v96m2_openshift-cluster-csi-drivers_csi-liveness-probe-70a3f2b62db7da67c618994444f5ff0e7baa37dbb66078e2a78e9c1bd7fa5da7.log",podname="aws-ebs-csi-driver-node-v96m2"} 134411
      log_logged_bytes_total{containername="fluentd",namespace="openshift-logging",path="/var/log/containers/fluentd-6wzsj_openshift-logging_fluentd-0ad7d9d5da2326d5e3c563670b506d5c326f15adacb9dc65343aceb94b092246.log",podname="fluentd-6wzsj"} 307
      log_logged_bytes_total{containername="indexmanagement",namespace="openshift-logging",path="/var/log/containers/elasticsearch-im-audit-27114150-mmcbl_openshift-logging_indexmanagement-045b802df01d62d62a6d2cf367f1540119b56ee02dfb8624b4cf0748dc9f59a1.log",podname="elasticsearch-im-audit-27114150-mmcbl"} 0
      log_logged_bytes_total{containername="indexmanagement",namespace="openshift-logging",path="/var/log/containers/elasticsearch-im-audit-27114165-7v4ts_openshift-logging_indexmanagement-b9d6fd404dbef3c696dd2841c11c86b828387667b4e8984dd888ded3e3a28e3f.log",podname="elasticsearch-im-audit-27114165-7v4ts"} 1920
      log_logged_bytes_total{containername="indexmanagement",namespace="openshift-logging",path="/var/log/containers/elasticsearch-im-infra-27114150-229gw_openshift-logging_indexmanagement-90f43be0349b5ef76e6082cfc63e570c0611b9a3b7a0781125f25009acd84c8d.log",podname="elasticsearch-im-infra-27114150-229gw"} 0
      log_logged_bytes_total{containername="indexmanagement",namespace="openshift-logging",path="/var/log/containers/elasticsearch-im-infra-27114165-6mfbx_openshift-logging_indexmanagement-5d54328d1d960d9356604c6dd3f56edac01b053ff6abb6e9f213172753136859.log",podname="elasticsearch-im-infra-27114165-6mfbx"} 2361
      log_logged_bytes_total{containername="logfilesmetricexporter",namespace="openshift-logging",path="/var/log/containers/fluentd-6wzsj_openshift-logging_logfilesmetricexporter-161b31d2def320b6aa85224796ff86c7bafecaf310f2543cddf4b37365ca7fac.log",podname="fluentd-6wzsj"} 0
      log_logged_bytes_total{containername="network-metrics-daemon",namespace="openshift-multus",path="/var/log/containers/network-metrics-daemon-82trc_openshift-multus_network-metrics-daemon-aa6560d525bd2c38fd5be4171850a3661415777a121d17677a25448fb8a57b9b.log",podname="network-metrics-daemon-82trc"} 13713
      log_logged_bytes_total{containername="node-ca",namespace="openshift-image-registry",path="/var/log/containers/node-ca-dwhm4_openshift-image-registry_node-ca-f975f25e4500a492191f86bb93fe77592303646521cbb6e7ce155ffdd3048640.log",podname="node-ca-dwhm4"} 70644
      log_logged_bytes_total{containername="sdn",namespace="openshift-sdn",path="/var/log/containers/sdn-j7nbs_openshift-sdn_sdn-d84155740d733c1d9ee785ba90071440ae54e36d1084c33e622e412790f28f45.log",podname="sdn-j7nbs"} 339079
      log_logged_bytes_total{containername="serve-healthcheck-canary",namespace="openshift-ingress-canary",path="/var/log/containers/ingress-canary-rw2fh_openshift-ingress-canary_serve-healthcheck-canary-c7595fb461518c12667a849515bbde66f54afef144fc1994751dcbc0c1811f2e.log",podname="ingress-canary-rw2fh"} 4602
      # HELP process_cpu_seconds_total Total user and system CPU time spent in seconds.
      # TYPE process_cpu_seconds_total counter
      process_cpu_seconds_total 0.23
      # HELP process_max_fds Maximum number of open file descriptors.
      # TYPE process_max_fds gauge
      process_max_fds 1.048576e+06
      # HELP process_open_fds Number of open file descriptors.
      # TYPE process_open_fds gauge
      process_open_fds 15
      # HELP process_resident_memory_bytes Resident memory size in bytes.
      # TYPE process_resident_memory_bytes gauge
      process_resident_memory_bytes 2.4068096e+07
      # HELP process_start_time_seconds Start time of the process since unix epoch in seconds.
      # TYPE process_start_time_seconds gauge
      process_start_time_seconds 1.62684967831e+09
      # HELP process_virtual_memory_bytes Virtual memory size in bytes.
      # TYPE process_virtual_memory_bytes gauge
      process_virtual_memory_bytes 1.274748928e+09
      # HELP process_virtual_memory_max_bytes Maximum amount of virtual memory available in bytes.
      # TYPE process_virtual_memory_max_bytes gauge
      process_virtual_memory_max_bytes 1.8446744073709552e+19
      # HELP promhttp_metric_handler_requests_in_flight Current number of scrapes being served.
      # TYPE promhttp_metric_handler_requests_in_flight gauge
      promhttp_metric_handler_requests_in_flight 1
      # HELP promhttp_metric_handler_requests_total Total number of scrapes by HTTP status code.
      # TYPE promhttp_metric_handler_requests_total counter
      promhttp_metric_handler_requests_total{code="200"} 54
      promhttp_metric_handler_requests_total{code="500"} 0
      promhttp_metric_handler_requests_total{code="503"} 0
      

      Attachments

        Issue Links

          Activity

            People

              pmoogi pratibha moogi
              qitang@redhat.com Qiaoling Tang
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: