Uploaded image for project: 'RHEL'
  1. RHEL
  2. RHEL-46758

[Dev] Output of pcp2openmetrics can not be parsed by Prometheus parser

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Undefined Undefined
    • None
    • rhel-9.5
    • pcp
    • None
    • None
    • None
    • 1
    • rhel-sst-pt-pcp
    • ssg_platform_tools
    • 2
    • None
    • PCP Sprint 6
    • None
    • None
    • None

      What were you trying to do that didn't work?

      As the Prometheus project is the origin of the OpenMetrics format, I would expect the parser in the Prometheus project will be able to parse the output of pcp2openmetrics tool.

      If there is any other way how ensure the syntax of the generated metrics file is correct (except the Prometheus parser I am using) please let me know.

      Also feel free to close this issue as Wont-Do if there is no requirement to be compliant with the syntax used by the Prometheus project.

      Please provide the package NVR for which bug is seen:

      pcp-6.2.2-1.el9

      How reproducible:

      Always

      Steps to reproduce

      1. Generate a file containing some metrics data from the pcp2openmetrics tool 
        # pcp2openmetrics -a $(ls -1 /var/log/pcp/pmlogger/$(hostname)/*.0 | tail -n 1) -s 1 -F /tmp/x 'kernel.all.load'
      2. Use the data file as an input of the Prometheus parser 
        # pip3 install --upgrade prometheus_client
# python3 -c 'import sys; from prometheus_client.openmetrics import parser; list(parser.text_string_to_metric_families(sys.stdin.buffer.read().decode("utf-8")))' < /tmp/x

      Expected results

      The parser is able to parse the file generated by the pcp2openmetrics tool

      Actual results

      # python3 -c 'import sys; from prometheus_client.openmetrics import parser; list(parser.text_string_to_metric_families(sys.stdin.buffer.read().decode("utf-8")))' < /tmp/x
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python3.9/site-packages/prometheus_client/openmetrics/parser.py", line 18, in text_string_to_metric_families
    yield from text_fd_to_metric_families(StringIO.StringIO(text))
  File "/usr/local/lib/python3.9/site-packages/prometheus_client/openmetrics/parser.py", line 497, in text_fd_to_metric_families
    raise ValueError("Received blank line")
ValueError: Received blank line

      Let's remove the first blank line and check it again:

      # vim /tmp/x
# python3 -c 'import sys; from prometheus_client.openmetrics import parser; list(parser.text_string_to_metric_families(sys.stdin.buffer.read().decode("utf-8")))' < /tmp/x
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/prometheus_client/openmetrics/parser.py", line 81, in _parse_value
    return int(value)
ValueError: invalid literal for int() with base 10: 'kernel.all.load'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python3.9/site-packages/prometheus_client/openmetrics/parser.py", line 18, in text_string_to_metric_families
    yield from text_fd_to_metric_families(StringIO.StringIO(text))
  File "/usr/local/lib/python3.9/site-packages/prometheus_client/openmetrics/parser.py", line 540, in text_fd_to_metric_families
    sample = _parse_sample(line)
  File "/usr/local/lib/python3.9/site-packages/prometheus_client/openmetrics/parser.py", line 260, in _parse_sample
    value, timestamp, exemplar = _parse_remaining_text(remaining_text)
  File "/usr/local/lib/python3.9/site-packages/prometheus_client/openmetrics/parser.py", line 283, in _parse_remaining_text
    val = _parse_value(split_text[0])
  File "/usr/local/lib/python3.9/site-packages/prometheus_client/openmetrics/parser.py", line 83, in _parse_value
    return float(value)
ValueError: could not convert string to float: 'kernel.all.load'

      Let's remove the comment line and check it again:

      # vim /tmp/x
# python3 -c 'import sys; from prometheus_client.openmetrics import parser; list(parser.text_string_to_metric_families(sys.stdin.buffer.read().decode("utf-8")))' < /tmp/x
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/prometheus_client/openmetrics/parser.py", line 81, in _parse_value
    return int(value)
ValueError: invalid literal for int() with base 10: 'PCP5'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python3.9/site-packages/prometheus_client/openmetrics/parser.py", line 18, in text_string_to_metric_families
    yield from text_fd_to_metric_families(StringIO.StringIO(text))
  File "/usr/local/lib/python3.9/site-packages/prometheus_client/openmetrics/parser.py", line 540, in text_fd_to_metric_families
    sample = _parse_sample(line)
  File "/usr/local/lib/python3.9/site-packages/prometheus_client/openmetrics/parser.py", line 260, in _parse_sample
    value, timestamp, exemplar = _parse_remaining_text(remaining_text)
  File "/usr/local/lib/python3.9/site-packages/prometheus_client/openmetrics/parser.py", line 283, in _parse_remaining_text
    val = _parse_value(split_text[0])
  File "/usr/local/lib/python3.9/site-packages/prometheus_client/openmetrics/parser.py", line 83, in _parse_value
    return float(value)
ValueError: could not convert string to float: 'PCP5'

      Let's remove the ':PCP' line and check it again:

      # vim /tmp/x
# python3 -c 'import sys; from prometheus_client.openmetrics import parser; list(parser.text_string_to_metric_families(sys.stdin.buffer.read().decode("utf-8")))' < /tmp/x
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python3.9/site-packages/prometheus_client/openmetrics/parser.py", line 18, in text_string_to_metric_families
    yield from text_fd_to_metric_families(StringIO.StringIO(text))
  File "/usr/local/lib/python3.9/site-packages/prometheus_client/openmetrics/parser.py", line 497, in text_fd_to_metric_families
    raise ValueError("Received blank line")
ValueError: Received blank line

      Let's remove the last blank line and check it again:

      # vim /tmp/x
# python3 -c 'import sys; from prometheus_client.openmetrics import parser; list(parser.text_string_to_metric_families(sys.stdin.buffer.read().decode("utf-8")))' < /tmp/x
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/local/lib/python3.9/site-packages/prometheus_client/openmetrics/parser.py", line 18, in text_string_to_metric_families
    yield from text_fd_to_metric_families(StringIO.StringIO(text))
  File "/usr/local/lib/python3.9/site-packages/prometheus_client/openmetrics/parser.py", line 614, in text_fd_to_metric_families
    raise ValueError("Missing # EOF at end")
ValueError: Missing # EOF at end

      Let's add the missing EOF line and check it again:

      # vim /tmp/x
# python3 -c 'import sys; from prometheus_client.openmetrics import parser; list(parser.text_string_to_metric_families(sys.stdin.buffer.read().decode("utf-8")))' < /tmp/x
#

      Not the Prometheus parser is able to parse the file. The file, after all the modifications looks like this:

      # cat /tmp/x
# HELP kernel_all_load 1, 5 and 15 minute load average
# TYPE kernel_all_load gauge
kernel_all_load{domainname="localdomain",groupid="993",hostname="vm-10-0-186-57.hosted.upshift.rdu2.redhat.com",machineid="03640ff5862a45f5b2944e3ded241caa",userid="993",instname="1 minute",instid="1",agent="linux"} 0.150 1717004975.904928
kernel_all_load{domainname="localdomain",groupid="993",hostname="vm-10-0-186-57.hosted.upshift.rdu2.redhat.com",machineid="03640ff5862a45f5b2944e3ded241caa",userid="993",instname="5 minute",instid="5",agent="linux"} 0.100 1717004975.904928
kernel_all_load{domainname="localdomain",groupid="993",hostname="vm-10-0-186-57.hosted.upshift.rdu2.redhat.com",machineid="03640ff5862a45f5b2944e3ded241caa",userid="993",instname="15 minute",instid="15",agent="linux"} 0.090 1717004975.904928
# EOF

              rh-ee-lchilton Lauren Chilton
              jkurik@redhat.com Jan Kurik
              pcp-maint pcp-maint
              Jan Kurik Jan Kurik
              Jacob Valdez Jacob Valdez (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

                Created:
                Updated:
                Resolved: