Uploaded image for project: 'RHEL'
  1. RHEL
  2. RHEL-6976

regex split code in tracer not scaling very well

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Undefined Undefined
    • None
    • rhel-8.6.0
    • tracer
    • None
    • Moderate
    • None
    • False
    • Hide

      None

      Show
      None
    • None
    • None
    • None
    • None
    • If docs needed, set a value
    • None

      Tracer appears to be hung due to taking too much time.

      Customer provided some data for debugging the issue.

      Based on some strace, perf and cProfile data, we can see that the bottleneck
      appears to be the regex split code. In a very short run, to profile a bit, it
      is called almost 2 million times, and using more than 90% of the processing
      time.

      Sample cProfile output:

      ncalls tottime percall cumtime percall filename:lineno(function)
      1745231 413.627 0.000 413.627 0.000

      {method 'split' of '_sre.SRE_Pattern' objects}

      9175435 35.478 0.000 473.426 0.000 /usr/lib/python3.6/site-packages/tracer/resources/processes.py:76(name)
      1746812 17.331 0.000 17.332 0.000

      {method 'join' of 'str' objects}

      23604269 4.621 0.000 4.621 0.000 /usr/lib/python3.6/site-packages/tracer/resources/processes.py:114(_attr)
      4362 3.370 0.001 3.371 0.001

      {method 'send_message_with_reply_and_block' of '_dbus_bindings.Connection' objects}

      164719 3.258 0.000 4.943 0.000 /usr/lib64/python3.6/site-packages/psutil/_pslinux.py:1800(get_blocks)
      1510 2.475 0.002 4.500 0.003

      {method 'block' of 'dbus.lowlevel.PendingCall' objects}

      3058071 2.466 0.000 478.441 0.000 /usr/lib/python3.6/site-packages/tracer/resources/applications.py:272(<lambda>)
      2294 2.111 0.001 2.111 0.001

      {built-in method posix.listdir}

      3058071 1.617 0.000 317.814 0.000 /usr/lib/python3.6/site-packages/tracer/resources/processes.py:276(real_name)
      1512 1.560 0.001 1.968 0.001

      {method 'Parse' of 'pyexpat.xmlparser' objects}

      3935101 1.324 0.000 1.324 0.000

      {method 'split' of 'bytes' objects}

      2281 1.206 0.001 6.560 0.003 /usr/lib/python3.6/site-packages/tracer/resources/processes.py:41(all)

              jkadlcik Jakub Kadlčík
              rhn-support-pandrade Paulo Andrade
              Jakub Kadlčík Jakub Kadlčík
              rhel8-maint@redhat.com rhel8-maint@redhat.com
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Created:
                Updated: