Uploaded image for project: 'RHEL'
  1. RHEL
  2. RHEL-25613

Inter-processor interrupts when reloading TuneD

    • tuned-2.24.0-0.1.rc1.el9
    • No
    • Important
    • Patch, Upstream
    • rhel-sst-cs-net-perf-services
    • ssg_core_services
    • 23
    • 3
    • False
    • Hide

      None

      Show
      None
    • None
    • Red Hat Enterprise Linux
    • None
    • All
    • None

      What were you trying to do that didn't work?

      On SMP systems, inter-processor interrupts are triggered by TuneD when reloading TuneD even when using the same profile after the reload. This causes unwanted disruption (latency spikes) on latency-sensitive workloads.

      Please provide the package NVR for which bug is seen:

      1. rpm -q tuned
        tuned-2.22.0-0.1.rc1.el9.noarch
      2. uname -r
        5.14.0-284.30.1.rt14.315.el9_2.x86_64

      How reproducible:

      Always when using the affected TuneD plug-in and plug-in options.

      Steps to reproduce

      1. systemctl enable tuned --now
      2. stress-ng -c 1 --taskset 1
      3. trace-cmd record -e 'call_function*' -f 'CPU == 1' sleep 10
      4. pkill -e -HUP tuned
      5. trace-cmd report

      Expected results

      No unnecessary IPIs during TuneD reload/restart.

      Actual results

      No IPIs found during reload in the following plug-ins: [acpi], [bootloader], [disk], [irqbalance], [net], [scheduler], [scsi_host], [selinux], [service], [systemd]
      Plug-ins not tested: [audio], [eeepc_she], [modules], [mounts], [script], [usb], [video], [vm]

      Plug-ins with IPIs found during profile reload.

      [rtentsk]
      =========
      When reloading the same profile while using the [rtentsk] plug-in triggers
      a lagre amount of IPIs, but they all seem to be in 0-2us range.

      [cpu]
      =====
      1) governor; [1-30us] even when writing the same/existing value causes IPIs

      stress-ng-cpu-7772 [001] 12211.893188: call_function_single_entry: vector=251
      stress-ng-cpu-7772 [001] 12211.893218: call_function_single_exit: vector=251

      echo performance > /sys/devices/system/cpu/cpufreq/policy1/scaling_governor

      2) energy_perf_bias; [3-4us] even when writing the same/existing value causes IPIs; IPIs are also caused when reading the value of energy_perf_bias of the CPU the workload is running on.
      echo performance > /sys/devices/system/cpu/cpu1/power/energy_perf_bias

      3) min_perf_pct; [0-6us] even when writing the same/existing value causes IPIs;
      echo 23 > /sys/devices/system/cpu/intel_pstate/min_perf_pct

      4) force_latency; [3-6us] calling x86_energy_perf_policy [-r] causes IPIs

      5) energy_performance_preference; [1-2us] even when writing the same/existing value causes IPIs; IPIs are also caused when reading the value of energy_performance_preference of the CPU the workload is running on.
      echo performance > /sys/devices/system/cpu/cpufreq/policy1/energy_performance_preference

      6) no_turbo; [1-7us] even when writing the same/existing value
      echo 0 > /sys/devices/system/cpu/intel_pstate/no_turbo

      [sys]
      =====
      changing a value of sysctls can also cause [1-4us] IPIs; examples:
      kernel.nmi_watchdog
      kernel.timer_migration

      [sysfs]
      =======
      changing a value of a sysfs file can also cause [1-6us] IPIs; examples:
      /sys/devices/system/cpu/cpufreq/policy1/scaling_min_freq
      /sys/devices/system/cpu/cpufreq/policy1/scaling_max_freq

              pzacik@redhat.com Pavol Zacik
              jmencak Jiri Mencak
              Jaroslav Skarvada Jaroslav Skarvada
              Robin Hack Robin Hack
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

                Created:
                Updated:
                Resolved: