Description of problem:

Customer is running the Registered Hosts report in Satellite 6.8 and using a search term : installed_package_name = splunkforwarder. Customer has about 1200 hosts registered to satellite. The report errors out and seems to cause a OOM error killing off a sidekiq process.

Version-Release number of selected component (if applicable):

How reproducible:

Every time certain search terms are used. Not all search terms error out.

Jun 24 11:57:26 rhnsat-03 kernel: Out of memory: Kill process 1323 (sidekiq) score 454 or sacrifice child
Jun 24 11:57:26 rhnsat-03 kernel: Killed process 1323 (sidekiq), UID 994, total-vm:18395528kB, anon-rss:17486112kB, file-rss:0kB, shmem-rss:0kB
Jun 24 11:57:28 rhnsat-03 systemd: dynflow-sidekiq@worker.service: main process exited, code=killed, status=9/KILL
Jun 24 11:57:28 rhnsat-03 systemd: Unit dynflow-sidekiq@worker.service entered failed state.
Jun 24 11:57:28 rhnsat-03 systemd: dynflow-sidekiq@worker.service failed.
Jun 24 11:57:29 rhnsat-03 systemd: dynflow-sidekiq@worker.service holdoff time over, scheduling restart.
Jun 24 11:57:29 rhnsat-03 systemd: Stopped Foreman jobs daemon - worker on sidekiq.
Jun 24 11:57:29 rhnsat-03 systemd: Starting Foreman jobs daemon - worker on sidekiq...
Jun 24 11:57:32 rhnsat-03 dynflow-sidekiq@worker: 2021-06-24T15:57:32.486Z 43034 TID-b8smm INFO: GitLab reliable fetch activated!
Jun 24 11:57:32 rhnsat-03 dynflow-sidekiq@worker: 2021-06-24T15:57:32.486Z 43034 TID-mnx0u INFO: Booting Sidekiq 5.2.7 with redis options {:id=>"Sidekiq-server-PID-43034", :url=>"redis://localhost:6379/0"}
Jun 24 11:57:42 rhnsat-03 dynflow-sidekiq@worker: /usr/share/foreman/lib/foreman.rb:8: warning: already initialized constant Foreman::UUID_REGEXP
Jun 24 11:57:42 rhnsat-03 dynflow-sidekiq@worker: /usr/share/foreman/lib/foreman.rb:8: warning: previous definition of UUID_REGEXP was here
Jun 24 11:57:55 rhnsat-03 dynflow-sidekiq@worker: /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_theme_satellite-7.0.1.5/app/models/concerns/distributor_version.rb:5: warning: already initialized constant Katello::Glue::Provider::DISTRIBUTOR_VERSION
Jun 24 11:57:55 rhnsat-03 dynflow-sidekiq@worker: /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.1.27/app/models/katello/glue/provider.rb:3: warning: previous definition of DISTRIBUTOR_VERSION was here
Jun 24 11:58:01 rhnsat-03 dynflow-sidekiq@worker: /opt/theforeman/tfm/root/usr/share/gems/gems/foreman_azure_rm-2.1.3/app/models/foreman_azure_rm/azure_rm_compute.rb:13: warning: circular argument reference - sdk
Jun 24 11:58:17 rhnsat-03 systemd: Started Foreman jobs daemon - worker on sidekiq.

Task export data:

INput:

—
job_class: TemplateRenderJob
queue: default
job_data:
job_class: TemplateRenderJob
job_id: beebaa52-1352-4772-be55-80e2ef0f7abb
provider_job_id: beebaa52-1352-4772-be55-80e2ef0f7abb
queue_name: default
priority:
arguments:

• template_id: 108-Host - Registered Content Hosts
  gzip: false
  send_mail: false
  mail_to: casl@andrew.cmu.edu
  format: html
  input_values:
  '5':
  value: installed_package_name = splunkforwarder
  _aj_hash_with_indifferent_access: true
  _aj_hash_with_indifferent_access: true
  _aj_hash_with_indifferent_access: true
• user_id: 35
  _aj_symbol_keys:
• user_id
  executions: 0
  exception_executions: {}
  locale: en
  timezone: America/New_York
  enqueued_at: '2021-06-24T15:56:42Z'
  current_request_id: 84477708-32cb-4383-96c5-c1459b574e60
  current_timezone: America/New_York
  current_user_id: 35
  current_organization_id: 5
  current_location_id: 2

output:

Error:

StandardError

Could not transition step from pending to running, step already in running.

—

• "/opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matchers/abstract.rb:74:in
  `block in assigns'"
• "/opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matchers/abstract.rb:73:in
  `tap'"
• "/opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matchers/abstract.rb:73:in
  `assigns'"
• "/opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matching.rb:56:in
  `match_value'"
• "/opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matching.rb:36:in
  `block in match?'"
• "/opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matching.rb:35:in
  `each'"
• "/opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matching.rb:35:in
  `match?'"
• "/opt/theforeman/tfm/root/usr/share/gems/gems/algebrick-0.7.3/lib/algebrick/matching.rb:23:in
  `match'"
• "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/execution_plan/steps/error.rb:13:in
  `new'"
• "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action.rb:501:in
  `set_error'"
• "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action.rb:559:in
  `execute_run'"
• "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action.rb:285:in
  `execute'"
• "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:18:in
  `block (2 levels) in execute'"
• "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/execution_plan/steps/abstract.rb:167:in
  `with_meta_calculation'"
• "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:17:in
  `block in execute'"
• "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:32:in
  `open_action'"
• "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:16:in
  `execute'"
• "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/director.rb:68:in
  `execute'"
• "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/executors/sidekiq/worker_jobs.rb:11:in
  `block (2 levels) in perform'"
• "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/executors.rb:18:in
  `run_user_code'"
• "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/executors/sidekiq/worker_jobs.rb:9:in
  `block in perform'"
• "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/executors/sidekiq/worker_jobs.rb:25:in
  `with_telemetry'"
• "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/executors/sidekiq/worker_jobs.rb:8:in
  `perform'"
• "/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/executors/sidekiq/serialization.rb:27:in
  `perform'"
• "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:192:in
  `execute_job'"
• "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:165:in
  `block (2 levels) in process'"
• "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:128:in
  `block in invoke'"
• "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/middleware/chain.rb:133:in
  `invoke'"
• "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:164:in
  `block in process'"
• "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:137:in
  `block (6 levels) in dispatch'"
• "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/job_retry.rb:109:in
  `local'"
• "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:136:in
  `block (5 levels) in dispatch'"
• "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq.rb:37:in
  `block in '"
• "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:132:in
  `block (4 levels) in dispatch'"
• "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:250:in
  `stats'"
• "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:127:in
  `block (3 levels) in dispatch'"
• "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/job_logger.rb:8:in
  `call'"
• "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:126:in
  `block (2 levels) in dispatch'"
• "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/job_retry.rb:74:in
  `global'"
• "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:125:in
  `block in dispatch'"
• "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/logging.rb:48:in
  `with_context'"
• "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/logging.rb:42:in
  `with_job_hash_context'"
• "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:124:in
  `dispatch'"
• "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:163:in
  `process'"
• "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:83:in
  `process_one'"
• "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/processor.rb:71:in
  `run'"
• "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/util.rb:16:in
  `watchdog'"
• "/opt/theforeman/tfm/root/usr/share/gems/gems/sidekiq-5.2.7/lib/sidekiq/util.rb:25:in
  `block in safe_thread'"
• "/opt/theforeman/tfm/root/usr/share/gems/gems/logging-2.3.0/lib/logging/diagnostic_context.rb:474:in
  `block in create_with_logging_context'"

Actual results:

report fails to complete, OOM error

Expected results:

Report generates as expected