Description of problem:
During normal operations puma sometimes throw

Errno::ENOENT: No such file or directory @ rb_sysopen - /usr/share/foreman/tmp/cache/0E3/781/.permissions_check.220838320.2072127.637105
...
2022-03-03T06:05:09 [I|app|5d13507c] Completed 500 Internal Server Error in 1129ms (Views: 0.2ms | ActiveRecord: 38.9ms | Allocations: 87301)

Version-Release number of selected component (if applicable):
satellite-6.9.6.1-1.el7sat.noarch

How reproducible:
Only on customer side

Actual results:
2022-03-03T06:05:08 [I|app|5d13507c] Started GET "/api/v2/hosts/548/errata?page=1&per_page=1" for 10.196.65.30 at 2022-03-03 06:05:08 +0000
2022-03-03T06:05:08 [I|app|5d13507c] Processing by Katello::Api::V2::HostErrataController#index as HTML
2022-03-03T06:05:08 [I|app|5d13507c] Parameters:

2022-03-03T06:05:09 [W|app|5d13507c] Scoped order is ignored, it's forced to be batch order.
2022-03-03T06:05:09 [I|app|5d13507c] Authorized user orchit_prod(orchit_prod)
2022-03-03T06:05:09 [E|app|5d13507c] Errno::ENOENT: No such file or directory @ rb_sysopen - /usr/share/foreman/tmp/cache/0E3/781/.permissions_check.220838320.2072127.637105
5d13507c | /opt/rh/rh-ruby25/root/usr/share/ruby/fileutils.rb:1057:in `initialize'
5d13507c | /opt/rh/rh-ruby25/root/usr/share/ruby/fileutils.rb:1057:in `open'
5d13507c | /opt/rh/rh-ruby25/root/usr/share/ruby/fileutils.rb:1057:in `rescue in block in touch'
5d13507c | /opt/rh/rh-ruby25/root/usr/share/ruby/fileutils.rb:1053:in `block in touch'
5d13507c | /opt/rh/rh-ruby25/root/usr/share/ruby/fileutils.rb:1051:in `each'
5d13507c | /opt/rh/rh-ruby25/root/usr/share/ruby/fileutils.rb:1051:in `touch'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.4/lib/active_support/core_ext/file/atomic.rb:65:in `probe_stat_in'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.4/lib/active_support/core_ext/file/atomic.rb:35:in `block in atomic_write'
5d13507c | /opt/rh/rh-ruby25/root/usr/share/ruby/tempfile.rb:295:in `open'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.4/lib/active_support/core_ext/file/atomic.rb:24:in `atomic_write'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.4/lib/active_support/cache/file_store.rb:87:in `write_entry'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.4/lib/active_support/cache/strategy/local_cache.rb:150:in `write_entry'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.4/lib/active_support/cache.rb:465:in `block in write'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.4/lib/active_support/cache.rb:686:in `block in instrument'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.4/lib/active_support/notifications.rb:182:in `instrument'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.4/lib/active_support/cache.rb:686:in `instrument'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.4/lib/active_support/cache.rb:463:in `write'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.4/lib/active_support/cache.rb:720:in `save_block_result_to_cache'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.4/lib/active_support/cache.rb:332:in `fetch'
5d13507c | /usr/share/foreman/app/models/user.rb:522:in `taxonomy_and_child_ids'
5d13507c | /usr/share/foreman/app/models/user.rb:536:in `location_and_child_ids'
5d13507c | /usr/share/foreman/app/models/taxonomies/location.rb:22:in `block in <class:Location>'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.4/lib/active_record/relation.rb:412:in `instance_exec'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.4/lib/active_record/relation.rb:412:in `block in _exec_scope'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.4/lib/active_record/relation.rb:784:in `_scoping'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.4/lib/active_record/relation.rb:412:in `_exec_scope'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.4/lib/active_record/scoping/named.rb:192:in `block in scope'
5d13507c | /usr/share/foreman/app/models/user.rb:497:in `my_locations'
5d13507c | /usr/share/foreman/app/controllers/concerns/application_shared.rb:28:in `block in set_taxonomy'
5d13507c | /usr/share/foreman/app/controllers/concerns/application_shared.rb:27:in `each'
5d13507c | /usr/share/foreman/app/controllers/concerns/application_shared.rb:27:in `set_taxonomy'
5d13507c | /usr/share/foreman/app/controllers/concerns/application_shared.rb:18:in `set_current_user'
5d13507c | /usr/share/foreman/app/controllers/concerns/foreman/controller/authentication.rb:17:in `authenticate'
5d13507c | /usr/share/foreman/app/controllers/api/base_controller.rb:192:in `authorize'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.4/lib/active_support/callbacks.rb:428:in `block in make_lambda'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.4/lib/active_support/callbacks.rb:200:in `block (2 levels) in halting'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/abstract_controller/callbacks.rb:34:in `block (2 levels) in <module:Callbacks>'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.4/lib/active_support/callbacks.rb:201:in `block in halting'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.4/lib/active_support/callbacks.rb:513:in `block in invoke_before'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.4/lib/active_support/callbacks.rb:513:in `each'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.4/lib/active_support/callbacks.rb:513:in `invoke_before'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.4/lib/active_support/callbacks.rb:110:in `block in run_callbacks'
5d13507c | /usr/share/foreman/app/models/concerns/foreman/thread_session.rb:32:in `clear_thread'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.4/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
5d13507c | /usr/share/foreman/app/controllers/concerns/foreman/controller/topbar_sweeper.rb:12:in `set_topbar_sweeper_controller'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.4/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/audited-4.9.0/lib/audited/sweeper.rb:14:in `around'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.4/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/audited-4.9.0/lib/audited/sweeper.rb:14:in `around'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.4/lib/active_support/callbacks.rb:121:in `block in run_callbacks'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.4/lib/active_support/callbacks.rb:139:in `run_callbacks'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/abstract_controller/callbacks.rb:41:in `process_action'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_controller/metal/rescue.rb:22:in `process_action'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_controller/metal/instrumentation.rb:33:in `block in process_action'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.4/lib/active_support/notifications.rb:180:in `block in instrument'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.4/lib/active_support/notifications/instrumenter.rb:24:in `instrument'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.4/lib/active_support/notifications.rb:180:in `instrument'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_controller/metal/instrumentation.rb:32:in `process_action'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_controller/metal/params_wrapper.rb:245:in `process_action'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/activerecord-6.0.3.4/lib/active_record/railties/controller_runtime.rb:27:in `process_action'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.1.40/app/controllers/katello/concerns/api/api_controller.rb:40:in `process_action'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/abstract_controller/base.rb:136:in `process'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/actionview-6.0.3.4/lib/action_view/rendering.rb:39:in `process'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_controller/metal.rb:190:in `dispatch'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_controller/metal.rb:254:in `dispatch'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_dispatch/routing/route_set.rb:50:in `dispatch'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_dispatch/routing/route_set.rb:33:in `serve'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_dispatch/routing/mapper.rb:18:in `block in <class:Constraints>'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_dispatch/routing/mapper.rb:48:in `serve'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_dispatch/journey/router.rb:49:in `block in serve'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_dispatch/journey/router.rb:32:in `each'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_dispatch/journey/router.rb:32:in `serve'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_dispatch/routing/route_set.rb:834:in `call'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.1.40/lib/katello/middleware/event_daemon.rb:10:in `call'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_dispatch/middleware/static.rb:126:in `call'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_dispatch/middleware/static.rb:126:in `call'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_dispatch/middleware/static.rb:126:in `call'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/apipie-dsl-2.3.0/lib/apipie_dsl/static_dispatcher.rb:67:in `call'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/apipie-rails-0.5.17/lib/apipie/static_dispatcher.rb:66:in `call'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/apipie-rails-0.5.17/lib/apipie/extractor/recorder.rb:137:in `call'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_dispatch/middleware/static.rb:126:in `call'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_dispatch/middleware/static.rb:126:in `call'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_dispatch/middleware/static.rb:126:in `call'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_dispatch/middleware/static.rb:126:in `call'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_dispatch/middleware/static.rb:126:in `call'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_dispatch/middleware/static.rb:126:in `call'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_dispatch/middleware/static.rb:126:in `call'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_dispatch/middleware/static.rb:126:in `call'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_dispatch/middleware/static.rb:126:in `call'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_dispatch/middleware/static.rb:126:in `call'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_dispatch/middleware/static.rb:126:in `call'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_dispatch/middleware/static.rb:126:in `call'
5d13507c | /usr/share/foreman/lib/foreman/middleware/telemetry.rb:10:in `call'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/apipie-rails-0.5.17/lib/apipie/middleware/checksum_in_headers.rb:27:in `call'
5d13507c | /usr/share/foreman/lib/foreman/middleware/catch_json_parse_errors.rb:9:in `call'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/tempfile_reaper.rb:15:in `call'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/etag.rb:27:in `call'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/conditional_get.rb:27:in `call'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/head.rb:12:in `call'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_dispatch/http/content_security_policy.rb:18:in `call'
5d13507c | /usr/share/foreman/lib/foreman/middleware/logging_context_session.rb:22:in `call'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/session/abstract/id.rb:266:in `context'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/session/abstract/id.rb:260:in `call'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_dispatch/middleware/cookies.rb:648:in `call'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_dispatch/middleware/callbacks.rb:27:in `block in call'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.4/lib/active_support/callbacks.rb:101:in `run_callbacks'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_dispatch/middleware/callbacks.rb:26:in `call'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_dispatch/middleware/actionable_exceptions.rb:18:in `call'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_dispatch/middleware/debug_exceptions.rb:32:in `call'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_dispatch/middleware/show_exceptions.rb:33:in `call'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.4/lib/rails/rack/logger.rb:37:in `call_app'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.4/lib/rails/rack/logger.rb:28:in `call'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/sprockets-rails-3.2.1/lib/sprockets/rails/quiet_assets.rb:13:in `call'
5d13507c | /usr/share/foreman/lib/foreman/middleware/logging_context_request.rb:11:in `call'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_dispatch/middleware/remote_ip.rb:81:in `call'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_dispatch/middleware/request_id.rb:27:in `call'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.18.1.40/lib/katello/prevent_json_parsing.rb:12:in `call'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/method_override.rb:24:in `call'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/runtime.rb:22:in `call'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/activesupport-6.0.3.4/lib/active_support/cache/strategy/local_cache_middleware.rb:29:in `call'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_dispatch/middleware/executor.rb:14:in `call'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_dispatch/middleware/static.rb:126:in `call'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_dispatch/middleware/static.rb:126:in `call'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/sendfile.rb:110:in `call'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/actionpack-6.0.3.4/lib/action_dispatch/middleware/host_authorization.rb:76:in `call'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/secure_headers-6.3.0/lib/secure_headers/middleware.rb:11:in `call'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.4/lib/rails/engine.rb:527:in `call'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.4/lib/rails/railtie.rb:190:in `public_send'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/railties-6.0.3.4/lib/rails/railtie.rb:190:in `method_missing'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/urlmap.rb:74:in `block in call'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/urlmap.rb:58:in `each'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/rack-2.2.3/lib/rack/urlmap.rb:58:in `call'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/puma-4.3.6/lib/puma/configuration.rb:228:in `call'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/puma-4.3.6/lib/puma/server.rb:713:in `handle_request'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/puma-4.3.6/lib/puma/server.rb:472:in `process_client'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/puma-4.3.6/lib/puma/server.rb:328:in `block in run'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/puma-4.3.6/lib/puma/thread_pool.rb:134:in `block in spawn_thread'
5d13507c | /opt/theforeman/tfm/root/usr/share/gems/gems/logging-2.3.0/lib/logging/diagnostic_context.rb:474:in `block in create_with_logging_context'
2022-03-03T06:05:09 [I|app|5d13507c] Completed 500 Internal Server Error in 1129ms (Views: 0.2ms | ActiveRecord: 38.9ms | Allocations: 87301)

Expected results:
Completed 200 OK

Additional info: