Major Description of problem:
Satellite is no longer returning the expected error information for a failure with insights-client, particularly when trying to upload a --payload
with incorrect --content-type.
How reproducible:
100%
Is this issue a regression from an earlier version:
Yes
Steps to Reproduce:
1. insights-client --payload <tarfile> --content-type bzip
2.
3.
Actual behavior:
Against satellite 6.15:
[root@dhcp-8-29-228 ~]# insights-client --payload insights-archive.tar.gz --content-type bzip Uploading Insights data. Invalid content-type. Upload failed.
We see the expected returned error.
Against Satellite 6.16:
[root@dhcp-8-29-228 ~]# insights-client --payload insights-archive.tar.gz --content-type bzip Uploading Insights data. Invalid content-type. Upload failed.
We see the expected returned error.
Against Satellite 6.17.0.1 / 6.17.4 (Updated for Errata that fixed another 500 ISE related to insights upload):
[root@dhcp-8-29-228 ~]# insights-client --payload insights-archive.tar.gz --content-type bzip Uploading Insights data. Upload attempt 1 of 1 failed! Reason: 500: Internal Server Error All attempts to upload have failed! Please see /var/log/insights-client/insights-client-payload.log for additional information Upload failed.
The following information is captured in the referenced client log:
2025-09-05 15:50:41,013 DEBUG insights.client.connection:972 content-type: bzip 2025-09-05 15:50:41,013 DEBUG insights.client.connection:973 Uploading insights-archive.tar.gz to https://cct-sat617.usersys.redhat.com:443/redhat_access/r/insights/platform/ingress/v1/upload 2025-09-05 15:50:41,013 NETWORK insights.client.connection:207 POST https://cct-sat617.usersys.redhat.com:443/redhat_access/r/insights/platform/ingress/v1/upload headers={} attachments=file:insights-archive.tar.gz,metadata 2025-09-05 15:50:41,170 NETWORK insights.client.connection:212 HTTP Status: 500 Internal Server Error 2025-09-05 15:50:41,170 NETWORK insights.client.connection:214 HTTP Response Text: {"status":500,"error":"Internal Server Error"} 2025-09-05 15:50:41,171 DEBUG insights.client.connection:979 Request ID: None 2025-09-05 15:50:41,171 DEBUG insights.client.connection:993 Upload archive failed with status code 500 2025-09-05 15:50:41,171 ERROR insights.client.client:445 Upload attempt 1 of 1 failed! Reason: 500: Internal S erver Error 2025-09-05 15:50:41,171 ERROR insights.client.client:452 All attempts to upload have failed! 2025-09-05 15:50:41,171 ERROR insights.client.phase.v1:348 Upload failed.
The foreman traceback is as follows:
2025-09-05T15:56:37 [I|app|5a689c1d] Started POST "/redhat_access/r/insights/platform/ingress/v1/upload" for 10.8.29.228 at 2025-09-05 15:56:37 -0400 2025-09-05T15:56:37 [I|app|5a689c1d] Processing by InsightsCloud::Api::MachineTelemetriesController#forward_request as JSON 2025-09-05T15:56:37 [I|app|5a689c1d] Parameters: {"file"=>#[ActionDispatch::Http::UploadedFile:0x00007f99c2db3180 @tempfile=#<Tempfile:/tmp/RackMultipart20250905-1981-4o397r.gz], @content_type="bzip", @original_filename="insights-archive.tar.gz", @headers="Content-Disposition: form-data; name=\"file\"; filename=\"insights-archive.tar.gz\"\r\nContent-Type: bzip\r\n">, "metadata"=>#[ActionDispatch::Http::UploadedFile:0x00007f99c2db3090 @tempfile=#<Tempfile:/tmp/RackMultipart20250905-1981-204i2a], @content_type=nil, @original_filename="metadata", @headers="Content-Disposition: form-data; name=\"metadata\"; filename=\"metadata\"\r\n">, "path"=>"platform/ingress/v1/upload"} 2025-09-05T15:56:37 [D|tax|5a689c1d] Current location set to none 2025-09-05T15:56:37 [D|tax|5a689c1d] Current organization set to none 2025-09-05T15:56:37 [D|app|5a689c1d] Request parameters for telemetry request: {"branch_id"=>"d8622990-a6a5-4ed1-b972-ba1e1ae27a7a"} 2025-09-05T15:56:37 [D|app|5a689c1d] User agent for telemetry is: Satellite/6.17.4;foreman_rh_cloud/11.4.4;insights-client/3.10.1 (Core 3.6.3; requests 2.32.4) Red Hat Enterprise Linux 10.1 (CPython 3.12.11; Linux 6.12.0-122.el10.x86_64); bash 2025-09-05T15:56:37 [D|app|5a689c1d] Sending headers: {} 2025-09-05T15:56:37 [D|app|5a689c1d] Sending request to: https://cert.cloud.stage.redhat.com/api/ingress/v1/upload 2025-09-05T15:56:37 [D|app|5a689c1d] Response headers for request url https://cert.cloud.stage.redhat.com/api/ingress/v1/upload with status code 415 are: {:server=>"openresty", :content_length=>"0", :x_rh_insights_request_id=>"920bef05a5ec4ea4bae1a71e7f68beab, 920bef05a5ec4ea4bae1a71e7f68beab", :x_content_type_options=>"nosniff", :date=>"Fri, 05 Sep 2025 19:56:37 GMT", :connection=>"close", :set_cookie=>["e15dd7c1deaec3a6b67fca153d520b0b=8fe2628a7bdeed29aae2fc6a971e69a3; path=/; HttpOnly; Secure; SameSite=None"], :x_rh_edge_request_id=>"129b1d89", :x_rh_edge_reference_id=>"0.966bdc17.1757102197.129b1d89", :x_rh_edge_cache_status=>"NotCacheable from child", :x_frame_options=>"SAMEORIGIN", :strict_transport_security=>"max-age=31536000; includeSubDomains"} and body: 2025-09-05T15:56:37 [W|app|5a689c1d] Action failed 2025-09-05T15:56:37 [I|app|5a689c1d] Backtrace for 'Action failed' error (NoMethodError): undefined method {{code' for nil:NilClass 5a689c1d | /usr/share/gems/gems/foreman_rh_cloud-11.4.4/app/controllers/insights_cloud/api/machine_telemetries_controller.rb:124:in }}update_host_insights_status' 5a689c1d | /usr/share/gems/gems/activesupport-7.0.8.7/lib/active_support/callbacks.rb:400:in {{block in make_lambda' 5a689c1d | /usr/share/gems/gems/activesupport-7.0.8.7/lib/active_support/callbacks.rb:236:in }}block in halting_and_conditional' 5a689c1d | /usr/share/gems/gems/activesupport-7.0.8.7/lib/active_support/callbacks.rb:599:in {{block in invoke_after' 5a689c1d | /usr/share/gems/gems/activesupport-7.0.8.7/lib/active_support/callbacks.rb:599:in }}each' 5a689c1d | /usr/share/gems/gems/activesupport-7.0.8.7/lib/active_support/callbacks.rb:599:in {{invoke_after' 5a689c1d | /usr/share/gems/gems/activesupport-7.0.8.7/lib/active_support/callbacks.rb:132:in }}block in run_callbacks' 5a689c1d | /usr/share/foreman/app/controllers/concerns/foreman/controller/timezone.rb:10:in {{set_timezone' 5a689c1d | /usr/share/gems/gems/activesupport-7.0.8.7/lib/active_support/callbacks.rb:127:in }}block in run_callbacks' 5a689c1d | /usr/share/foreman/app/models/concerns/foreman/thread_session.rb:32:in {{clear_thread' 5a689c1d | /usr/share/gems/gems/activesupport-7.0.8.7/lib/active_support/callbacks.rb:127:in }}block in run_callbacks' 5a689c1d | /usr/share/foreman/app/controllers/concerns/foreman/controller/topbar_sweeper.rb:12:in {{set_topbar_sweeper_controller' 5a689c1d | /usr/share/gems/gems/activesupport-7.0.8.7/lib/active_support/callbacks.rb:127:in }}block in run_callbacks' 5a689c1d | /usr/share/gems/gems/audited-5.8.0/lib/audited/sweeper.rb:16:in {{around' 5a689c1d | /usr/share/gems/gems/activesupport-7.0.8.7/lib/active_support/callbacks.rb:127:in }}block in run_callbacks' 5a689c1d | /usr/share/gems/gems/audited-5.8.0/lib/audited/sweeper.rb:16:in {{around' 5a689c1d | /usr/share/gems/gems/activesupport-7.0.8.7/lib/active_support/callbacks.rb:127:in }}block in run_callbacks' 5a689c1d | /usr/share/gems/gems/activesupport-7.0.8.7/lib/active_support/callbacks.rb:138:in {{run_callbacks' 5a689c1d | /usr/share/gems/gems/actionpack-7.0.8.7/lib/abstract_controller/callbacks.rb:233:in }}process_action' 5a689c1d | /usr/share/gems/gems/actionpack-7.0.8.7/lib/action_controller/metal/rescue.rb:23:in {{process_action' 5a689c1d | /usr/share/gems/gems/actionpack-7.0.8.7/lib/action_controller/metal/instrumentation.rb:67:in }}block in process_action' 5a689c1d | /usr/share/gems/gems/activesupport-7.0.8.7/lib/active_support/notifications.rb:206:in {{block in instrument' 5a689c1d | /usr/share/gems/gems/activesupport-7.0.8.7/lib/active_support/notifications/instrumenter.rb:24:in }}instrument' 5a689c1d | /usr/share/gems/gems/activesupport-7.0.8.7/lib/active_support/notifications.rb:206:in {{instrument' 5a689c1d | /usr/share/gems/gems/actionpack-7.0.8.7/lib/action_controller/metal/instrumentation.rb:66:in }}process_action' 5a689c1d | /usr/share/gems/gems/actionpack-7.0.8.7/lib/action_controller/metal/params_wrapper.rb:259:in {{process_action' 5a689c1d | /usr/share/gems/gems/activerecord-7.0.8.7/lib/active_record/railties/controller_runtime.rb:27:in }}process_action' 5a689c1d | /usr/share/gems/gems/actionpack-7.0.8.7/lib/abstract_controller/base.rb:151:in {{process' 5a689c1d | /usr/share/gems/gems/actionview-7.0.8.7/lib/action_view/rendering.rb:39:in }}process' 5a689c1d | /usr/share/gems/gems/actionpack-7.0.8.7/lib/action_controller/metal.rb:188:in {{dispatch' 5a689c1d | /usr/share/gems/gems/actionpack-7.0.8.7/lib/action_controller/metal.rb:251:in }}dispatch' 5a689c1d | /usr/share/gems/gems/actionpack-7.0.8.7/lib/action_dispatch/routing/route_set.rb:49:in {{dispatch' 5a689c1d | /usr/share/gems/gems/actionpack-7.0.8.7/lib/action_dispatch/routing/route_set.rb:32:in }}serve' 5a689c1d | /usr/share/gems/gems/actionpack-7.0.8.7/lib/action_dispatch/routing/mapper.rb:18:in {{block in [class:Constraints]' 5a689c1d | /usr/share/gems/gems/actionpack-7.0.8.7/lib/action_dispatch/routing/mapper.rb:48:in }}serve' 5a689c1d | /usr/share/gems/gems/actionpack-7.0.8.7/lib/action_dispatch/journey/router.rb:50:in {{block in serve' 5a689c1d | /usr/share/gems/gems/actionpack-7.0.8.7/lib/action_dispatch/journey/router.rb:32:in }}each' 5a689c1d | /usr/share/gems/gems/actionpack-7.0.8.7/lib/action_dispatch/journey/router.rb:32:in {{serve' 5a689c1d | /usr/share/gems/gems/actionpack-7.0.8.7/lib/action_dispatch/routing/route_set.rb:852:in }}call' 5a689c1d | /usr/share/gems/gems/katello-4.16.0.9/lib/katello/middleware/organization_created_enforcer.rb:18:in {{call' 5a689c1d | /usr/share/gems/gems/katello-4.16.0.9/lib/katello/middleware/event_daemon.rb:10:in }}call' 5a689c1d | /usr/share/gems/gems/actionpack-7.0.8.7/lib/action_dispatch/middleware/static.rb:23:in {{call' 5a689c1d | /usr/share/gems/gems/actionpack-7.0.8.7/lib/action_dispatch/middleware/static.rb:23:in }}call' 5a689c1d | /usr/share/gems/gems/apipie-dsl-2.6.2/lib/apipie_dsl/static_dispatcher.rb:67:in {{call' 5a689c1d | /usr/share/gems/gems/apipie-rails-1.4.2/lib/apipie/static_dispatcher.rb:74:in }}call' 5a689c1d | /usr/share/gems/gems/actionpack-7.0.8.7/lib/action_dispatch/middleware/static.rb:23:in {{call' 5a689c1d | /usr/share/gems/gems/actionpack-7.0.8.7/lib/action_dispatch/middleware/static.rb:23:in }}call' 5a689c1d | /usr/share/gems/gems/actionpack-7.0.8.7/lib/action_dispatch/middleware/static.rb:23:in {{call' 5a689c1d | /usr/share/gems/gems/actionpack-7.0.8.7/lib/action_dispatch/middleware/static.rb:23:in }}call' 5a689c1d | /usr/share/gems/gems/actionpack-7.0.8.7/lib/action_dispatch/middleware/static.rb:23:in {{call' 5a689c1d | /usr/share/gems/gems/actionpack-7.0.8.7/lib/action_dispatch/middleware/static.rb:23:in }}call' 5a689c1d | /usr/share/gems/gems/actionpack-7.0.8.7/lib/action_dispatch/middleware/static.rb:23:in {{call' 5a689c1d | /usr/share/gems/gems/actionpack-7.0.8.7/lib/action_dispatch/middleware/static.rb:23:in }}call' 5a689c1d | /usr/share/gems/gems/actionpack-7.0.8.7/lib/action_dispatch/middleware/static.rb:23:in {{call' 5a689c1d | /usr/share/gems/gems/actionpack-7.0.8.7/lib/action_dispatch/middleware/static.rb:23:in }}call' 5a689c1d | /usr/share/gems/gems/actionpack-7.0.8.7/lib/action_dispatch/middleware/static.rb:23:in {{call' 5a689c1d | /usr/share/gems/gems/actionpack-7.0.8.7/lib/action_dispatch/middleware/static.rb:23:in }}call' 5a689c1d | /usr/share/gems/gems/actionpack-7.0.8.7/lib/action_dispatch/middleware/static.rb:23:in {{call' 5a689c1d | /usr/share/gems/gems/actionpack-7.0.8.7/lib/action_dispatch/middleware/static.rb:23:in }}call' 5a689c1d | /usr/share/gems/gems/actionpack-7.0.8.7/lib/action_dispatch/middleware/static.rb:23:in {{call' 5a689c1d | /usr/share/foreman/lib/foreman/middleware/libvirt_connection_cleaner.rb:9:in }}call' 5a689c1d | /usr/share/foreman/lib/foreman/middleware/telemetry.rb:10:in {{call' 5a689c1d | /usr/share/gems/gems/apipie-rails-1.4.2/lib/apipie/middleware/checksum_in_headers.rb:27:in }}call' 5a689c1d | /usr/share/gems/gems/rack-2.2.14/lib/rack/tempfile_reaper.rb:15:in {{call' 5a689c1d | /usr/share/gems/gems/rack-2.2.14/lib/rack/etag.rb:27:in }}call' 5a689c1d | /usr/share/gems/gems/rack-2.2.14/lib/rack/conditional_get.rb:40:in {{call' 5a689c1d | /usr/share/gems/gems/rack-2.2.14/lib/rack/head.rb:12:in }}call' 5a689c1d | /usr/share/gems/gems/actionpack-7.0.8.7/lib/action_dispatch/http/permissions_policy.rb:38:in {{call' 5a689c1d | /usr/share/gems/gems/actionpack-7.0.8.7/lib/action_dispatch/http/content_security_policy.rb:39:in }}call' 5a689c1d | /usr/share/foreman/lib/foreman/middleware/logging_context_session.rb:22:in {{call' 5a689c1d | /usr/share/gems/gems/rack-2.2.14/lib/rack/session/abstract/id.rb:266:in }}context' 5a689c1d | /usr/share/gems/gems/rack-2.2.14/lib/rack/session/abstract/id.rb:260:in {{call' 5a689c1d | /usr/share/gems/gems/actionpack-7.0.8.7/lib/action_dispatch/middleware/cookies.rb:704:in }}call' 5a689c1d | /usr/share/gems/gems/actionpack-7.0.8.7/lib/action_dispatch/middleware/callbacks.rb:27:in {{block in call' 5a689c1d | /usr/share/gems/gems/activesupport-7.0.8.7/lib/active_support/callbacks.rb:99:in }}run_callbacks' 5a689c1d | /usr/share/gems/gems/actionpack-7.0.8.7/lib/action_dispatch/middleware/callbacks.rb:26:in {{call' 5a689c1d | /usr/share/gems/gems/actionpack-7.0.8.7/lib/action_dispatch/middleware/debug_exceptions.rb:28:in }}call' 5a689c1d | /usr/share/gems/gems/actionpack-7.0.8.7/lib/action_dispatch/middleware/show_exceptions.rb:29:in {{call' 5a689c1d | /usr/share/gems/gems/railties-7.0.8.7/lib/rails/rack/logger.rb:40:in }}call_app' 5a689c1d | /usr/share/gems/gems/railties-7.0.8.7/lib/rails/rack/logger.rb:27:in {{call' 5a689c1d | /usr/share/gems/gems/sprockets-rails-3.5.2/lib/sprockets/rails/quiet_assets.rb:17:in }}call' 5a689c1d | /usr/share/foreman/lib/foreman/middleware/logging_context_request.rb:11:in {{call' 5a689c1d | /usr/share/gems/gems/actionpack-7.0.8.7/lib/action_dispatch/middleware/remote_ip.rb:93:in }}call' 5a689c1d | /usr/share/gems/gems/actionpack-7.0.8.7/lib/action_dispatch/middleware/request_id.rb:26:in {{call' 5a689c1d | /usr/share/gems/gems/katello-4.16.0.9/lib/katello/prevent_json_parsing.rb:12:in }}call' 5a689c1d | /usr/share/gems/gems/rack-2.2.14/lib/rack/method_override.rb:24:in {{call' 5a689c1d | /usr/share/gems/gems/rack-2.2.14/lib/rack/runtime.rb:22:in }}call' 5a689c1d | /usr/share/gems/gems/activesupport-7.0.8.7/lib/active_support/cache/strategy/local_cache_middleware.rb:29:in {{call' 5a689c1d | /usr/share/gems/gems/actionpack-7.0.8.7/lib/action_dispatch/middleware/executor.rb:14:in }}call' 5a689c1d | /usr/share/gems/gems/rack-2.2.14/lib/rack/sendfile.rb:110:in {{call' 5a689c1d | /usr/share/gems/gems/actionpack-7.0.8.7/lib/action_dispatch/middleware/ssl.rb:77:in }}call' 5a689c1d | /usr/share/gems/gems/actionpack-7.0.8.7/lib/action_dispatch/middleware/host_authorization.rb:131:in {{call' 5a689c1d | /usr/share/gems/gems/secure_headers-6.7.0/lib/secure_headers/middleware.rb:11:in }}call' 5a689c1d | /usr/share/gems/gems/railties-7.0.8.7/lib/rails/engine.rb:530:in {{call' 5a689c1d | /usr/share/gems/gems/railties-7.0.8.7/lib/rails/railtie.rb:226:in }}public_send' 5a689c1d | /usr/share/gems/gems/railties-7.0.8.7/lib/rails/railtie.rb:226:in {{method_missing' 5a689c1d | /usr/share/gems/gems/rack-2.2.14/lib/rack/urlmap.rb:74:in }}block in call' 5a689c1d | /usr/share/gems/gems/rack-2.2.14/lib/rack/urlmap.rb:58:in {{each' 5a689c1d | /usr/share/gems/gems/rack-2.2.14/lib/rack/urlmap.rb:58:in }}call' 5a689c1d | /usr/share/gems/gems/puma-6.6.0/lib/puma/configuration.rb:279:in {{call' 5a689c1d | /usr/share/gems/gems/puma-6.6.0/lib/puma/request.rb:99:in }}block in handle_request' 5a689c1d | /usr/share/gems/gems/puma-6.6.0/lib/puma/thread_pool.rb:390:in {{with_force_shutdown' 5a689c1d | /usr/share/gems/gems/puma-6.6.0/lib/puma/request.rb:98:in }}handle_request' 5a689c1d | /usr/share/gems/gems/puma-6.6.0/lib/puma/server.rb:472:in {{process_client' 5a689c1d | /usr/share/gems/gems/puma-6.6.0/lib/puma/server.rb:254:in }}block in run' 5a689c1d | /usr/share/gems/gems/puma-6.6.0/lib/puma/thread_pool.rb:167:in {{block in spawn_thread' 5a689c1d | /usr/share/gems/gems/logging-2.4.0/lib/logging/diagnostic_context.rb:474:in }}block in create_with_logging_context' 2025-09-05T15:56:37 [I|app|5a689c1d] Completed 500 Internal Server Error in 142ms (Views: 0.3ms | ActiveRecord: 4.6ms | Allocations: 14086) 2025-09-05T15:56:37 [F|app|5a689c1d] 5a689c1d | AbstractController::DoubleRenderError (Render and/or redirect were called multiple times in this action. Please note that you may only call render OR redirect, and at most once per action. Also note that neither redirect nor render terminate execution of the action, so if you want to exit an action after redirecting, you need to do something like "redirect_to(...) and return".): 5a689c1d | 5a689c1d | app/controllers/api/v2/base_controller.rb:168:in {{render_error' 5a689c1d | app/controllers/api/base_controller.rb:29:in }}block in [class:BaseController]' 5a689c1d | katello (4.16.0.9) lib/katello/middleware/organization_created_enforcer.rb:18:in {{call' 5a689c1d | katello (4.16.0.9) lib/katello/middleware/event_daemon.rb:10:in }}call' 5a689c1d | lib/foreman/middleware/libvirt_connection_cleaner.rb:9:in {{call' 5a689c1d | lib/foreman/middleware/telemetry.rb:10:in }}call' 5a689c1d | lib/foreman/middleware/logging_context_session.rb:22:in {{call' 5a689c1d | lib/foreman/middleware/logging_context_request.rb:11:in }}call' 5a689c1d | katello (4.16.0.9) lib/katello/prevent_json_parsing.rb:12:in `call'
Expected behavior:
Should return error as seen in 6.15/6.16.
Business Impact / Additional info: