Uploaded image for project: 'Red Hat OpenStack Services on OpenShift'
  1. Red Hat OpenStack Services on OpenShift
  2. OSPRH-16586

additionalConfig blocks rabbitmq pod start if defined

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done-Errata
    • Icon: Minor Minor
    • rhos-18.0.10 FR 3
    • rhos-18.0.9
    • infra-operator
    • None
    • 3
    • False
    • Hide

      None

      Show
      None
    • False
    • ?
    • infra-operator-container-1.0.12-2
    • rhos-ops-platform-services-pidone
    • None
    • PIDONE 18.0.8, Sprint 1, Sprint 2, Sprint 3
    • 4
    • Important

      To Reproduce Steps to reproduce the behavior:
      I have tried to customize RabbitMQ configuration files inside rabbitmq-server-0 pod via additionalConfig definition (spec.rabbitmq.templates.rabbitmq.rabbitmq.additionalConfig basically) in RHOSO controlplane with openstack-operator.v1.0.9. It looks like when any configuration definitions are provided this way, they trigger some problem inside the pod when it tries to write to local log file and fails.

      Example config extract:

        rabbitmq:
          enabled: true
          templates:
            rabbitmq:
              delayStartSeconds: 30
              override:
                service:
                  metadata:
                    annotations:
                      metallb.universe.tf/address-pool: internalapi
                      metallb.universe.tf/loadBalancerIPs: 172.17.0.85
                  spec:
                    type: LoadBalancer
              persistence:
                storage: 10Gi
              rabbitmq:
                additionalConfig: |
                  disk_free_limit.absolute = 2GB

      Example error:

      # oc logs rabbitmq-server-0
      Defaulted container "rabbitmq" out of: rabbitmq, setup-container (init)
      =ERROR REPORT==== 12-May-2025::18:20:57.446183 ===
      inet_config: syntax error in /etc/rabbitmq/erl_inetrc
      
      =ERROR REPORT==== 12-May-2025::18:20:57.446183 ===
      inet_config: syntax error in /etc/rabbitmq/erl_inetrc
      
      2025-05-12 18:20:58.758094+00:00 [warn] <0.158.0> TLS distribution: unexpected message: {'EXIT',<0.157.0>,shutdown}
      2025-05-12 18:20:58.758094+00:00 [warn] <0.158.0> 
      2025-05-12 18:20:59.369906+00:00 [info] <0.245.0> Feature flags: list of feature flags found:
      2025-05-12 18:20:59.369962+00:00 [info] <0.245.0> Feature flags:   [x] implicit_default_bindings
      2025-05-12 18:20:59.369982+00:00 [info] <0.245.0> Feature flags:   [x] maintenance_mode_status
      2025-05-12 18:20:59.369994+00:00 [info] <0.245.0> Feature flags:   [x] quorum_queue
      2025-05-12 18:20:59.370037+00:00 [info] <0.245.0> Feature flags:   [x] stream_queue
      2025-05-12 18:20:59.370047+00:00 [info] <0.245.0> Feature flags:   [x] user_limits
      2025-05-12 18:20:59.370056+00:00 [info] <0.245.0> Feature flags:   [x] virtual_host_metadata
      2025-05-12 18:20:59.370088+00:00 [info] <0.245.0> Feature flags: feature flag states written to disk: yes
      2025-05-12 18:20:59.536052+00:00 [noti] <0.44.0> Application syslog exited with reason: stopped
      2025-05-12 18:20:59.536128+00:00 [noti] <0.245.0> Logging: switching to configured handler(s); following messages may not be visible in this log output
      2025-05-12 18:20:59.552430+00:00 [erro] <0.245.0> 
      2025-05-12 18:20:59.552430+00:00 [erro] <0.245.0> BOOT FAILED
      2025-05-12 18:20:59.552430+00:00 [erro] <0.245.0> ===========
      2025-05-12 18:20:59.552430+00:00 [erro] <0.245.0> failed to open log file at '/var/log/rabbitmq/rabbit@rabbitmq-server-0.rabbitmq-nodes.openstack.log', reason: permission denied
      2025-05-12 18:20:59.552430+00:00 [erro] <0.245.0> 
      
      BOOT FAILED
      ===========
      failed to open log file at '/var/log/rabbitmq/rabbit@rabbitmq-server-0.rabbitmq-nodes.openstack.log', reason: permission denied
      
      2025-05-12 18:21:00.553961+00:00 [erro] <0.244.0>   crasher:
      2025-05-12 18:21:00.553961+00:00 [erro] <0.244.0>     initial call: application_master:init/4
      2025-05-12 18:21:00.553961+00:00 [erro] <0.244.0>     pid: <0.244.0>
      2025-05-12 18:21:00.553961+00:00 [erro] <0.244.0>     registered_name: []
      2025-05-12 18:21:00.553961+00:00 [erro] <0.244.0>     exception exit: {{cannot_log_to_file,
      2025-05-12 18:21:00.553961+00:00 [erro] <0.244.0>                          "/var/log/rabbitmq/rabbit@rabbitmq-server-0.rabbitmq-nodes.openstack.log",
      2025-05-12 18:21:00.553961+00:00 [erro] <0.244.0>                          eacces},
      2025-05-12 18:21:00.553961+00:00 [erro] <0.244.0>                      {rabbit,start,[normal,[]]}}
      2025-05-12 18:21:00.553961+00:00 [erro] <0.244.0>       in function  application_master:init/4 (application_master.erl, line 142)
      2025-05-12 18:21:00.553961+00:00 [erro] <0.244.0>     ancestors: [<0.243.0>]
      2025-05-12 18:21:00.553961+00:00 [erro] <0.244.0>     message_queue_len: 1
      2025-05-12 18:21:00.553961+00:00 [erro] <0.244.0>     messages: [{'EXIT',<0.245.0>,normal}]
      2025-05-12 18:21:00.553961+00:00 [erro] <0.244.0>     links: [<0.243.0>,<0.44.0>]
      2025-05-12 18:21:00.553961+00:00 [erro] <0.244.0>     dictionary: []
      2025-05-12 18:21:00.553961+00:00 [erro] <0.244.0>     trap_exit: true
      2025-05-12 18:21:00.553961+00:00 [erro] <0.244.0>     status: running
      2025-05-12 18:21:00.553961+00:00 [erro] <0.244.0>     heap_size: 987
      2025-05-12 18:21:00.553961+00:00 [erro] <0.244.0>     stack_size: 29
      2025-05-12 18:21:00.553961+00:00 [erro] <0.244.0>     reductions: 162
      2025-05-12 18:21:00.553961+00:00 [erro] <0.244.0>   neighbours:
      2025-05-12 18:21:00.553961+00:00 [erro] <0.244.0> 
      2025-05-12 18:21:00.560234+00:00 [noti] <0.44.0> Application rabbit exited with reason: {{cannot_log_to_file,"/var/log/rabbitmq/rabbit@rabbitmq-server-0.rabbitmq-nodes.openstack.log",eacces},{rabbit,start,[normal,[]]}}
      {"Kernel pid terminated",application_controller,"{application_start_failure,rabbit,{{cannot_log_to_file,\"/var/log/rabbitmq/rabbit@rabbitmq-server-0.rabbitmq-nodes.openstack.log\",eacces},{rabbit,start,[normal,[]]}}}"}
      Kernel pid terminated (application_controller) ({application_start_failure,rabbit,{{cannot_log_to_file,"/var/log/rabbitmq/rabbit@rabbitmq-server-0.rabbitmq-nodes.openstack.log",eacces},{rabbit,start,[normal,[]]}}})
      
      Crash dump is being written to: /var/log/rabbitmq/erl_crash.dump...
      

      I can see that config was translated properly to /etc/rabbitmq/conf.d/90-userDefinedConfiguration.conf inside the pod + parameter itself is already used. So it looks like the reason is not coming directly from this definition.

      Expected behavior
      It is expected that additionalConfig will cause such impact

      Bug impact
      Impossible to customize rabbitmq configuration

              rhn-support-lmiccini Luca Miccini
              rhn-support-astupnik Alex Stupnikov
              rhos-dfg-pidone
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Created:
                Updated:
                Resolved: