Uploaded image for project: 'AMQ Broker'
  1. AMQ Broker
  2. ENTMQBR-8903

Make AMQ 7 liveness and readiness checks more robust

    XMLWordPrintable

Details

    • Story
    • Resolution: Unresolved
    • Undefined
    • None
    • None
    • operator
    • None
    • False
    • None
    • False

    Description

      WIth AMQ 7 on OpenShift, by default a broker is 'live' if a connection can be made to its console port, and 'ready' if a connection can be made to its acceptors. These tests fail to spot a number of error conditions, such as memory exhaustion.

      The checks are configurable, and we document how to configure them. For example, we explain in the documentation [1] how to run `artemis check node`.

      The problem with this approach is that it requires instantiating a JVM in the running broker pod. Even for a simple task, the JVM requires a non-trivial amount of memory. In fact, even the `readinessProbe.sh` script that the default readiness probe uses invokes `python`, which has the side-effect of needing to include a Python installation in the image.

      It would be better if robust checks could be included in the broker itself, that are compatible with the Kubernetes liveness/readiness mechanism. There are various tests that might be made using Jolokia, but Jolokia itself does not produce output in a format that Kubernetes can use, without additional parsing.

      Although the tests are configurable, the broker image does not include the tooling necessary to implement a check that is both thorough and lightweight. If – to take just one example – the image contained `mosquitto_rr`, the installer could define a message request/response test using MQTT on a private topic. This utility is tiny and uses minimal memory, but provides quite a robust test, with an output (exit value) that is directly compatible with Kubernetes.

      I would expect that other utilities might be useful as well.

      But it would be better, I think, if robust tests were implemented in the broker code itself, rather than relying on external utilities.

      [1] https://access.redhat.com/documentation/en-us/red_hat_amq_broker/7.11/html-single/deploying_amq_broker_on_openshift/index

       

      Attachments

        Activity

          People

            Unassigned Unassigned
            rhn-support-kboone Kevin Boone
            Votes:
            7 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated: