Uploaded image for project: 'RH-SSO'
  1. RH-SSO
  2. RHSSO-2363

RH SSO 7.6.2 image in OpenShift (sso76-openshift-rhel8:7.6-20) is failing the Liveness and Readiness Probes

XMLWordPrintable

    • False
    • None
    • False
    • Hide

      1. Configure the Volume the probes are complaining:

      $ oc set volume dc/sso --name sso-probe-netrc-volume --add --source='{"emptyDir":{"medium":"Memory"}}' --mount-path /mnt/rh-sso

      2. In my own local instance, it was also necessary to increase the timeoutSeconds: value from both Probes to 3 as they seem to be taking longer than 1 second to execute. This could also potentially be true for some customers:

      41s         Warning   Unhealthy                     pod/sso-2-tthdw                          Readiness probe failed: command timed out

      (This timeout value never necessary to be increased before)

      Show
      1. Configure the Volume the probes are complaining: $ oc set volume dc/sso --name sso-probe-netrc-volume --add --source= '{ "emptyDir" :{ "medium" : "Memory" }}' --mount-path /mnt/rh-sso 2. In my own local instance, it was also necessary to increase the timeoutSeconds: value from both Probes to 3 as they seem to be taking longer than 1 second to execute. This could also potentially be true for some customers: 41s Warning Unhealthy pod/sso-2-tthdw Readiness probe failed: command timed out (This timeout value never necessary to be increased before)
    • Hide

      Deploy a fresh new install from RH SSO 7.6 or trigger an image upgrade to sso76-openshift-rhel8:7.6-20 (latest).

      Show
      Deploy a fresh new install from RH SSO 7.6 or trigger an image upgrade to sso76-openshift-rhel8:7.6-20 (latest).

      After upgrading RH SSO to 7.6.2 in OpenShift (image upgrade from 7.6-17 to 7.6-20), the Liveness and Readiness Probes fails:

      33s         Warning   Unhealthy        pod/sso-1-v9bdw    Readiness probe failed: INFO Creating a new DMR API probe netrc file.
ERROR Please mount a tmpfs volume at: '/mnt/rh-sso' path.
33s         Warning   Unhealthy        pod/sso-1-v9bdw    Liveness probe failed: INFO Creating a new DMR API probe netrc file.
ERROR Please mount a tmpfs volume at: '/mnt/rh-sso' path.
33s         Normal    Killing                    pod/sso-1-v9bdw                          Container sso failed liveness probe, will be restarted

      I was able to reproduce this issue locally and will attach the events.txt file from my own local instance, hence this Jira might be set as Public in order for the customers to follow-up any updates (since the data attached is from Red Hat only).

      This also happens when deploying a fresh new RH SSO 7.6.2 install.

      There is a workaround for it (further detailed in Workaround Description) but the impact is very high: It currently affects every customer deploying or upgrading RH SSO 7.6.2 as Templates and also most likely using the JDBC build as well, causing a full outage.

      Currently it doesn't seem to affect the RH SSO Operator.

        1. openshift-events.txt
          7 kB
        2. openshift-events.yaml
          56 kB

              rh_vmuzikar Václav Muzikář
              rhn-support-ekonecsn Estevao Konecsni
              Votes:
              4 Vote for this issue
              Watchers:
              19 Start watching this issue

                Created:
                Updated:
                Resolved: