Uploaded image for project: 'Red Hat Developer Hub Bugs'
  1. Red Hat Developer Hub Bugs
  2. RHDHBUGS-2017

Helm Chart - rhdh-developer-hub pod fails its readiness probe (503)

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Duplicate
    • Icon: Undefined Undefined
    • None
    • 1.7.0, 1.7.1
    • Helm Chart
    • None
    • False
    • Hide

      None

      Show
      None
    • False
    • Important

      Bug Summary

      After upgrading from RHDH 1.6 to 1.7 and reinstalling RHDH via Helm with Orchestrator enabled, the rhdh-developer-hub pod fails its readiness probe.


      Symptoms

      • Pod status: Running, but not Ready.
      • backstage-backend container shows repeated log messages:
         
        Plugin initialization in progress, still initializing:'healthcheck', 'app', 'catalog', 'proxy', ...
      • Kubernetes reports:
         
        Readiness probe failed: HTTP probe failed with statuscode: 503

      Environment

      • Namespace: rhdh-chart
      • Image: quay.io/rhdh/rhdh-hub-rhel9:latest
      • Postgres host: rhdh-postgresql (auto-provisioned by chart)

       

      Cause:
      Missing Network policies – orchestrator plugin requires some network policies to allow communication between the sonataflow components and rhdh. 

      In an upgrade scenario, the user will be reusing the Openshift Serverless Logic and Openshift Serverless Installations they have from the previous set up.  Because they already have it, there is no need for the Backstage chart to install the sonataflow resources again. Therfore, the current logic is flawed [here](https://github.com/redhat-developer/rhdh-chart/blob/main/charts/backstage/templates/network-policies.yaml). 

      When installing the Helm Chart with these values:
         ```yaml
         orchestrator:
           enabled: true
           serverlessLogicOperator:
             enabled: false
           serverlessOperator:
             enabled: false # optional
         ```
      The Helm Chart has a flow control to not install the network policies. These are the fault and should be fixed to allow the installation of the network policies, even if the serverlessLogicOperator and serverlessOperator values are false. 
      **

      Workaround:

      manually add the network policies to the rhdh namespace.

      Prerequisites (if any, like setup, operators/versions):{}{}{}

      Steps to Reproduce

      1. Install Orchestrator Operator v1.6 
      2. Follow the [upgrade guide](https://github.com/elai-shalev/rhdhorchestrator.github.io/blob/main/content/blog/upgrading-orchestrator-1.7.md) steps to upgrade to RHDH v1.7 via the helm chart.
      3. Mae sure to perserve te old sonataflow resources and follow the steps in the upgrade guide. 
      3. Delete rhdh v1.6 
      4. Install rhdh v1.7 per the upgrade guide via the helm chart

      Actual results:

      Backstage pod fails to finish, getting a 503 error

      Expected results:

      Backstage pod is OK{}

      Reproducibility (Always/Intermittent/Only Once):

      100%

      Build Details:

      Additional info (Such as Logs, Screenshots, etc):

              Unassigned Unassigned
              rh-ee-eshalev Elai Shalev
              RHIDP - Install
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated:
                Resolved: