Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-2499

ReadinessProbes failing after upgrade to OpenShift 4.10.25

XMLWordPrintable

    • Quality / Stability / Reliability
    • False
    • Hide

      None

      Show
      None
    • None
    • Moderate
    • None
    • None
    • None
    • None
    • None
    • Customer Escalated
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Description of problem:

      After upgrading from OpenShift 4.10.17 to 4.10.25 customer hitting several failed liveness and readiness probes
      

      Version-Release number of selected component (if applicable):

      4.10.25
      

      How reproducible:

      Not reproducible on other cluster, only one of many cluster is affected
      

      Steps to Reproduce:

      1.
      2.
      3.
      

      Actual results:

      Customer is hitting high connection time in liveness and readiness probes which causing NotReady Pods and redeployments.
      
      oc rsh prometheus-k8s-0 curl -w @- -o /dev/null -s "http://localhost:9090/-/ready" <<'EOF'
      >     time_namelookup:  %{time_namelookup}\n
      >        time_connect:  %{time_connect}\n
      >     time_appconnect:  %{time_appconnect}\n
      >    time_pretransfer:  %{time_pretransfer}\n
      >       time_edirect:  %{time_edirect}\n
      >  time_starttransfer:  %{time_starttransfer}\n
      >                     ----------\n
      >          time_total:  %{time_total}\n
      > EOF
          time_namelookup:  0.078855
             time_connect:  7.181930
          time_appconnect:  0.000000
         time_pretransfer:  7.181973
            time_edirect:  0.000000
       time_starttransfer:  7.182554
                          ----------
               time_total:  7.182616
      

      Expected results:

      No delay in connection to loopback interface
      

      Additional info:

      prometheus dump, must-gather, sosreports and further metrics attached to support case
      

              mskrivan@redhat.com Michal Skrivanek
              rhn-support-anowak Andreas Nowak
              None
              None
              Sunil Choudhary Sunil Choudhary
              None
              Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

                Created:
                Updated:
                Resolved: