Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-77056

[4.19] External Certificates Loading for Routes is Very Slow

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Normal Normal
    • None
    • 4.19.z
    • Networking / router
    • None
    • False
    • Hide

      None

      Show
      None
    • None
    • Critical
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Description of problem:

      Routes using externally managed certificates load these certificates pretty slowly, about 100ms for the secret manager to get a secret fully loaded, and about 100ms for it to start the next route secret. We don't specify a specific upper limit for these, so I have a customer that is trying to load about ~2000 with decent etcd and API latency on one cluster, and the amount of time to finish takes a very high amount of time (~20 minutes sometimes whereas without the external secrets it finishes under the 120s startup probe).
      
      [1] https://docs.redhat.com/en/documentation/openshift_container_platform/4.19/html/ingress_and_load_balancing/routes#nw-ingress-route-secret-load-external-cert_creating-advanced-routes

      Version-Release number of selected component (if applicable):

      4.19

      How reproducible:

      Customer can reproduce it frequently, though some of their clusters introduce etcd latency that makes it more expected, they've reproduced with low etcd latency and the timing makes no sense.

      Steps to Reproduce:

          1. Create a high number of external certificate routes (~2000)
          2. Restart the router pods
          

      Actual results:

      Time taken to load all routes and secrets is exponentially higher than if we just loaded the routes without external secrets (it will of course add time but it's a disproportionate amount)

      Expected results:

      Time taken to load all routes and secrets should be only proportionally higher than loading the routes without external secrets

      Additional info:

          

              btofelrh Brett Tofel
              rhn-support-jclarkmu Jade Clark-Muth
              None
              None
              Hongan Li Hongan Li
              None
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Created:
                Updated: