Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-29373

http/2 connection coalescing component routing broken with single certificate

XMLWordPrintable

    • Critical
    • No
    • 3
    • Sprint 253
    • 1
    • Rejected
    • False
    • Hide

      None

      Show
      None
    • Hide
      * Previously, if multiple routes were specified in the same certificate or if a route specified the default certificate as a custom certificate, and HTTP/2 was enabled on the router, an HTTP/2 client could perform connection coalescing on routes. Clients, such as a web browser, could re-use connections and potentially connect to the wrong backend server. With this release, the {product-title} router now checks when the same certificate is specified on more than one route or when a route specifies the default certificate as a custom certificate. When either one of these conditions is detected, the router configures the HAProxy load balancer so to not allow HTTP/2 client connections to any routes that use these certificate.(link:https://issues.redhat.com/browse/OCPBUGS-29373[*OCPBUGS-29373*])
      Show
      * Previously, if multiple routes were specified in the same certificate or if a route specified the default certificate as a custom certificate, and HTTP/2 was enabled on the router, an HTTP/2 client could perform connection coalescing on routes. Clients, such as a web browser, could re-use connections and potentially connect to the wrong backend server. With this release, the {product-title} router now checks when the same certificate is specified on more than one route or when a route specifies the default certificate as a custom certificate. When either one of these conditions is detected, the router configures the HAProxy load balancer so to not allow HTTP/2 client connections to any routes that use these certificate.(link: https://issues.redhat.com/browse/OCPBUGS-29373 [* OCPBUGS-29373 *])
    • Bug Fix
    • Done

      Description of problem:

      Due to HTTP/2 Connection Coalescing (https://daniel.haxx.se/blog/2016/08/18/http2-connection-coalescing/), routes which use the same certificate can present unexplained 503 errors when attempting to access an HTTP/2 enabled ingress controller.

      It appears that HAProxy supports the ability to force HTTP 1.1 on a route-by-route basis, but our Ingress Controller does not expose that option.

      This is especially problematic for component routes because generally speaking, customers use a wildcard or SAN to deploy custom component routes (console, OAuth, downloads), but with HTTP/2, this does not work properly.

      To address this issue, we're proposing the creation of an annotation haproxy.router.openshift.io/http2-disable, which will allow the disabling of HTTP/2 on a route-by-route basis, or smarter logic built into our Ingress operator to handle this situation.  

      Version-Release number of selected component (if applicable):

       OpenShift 4.14

      How reproducible:

      Serve routes to applications in Openshift.
      Observe the routes through a HTTP/2 enabled client.
      Notice that http/2 client connections are broken (returns 503 on second connection when using same certificates across a mix of re-encrypt and passthrough routes)

      Steps to Reproduce:

      (see above notes)

      Actual results:

      503 error    

      Expected results:

      no error    

      Additional info:

          

            mmasters1@redhat.com Miciah Masters
            rh-ee-adejong Aaren de Jong
            Shudi Li Shudi Li
            Votes:
            4 Vote for this issue
            Watchers:
            17 Start watching this issue

              Created:
              Updated: