Uploaded image for project: 'OpenShift Bugs'
  1. OpenShift Bugs
  2. OCPBUGS-65482

OCP4: Haproxy does not handle port being set in Host header, and fails to redirect properly

XMLWordPrintable

    • Quality / Stability / Reliability
    • False
    • Hide

      None

      Show
      None
    • None
    • Moderate
    • None
    • None
    • None
    • Rejected
    • None
    • None
    • None
    • None
    • None
    • None
    • None
    • None

      Description of problem:

          An exception has been identified with the way HAProxy handles Host header parameters. If a port is enforced in the header, we will not translate the redirect properly, including the port in the redirect a second time and failing the lookup:
      
      curl -v -H'Host: httpd-ex-edge-sunbro.apps.shrocp4upi416ovn.lab.psi.pnq2.redhat.com:80' 'http://httpd-ex-edge-sunbro.apps.shrocp4upi416ovn.lab.psi.pnq2.redhat.com'
      * Connected to httpd-ex-edge-sunbro.apps.shrocp4upi416ovn.lab.psi.pnq2.redhat.com (10.74.209.170) port 80
      < HTTP/1.1 302 Found
      < location: https://httpd-ex-edge-sunbro.apps.shrocp4upi416ovn.lab.psi.pnq2.redhat.com:80/ ##<-- re-defines :80 on subsequent https:// call?? (will fail to connect to the correct listener on next request).
      
      compare to access.redhat.com:
      curl -kv -H'Host: access.redhat.com:80' 'http://access.redhat.com:80'
      * Connected to access.redhat.com (23.48.203.137) port 80
      < HTTP/1.1 301 Moved Permanently
      < Location: https://access.redhat.com/ ##<--- does not define :80 on subsequent call (will succeed protocol adjustment at new port)
      
      
      
      # Edge entry (:443)
      sh-5.1$ cat os_edge_reencrypt_be.map | grep httpd-ex 
      ^httpd-ex-edge-sunbro\.apps\.shrocp4upi416ovn\.lab\.psi\.pnq2\.redhat\.com\.?(:[0-9]+)?(/.*)?$ be_edge_http:sunbro:httpd-ex-edge
      
      # Redirect rule enabled for route:
      sh-5.1$ cat os_route_http_redirect.map | grep httpd-ex
      ^httpd-ex-edge-sunbro\.apps\.shrocp4upi416ovn\.lab\.psi\.pnq2\.redhat\.com\.?(:[0-9]+)?(/.*)?$ 1 
      
      
      
      #curl:
      date; curl -L -H'Host: httpd-ex-edge-sunbro.apps.shrocp4upi416ovn.lab.psi.pnq2.redhat.com:80' 'http://httpd-ex-edge-sunbro.apps.shrocp4upi416ovn.lab.psi.pnq2.redhat.com:80/'
      Tue Nov 11 01:12:51 PM EST 2025
      * Host httpd-ex-edge-sunbro.apps.shrocp4upi416ovn.lab.psi.pnq2.redhat.com:80 was resolved.
      * IPv6: (none)
      * IPv4: 10.74.209.170
      *   Trying 10.74.209.170:80...
      * Connected to httpd-ex-edge-sunbro.apps.shrocp4upi416ovn.lab.psi.pnq2.redhat.com (10.74.209.170) port 80
      * using HTTP/1.x
      > GET / HTTP/1.1
      > Host: httpd-ex-edge-sunbro.apps.shrocp4upi416ovn.lab.psi.pnq2.redhat.com:80
      > User-Agent: curl/8.11.1
      > Accept: */*
      > 
      * Request completely sent off
      < HTTP/1.1 302 Found ##<-------------------- bounced off port 80, redirected to use https.
      < content-length: 0
      < location: https://httpd-ex-edge-sunbro.apps.shrocp4upi416ovn.lab.psi.pnq2.redhat.com:80/ ##<-- proto changed, port did not!
      < cache-control: no-cache
      * Ignoring the response-body
      * setting size while ignoring
      < 
      * Connection #0 to host httpd-ex-edge-sunbro.apps.shrocp4upi416ovn.lab.psi.pnq2.redhat.com left intact
      * Clear auth, redirects scheme from http to https
      * Issue another request to this URL: 'https://httpd-ex-edge-sunbro.apps.shrocp4upi416ovn.lab.psi.pnq2.redhat.com:80/'
      * Hostname httpd-ex-edge-sunbro.apps.shrocp4upi416ovn.lab.psi.pnq2.redhat.com was found in DNS cache
      *   Trying 10.74.209.170:80... ##<------------ PORT DOES NOT CHANGE
      * ALPN: curl offers h2,http/1.1
      * TLSv1.3 (OUT), TLS handshake, Client hello (1):
      *  CAfile: /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem
      *  CApath: none
      * TLSv1.3 (OUT), TLS alert, record overflow (534):
      * TLS connect error: error:0A0000C6:SSL routines::packet length too long
      * closing connection #1
      curl: (35) TLS connect error: error:0A0000C6:SSL routines::packet length too long ##(ssl on unencrypted listener - expected) 
      
      
      
      #access log:
      2025-11-11T18:12:51.941148+00:00 worker-2 worker-2.shrocp4upi416ovn.lab.psi.pnq2.redhat.com haproxy[20]: 10.74.209.170:39508 [11/Nov/2025:18:12:51.941] public public/<NOSRV> 0/-1/-1/-1/0 302 159 - - LR-- 27/1/0/0/0 0/0 "GET / HTTP/1.1"
      
      
      
      ==========================
      
      Compare to un-enforced port call (port not specified by Host header)
      
      #curl:
      date; curl -L -H'Host: httpd-ex-edge-sunbro.apps.shrocp4upi416ovn.lab.psi.pnq2.redhat.com' 'http://httpd-ex-edge-sunbro.apps.shrocp4upi416ovn.lab.psi.pnq2.redhat.com:80/'
      Tue Nov 11 01:23:46 PM EST 2025
      * Host httpd-ex-edge-sunbro.apps.shrocp4upi416ovn.lab.psi.pnq2.redhat.com:80 was resolved.
      * IPv6: (none)
      * IPv4: 10.74.209.170
      *   Trying 10.74.209.170:80...
      * Connected to httpd-ex-edge-sunbro.apps.shrocp4upi416ovn.lab.psi.pnq2.redhat.com (10.74.209.170) port 80
      * using HTTP/1.x
      > GET / HTTP/1.1
      > Host: httpd-ex-edge-sunbro.apps.shrocp4upi416ovn.lab.psi.pnq2.redhat.com
      > User-Agent: curl/8.11.1
      > Accept: */*
      > 
      * Request completely sent off
      < HTTP/1.1 302 Found ##<---- bounced on port 80, (redirected to https)
      < content-length: 0
      < location: https://httpd-ex-edge-sunbro.apps.shrocp4upi416ovn.lab.psi.pnq2.redhat.com/ ##<----- Protocol changed
      < cache-control: no-cache
      * Ignoring the response-body
      * setting size while ignoring
      < 
      * Connection #0 to host httpd-ex-edge-sunbro.apps.shrocp4upi416ovn.lab.psi.pnq2.redhat.com left intact
      * Clear auth, redirects to port from 80 to 443
      * Issue another request to this URL: 'https://httpd-ex-edge-sunbro.apps.shrocp4upi416ovn.lab.psi.pnq2.redhat.com/'
      * Host httpd-ex-edge-sunbro.apps.shrocp4upi416ovn.lab.psi.pnq2.redhat.com:443 was resolved.
      * IPv6: (none)
      * IPv4: 10.74.209.170
      *   Trying 10.74.209.170:443... ##<------------------------- PORT CHANGED
      * ALPN: curl offers h2,http/1.1
      * TLSv1.3 (OUT), TLS handshake, Client hello (1):
      *  CAfile: /etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem
      *  CApath: none
      * TLSv1.3 (IN), TLS handshake, Server hello (2):
      * TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
      * TLSv1.3 (IN), TLS handshake, Certificate (11):
      * TLSv1.3 (IN), TLS handshake, CERT verify (15):
      * TLSv1.3 (IN), TLS handshake, Finished (20):
      * TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
      * TLSv1.3 (OUT), TLS handshake, Finished (20):
      * SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256 / x25519 / RSASSA-PSS
      * ALPN: server did not agree on a protocol. Uses default.
      * Server certificate:
      *  subject: C=US; ST=North Carolina; L=Raleigh; O=Red Hat; OU=Information Technology; emailAddress=resource-hub-dev@redhat.com; CN=api.shrocp4upi416ovn.lab.upshift.rdu2.redhat.com
      *  start date: Jul  9 14:59:35 2025 GMT
      *  expire date: Jul  4 14:59:35 2026 GMT
      *  subjectAltName: host "httpd-ex-edge-sunbro.apps.shrocp4upi416ovn.lab.psi.pnq2.redhat.com" matched cert's "*.apps.shrocp4upi416ovn.lab.psi.pnq2.redhat.com"
      *  issuer: O=Red Hat; OU=prod; CN=2023 Certificate Authority RHCSv2
      *  SSL certificate verify ok.
      *   Certificate level 0: Public key type RSA (4096/152 Bits/secBits), signed using sha256WithRSAEncryption
      *   Certificate level 1: Public key type RSA (4096/152 Bits/secBits), signed using sha384WithRSAEncryption
      *   Certificate level 2: Public key type RSA (4096/152 Bits/secBits), signed using sha384WithRSAEncryption
      * Connected to httpd-ex-edge-sunbro.apps.shrocp4upi416ovn.lab.psi.pnq2.redhat.com (10.74.209.170) port 443
      * using HTTP/1.x
      > GET / HTTP/1.1
      > Host: httpd-ex-edge-sunbro.apps.shrocp4upi416ovn.lab.psi.pnq2.redhat.com
      > User-Agent: curl/8.11.1
      > Accept: */*
      > 
      * Request completely sent off
      * TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
      * TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
      < HTTP/1.1 200 OK
      ...
      
      #accesslogs:
      2025-11-11T18:23:47.818629+00:00 worker-1 worker-1.shrocp4upi416ovn.lab.psi.pnq2.redhat.com haproxy[20]: 10.74.209.170:42248 [11/Nov/2025:18:23:47.815] fe_sni~ be_edge_http:sunbro:httpd-ex-edge/pod:httpd-ex-7c8ffb95b9-r6kj7:httpd-ex:8080-tcp:10.131.1.93:8080 0/0/2/1/3 200 37718 - - --NI 33/16/0/0/0 0/0 "GET / HTTP/1.1"

      Version-Release number of selected component (if applicable):

          4.16.45 replicated but I expect this can be replicated on all current builds of OpenShift

      How reproducible:

          every time

      Steps to Reproduce:

          1. Create edge route
          2. Enable redirect with insecureEdgeTerminationPolicy: Redirect
          3. Curl to the route, forcing Port selection in the HOST header: 
      curl -L -H'Host: httpd-ex-edge-sunbro.apps.shrocp4upi416ovn.lab.psi.pnq2.redhat.com:80' 'http://httpd-ex-edge-sunbro.apps.shrocp4upi416ovn.lab.psi.pnq2.redhat.com:80/'
      
      Observe (examples above) that the port is re-inserted in the redirect.
      
      
           

      Actual results:

          Redirect fails because there is an exception to our logic that does not handle this being a possibility:
      
      ~~~
        acl secure_redirect base,map_reg_int(/var/lib/haproxy/conf/os_route_http_redirect.map) -m bool
        redirect scheme https if secure_redirect
      ~~~
      
      https://github.com/openshift/router/blob/b41f9d05467fb7b3f6c2dafa6ac4b5e25164c0b6/images/router/haproxy/conf/haproxy-config.template#L261-L262
      
      https://www.rfc-editor.org/rfc/rfc2616#section-14.23 
      RFC allows this use-case however.

      Expected results:

          Redirect should not include port string, and allow https:// to resolve at 443 successfully, rather than shipping SSL content to unencrypted public listener + failing call. 

      Additional info:

         see first comment for discussion with engineering team + case details. 

              btofelrh Brett Tofel
              rhn-support-wrussell Will Russell
              None
              None
              Shudi Li Shudi Li
              None
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Created:
                Updated: