Uploaded image for project: 'Red Hat 3scale API Management'
  1. Red Hat 3scale API Management
  2. THREESCALE-1781

Can't proxy POST requests to HTTPS API backend via forward proxy

    XMLWordPrintable

Details

    Description

      Environment:

      • APIcast v3.4 (3scale v2.4)
      • Forward HTTP proxy

      Behavior:

      • GET requests are forwarded successfully when both HTTP and HTTPS API backends are used.
      • POST requests work OK when the API backend is HTTP.
      • POST requests fail (either client or server timeout – depending on the setup) when the API backend is HTTPS.

      Steps to reproduce:

      1. Unzip the attached archive and start the environment with:

      docker-compose up -d

      2. Confirm that GET requests and POST to HTTP backend works OK:

      curl -v -k "https://localhost:8843/hello" -H "user-key: key"
      curl -v -k "https://localhost:8843/secure/hello" -H "user-key: key"
      curl -v -k -X POST "https://localhost:8843/hello" -H "user-key: key" -d '{"hello":"world"}'
      

      Note: there is an Upstream policy configured in APIcast that routes requests with /secure in path to a HTTPS backend.

      3. Try to make a POST request to a secure API backend:

      curl -v -k -X POST "https://localhost:8843/secure/hello" -H "user-key: key" -d '{"hello":"world"}'
      

      After some time (10 seconds, as client_body_timeout 10s; is set on the API backend – the behavior with longer timeouts is the same though) the client receives the following error:

      < HTTP/1.1 502 Bad Gateway
      < Server: openresty/1.13.6.2
      < Date: Fri, 11 Jan 2019 10:10:32 GMT
      < Content-Type: text/html
      < Content-Length: 179
      < Connection: keep-alive
      <
      <html>
      <head><title>502 Bad Gateway</title></head>
      <body bgcolor="white">
      <center><h1>502 Bad Gateway</h1></center>
      <hr><center>openresty/1.13.6.2</center>
      </body>
      </html>
      

      Relevant logs:

      2019/01/11 10:09:32 [debug] 27#27: *24 [lua] http.lua:50: connect(): connected to  ip:172.21.0.4 host: proxy.foo port: 3128 ok: 1 err: nil
      2019/01/11 10:09:32 [debug] 27#27: *24 [lua] proxy.lua:82: new(): connection to proxy.foo:3128 established, pool: proxy.foo:3128:echo.foo:8443 reused times: 1
      2019/01/11 10:09:32 [debug] 27#27: *24 [lua] http.lua:633: send_request():
      POST /secure/hello HTTP/1.1
      User-Agent: curl/7.60.0
      user-key: key
      Accept: */*
      Content-Type: application/x-www-form-urlencoded
      Content-Length: 17
      Host: echo.foo:8443
      
      
      2019/01/11 10:10:32 [error] 27#27: *24 lua tcp socket read timed out, client: 172.21.0.1, server: _, request: "POST /secure/hello HTTP/1.1", host: "echo.foo:8443"
      2019/01/11 10:10:32 [error] 27#27: *24 [lua] http_proxy.lua:111: forward_https_request(): failed to proxy request to: http://proxy.foo:3128 err : timeout, client: 172.21.0.1, server: _, request: "POST /secure/hello HTTP/1.1", host: "echo.foo:8443"
      

      So the 502 is apparently returned by https://github.com/3scale/apicast/blob/v3.4.0/gateway/src/apicast/http_proxy.lua#L112.

      The proxy is Squid, check squid/squid.conf for config.

      The docker-compose file also includes a tcpdump container that captures the packets.

      Note that the direct curl requests through proxy are successful (called from within docker network, for correct name resolution):

      curl -v -k -X POST -x http://proxy.foo:3128 "https://echo.foo:8443/secure/hello" -H "user-key: key" -d '{"hello":"world"}'
      

      which indicates that there is some problem with APIcast <> Forward Proxy communication.

      Attachments

        Activity

          People

            Unassigned Unassigned
            rhn-support-dmayorov Daria Mayorova
            David Ortiz David Ortiz (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: