Uploaded image for project: 'Knative Serving'
  1. Knative Serving
  2. SRVKS-661

sporadic "unexpected EOF" reading responses from a ksvc

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Minor Minor
    • 1.32.0
    • 1.10.0, 1.11.0, 1.21.0, 1.22.0
    • None
    • None
    • False
    • False
    • Undefined

      Having a simple go HTTP server responding with ~32kB responses, deployed as a ksvc with forced 1 replica

              autoscaling.knative.dev/maxScale: "1"
              autoscaling.knative.dev/minScale: "1"
      

      and default containerConcurrency (=0)

      and a client deployment connecting to the ksvc via its local http://<name>.<namespace>.svc.cluster.local

      The client has 32 parallel goroutines, each issuing repeatedly with 10ms sleep between requests (max. 100requests/s each)

      (The client limits its max connections via MaxIdleConnsPerHost = 100)

      The client sporadically* (*once per ~300.000 requests in my current run) sees "unexpected EOF" errors reading the response body.
      At the time of EOFs, the following errors sometimes appear in server's queue-proxy logs:

      httputil: ReverseProxy read error during body copy: read tcp 127.0.0.1:50158->127.0.0.1:8080: use of closed network connection
      

      And sometimes these in activator logs:

      httputil: ReverseProxy read error during body copy: unexpected EOF
      

      Deploying the same images as ordinary k8s Deployments don't seem to manifest any such errors.

      Reproducer: https://github.com/maschmid/queue-proxy-use-of-closed-network-connection-error

              skontopo@redhat.com Stavros Kontopoulos
              maschmid@redhat.com Marek Schmidt
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

                Created:
                Updated:
                Resolved: