Uploaded image for project: 'apiman (API Management)'
  1. apiman (API Management)
  2. APIMAN-1231

Stream closed / Broken Pipe issues with custom plugin

    XMLWordPrintable

Details

    Description

      Good afternoon,

      I’m having issues with on a production API which has started to receive high volumes of traffic. At peak volume times I see many exceptions being thrown in the logs. The stack is indicating that an error occurred in the policy chain, but when it tries to write the error to the response the connection has been closed. There seems to be a couple of flavors of IOException (broken pipe or stream closed from undertow). My plugin is calling chain.doApply(request) when it succeeds but the stack trace is indicating there is then an exception being caught in doApply(Chain.java:153). I am on version 1.2.2-Final.

      2017-01-05 18:28:49,095 ERROR [stderr] (default task-17) java.io.IOException: UT010029: Stream is closed
2017-01-05 18:28:49,095 ERROR [stderr] (default task-17)        at io.undertow.servlet.spec.ServletOutputStreamImpl.write(ServletOutputStreamImpl.java:136)
2017-01-05 18:28:49,095 ERROR [stderr] (default task-17)        at io.undertow.servlet.spec.ServletOutputStreamImpl.write(ServletOutputStreamImpl.java:128)
2017-01-05 18:28:49,095 ERROR [stderr] (default task-17)        at io.apiman.gateway.platforms.servlet.GatewayServlet$4.write(GatewayServlet.java:406)
2017-01-05 18:28:49,095 ERROR [stderr] (default task-17)        at io.apiman.gateway.platforms.servlet.GatewayServlet$4.write(GatewayServlet.java:395)
2017-01-05 18:28:49,095 ERROR [stderr] (default task-17)        at io.apiman.gateway.engine.impl.DefaultPolicyErrorWriter.write(DefaultPolicyErrorWriter.java:87)
2017-01-05 18:28:49,095 ERROR [stderr] (default task-17)        at io.apiman.gateway.platforms.servlet.GatewayServlet.writeError(GatewayServlet.java:392)
2017-01-05 18:28:49,095 ERROR [stderr] (default task-17)        at io.apiman.gateway.platforms.servlet.GatewayServlet$1.handle(GatewayServlet.java:210)
2017-01-05 18:28:49,095 ERROR [stderr] (default task-17)        at io.apiman.gateway.platforms.servlet.GatewayServlet$1.handle(GatewayServlet.java:157)
2017-01-05 18:28:49,095 ERROR [stderr] (default task-17)        at io.apiman.gateway.engine.impl.ApiRequestExecutorImpl.lambda$wrapResultHandler$0(ApiRequestExecutorImpl.java:159)
2017-01-05 18:28:49,095 ERROR [stderr] (default task-17)        at io.apiman.gateway.engine.impl.ApiRequestExecutorImpl.lambda$createPolicyErrorHandler$17(ApiRequestExecutorImpl.java:614)
2017-01-05 18:28:49,095 ERROR [stderr] (default task-17)        at io.apiman.gateway.engine.policy.Chain.throwError(Chain.java:249)
2017-01-05 18:28:49,095 ERROR [stderr] (default task-17)        at io.apiman.gateway.engine.policy.Chain.doApply(Chain.java:153)
2017-01-05 18:28:49,096 ERROR [stderr] (default task-17)        at com.markit.ondemand.auth.apiman.plugins.pingfed_oauth_policy.PingFedOauthPolicy.doSuccess(PingFedOauthPolicy.java:114)
2017-01-05 18:28:49,096 ERROR [stderr] (default task-17)        at com.markit.ondemand.auth.apiman.plugins.pingfed_oauth_policy.PingFedOauthPolicy.access$100(PingFedOauthPolicy.java:38)
2017-01-05 18:28:49,096 ERROR [stderr] (default task-17)        at com.markit.ondemand.auth.apiman.plugins.pingfed_oauth_policy.PingFedOauthPolicy$2.handle(PingFedOauthPolicy.java:193)
2017-01-05 18:28:49,096 ERROR [stderr] (default task-17)        at com.markit.ondemand.auth.apiman.plugins.pingfed_oauth_policy.PingFedOauthPolicy$2.handle(PingFedOauthPolicy.java:174)
2017-01-05 18:28:49,096 ERROR [stderr] (default task-17)        at io.apiman.gateway.platforms.servlet.components.HttpClientRequestImpl.end(HttpClientRequestImpl.java:140)
2017-01-05 18:28:49,096 ERROR [stderr] (default task-17)        at com.markit.ondemand.auth.apiman.plugins.pingfed_oauth_policy.PingFedOauthPolicy.retrieveAccessTokenFromPing(PingFedOauthPolicy.java:232)
2017-01-05 18:28:49,096 ERROR [stderr] (default task-17)        at com.markit.ondemand.auth.apiman.plugins.pingfed_oauth_policy.PingFedOauthPolicy.retrieveAccessTokenFromPing(PingFedOauthPolicy.java:174)
2017-01-05 18:28:49,096 ERROR [stderr] (default task-17)        at com.markit.ondemand.auth.apiman.plugins.pingfed_oauth_policy.PingFedOauthPolicy.doApply(PingFedOauthPolicy.java:83)
2017-01-05 18:28:49,096 ERROR [stderr] (default task-17)        at com.markit.ondemand.auth.apiman.plugins.pingfed_oauth_policy.PingFedOauthPolicy.doApply(PingFedOauthPolicy.java:38)

      My plugin is making either an external request to ElasticSearch or an HTTP request using IHttpClientComponent and then calling chain.doApply or chain.doFailure from within the IAsyncResultHandler handle method. Is there a problem with that pattern?

      Any help you can provide would be great.
      Thanks,

      Attachments

        Activity

          People

            msavy_jira Marc Savy (Inactive)
            david.rush_jira David Rush (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: