Uploaded image for project: 'Red Hat Fuse'
  1. Red Hat Fuse
  2. ENTESB-10290

camel-netty4-http unable to handle large data streams

XMLWordPrintable

    • 0
    • 0% 0%
    • Todo
    • Fuse 7.5 Sprint 53 - Blockers

      An example of a very common use case is the need to upload/download big files to backend systems.
      Customers look for a solution that integrates with their backends for big data uploads/downloads.
      Camel Netty4 HTTP is Fuse's most performant HTTP component and is stream based.

      There are 4 streaming scenarios to consider for the Netty4 HTTP component:
      1) Consumer receives (request) large stream from client
      2) Consumer sends (response) large stream to client
      3) Producer sends (request) large stream to backend
      4) Producer receives (response) large stream from backend

      Currently the component is unable to perform the above operations.
      It appears Camel Netty4 HTTP is trying to read the entire stream causing out-of-memory errors.

      Find attached (tester.tar.gz) a SpringBoot based project able to reproduce the above scenarios.
      It includes 7 JUnits to test streaming capabilities from which the following test cases are failing:

      1) testCamelNetty4HttpConsumerStreamUpload
      2) testCamelNetty4HttpConsumerStreamDownload
      3) testCamelNetty4httpProducerStreamUpload
      4) testCamelNetty4httpProducerStreamDownload

      The other JUnits are included to help us understand the mechanics of streaming in Camel.
      The project relies on SpringBoot's embedded server to showcase a working solution without Netty4 HTTP.

      To reproduce the problems, execute:
      > mvn clean test

      or one at a time:
      > mvn clean test -Dtest=StreamTest#testCamelNetty4httpProducerStreamUpload

      For your reference, there's an existing JIRA ticket in Camel related to the problem:
      https://issues.apache.org/jira/browse/CAMEL-12983

        1. tester.tar.gz
          7 kB
        2. jira-netty.zip
          38 kB

              zfeng@redhat.com Zheng Feng
              bmesegue@redhat.com Bruno Meseguer
              Francesco Mongiardo Francesco Mongiardo
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

                Created:
                Updated:
                Resolved: