Uploaded image for project: 'Red Hat Fuse'
  1. Red Hat Fuse
  2. ENTESB-10290

camel-netty4-http unable to handle large data streams

    XMLWordPrintable

Details

    • % %
    • fuse-7.7-build1
    • -
    • Fuse 7.5 Sprint 53 - Blockers

    Description

      An example of a very common use case is the need to upload/download big files to backend systems.
      Customers look for a solution that integrates with their backends for big data uploads/downloads.
      Camel Netty4 HTTP is Fuse's most performant HTTP component and is stream based.

      There are 4 streaming scenarios to consider for the Netty4 HTTP component:
      1) Consumer receives (request) large stream from client
      2) Consumer sends (response) large stream to client
      3) Producer sends (request) large stream to backend
      4) Producer receives (response) large stream from backend

      Currently the component is unable to perform the above operations.
      It appears Camel Netty4 HTTP is trying to read the entire stream causing out-of-memory errors.

      Find attached (tester.tar.gz) a SpringBoot based project able to reproduce the above scenarios.
      It includes 7 JUnits to test streaming capabilities from which the following test cases are failing:

      1) testCamelNetty4HttpConsumerStreamUpload
      2) testCamelNetty4HttpConsumerStreamDownload
      3) testCamelNetty4httpProducerStreamUpload
      4) testCamelNetty4httpProducerStreamDownload

      The other JUnits are included to help us understand the mechanics of streaming in Camel.
      The project relies on SpringBoot's embedded server to showcase a working solution without Netty4 HTTP.

      To reproduce the problems, execute:
      > mvn clean test

      or one at a time:
      > mvn clean test -Dtest=StreamTest#testCamelNetty4httpProducerStreamUpload

      For your reference, there's an existing JIRA ticket in Camel related to the problem:
      https://issues.apache.org/jira/browse/CAMEL-12983

      Attachments

        1. tester.tar.gz
          7 kB
        2. jira-netty.zip
          38 kB

        Activity

          People

            zfeng@redhat.com Zheng Feng
            bmesegue@redhat.com Bruno Meseguer
            Francesco Mongiardo Francesco Mongiardo
            Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: