Uploaded image for project: 'Red Hat Fuse'
  1. Red Hat Fuse
  2. ENTESB-12208

FTP Component with 'streamDownload' mode seems unstable

    XMLWordPrintable

Details

    • Bug
    • Status: Done
    • Major
    • Resolution: Done
    • fuse-7.4-GA
    • fuse-7.7-GA
    • Camel, Fuse Standalone
    • None
    • fuse-7.7-build6
    • Hide

      I've tested the component using a public FTP server available for testing:

      • For small files (5 bytes) all seems to work.
      • Large files (tested with 150 MBytes) it fails

      1) using the public server connection details, upload a large file.

      2) then, use the following Camel route definition to reproduce the problem:
      <route>
      <from uri="ftp://ftp.dlptest.com?noop=true&streamDownload=true&binary=true&fileName=ftptest2.xml&password=bbCKucPzfr4b9YXUY7tvsNKyh&username=dlpuser@dlptest.com"/>
      <log message="got FTP file"/>
      <to uri="file:src/data/out"/>
      <log message="done"/>
      </route>

      Large files cause the following error message:
      17:07:19.170 [Camel (MyCamel) thread #1 - ftp://ftp.dlptest.com] WARN o.a.c.c.file.remote.FtpConsumer - Error processing file RemoteFile[ftptest2.xml] due to File operation failed: 500 Unknown command
      Cannot change directory to: /. Code: 500. Caused by: [org.apache.camel.component.file.GenericFileOperationFailedException - File operation failed: 500 Unknown command
      Cannot change directory to: /. Code: 500]
      org.apache.camel.component.file.GenericFileOperationFailedException: File operation failed: 500 Unknown command
      Cannot change directory to: /. Code: 500
      at org.apache.camel.component.file.remote.FtpOperations.doChangeDirectory(FtpOperations.java:885)
      at org.apache.camel.component.file.remote.FtpOperations.changeCurrentDirectory(FtpOperations.java:848)
      at org.apache.camel.component.file.remote.FtpOperations.retrieveFileToStreamInBody(FtpOperations.java:443)
      at org.apache.camel.component.file.remote.FtpOperations.retrieveFile(FtpOperations.java:371)
      at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:409)
      at org.apache.camel.component.file.remote.RemoteFileConsumer.processExchange(RemoteFileConsumer.java:132)
      at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:219)
      at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:183)
      at org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:174)
      at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:101)
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
      at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
      at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      at java.lang.Thread.run(Thread.java:748)

      Show
      I've tested the component using a public FTP server available for testing: https://dlptest.com/ftp-test/ For small files (5 bytes) all seems to work. Large files (tested with 150 MBytes) it fails 1) using the public server connection details, upload a large file. 2) then, use the following Camel route definition to reproduce the problem: <route> <from uri="ftp://ftp.dlptest.com?noop=true&streamDownload=true&binary=true&fileName=ftptest2.xml&password=bbCKucPzfr4b9YXUY7tvsNKyh&username=dlpuser@dlptest.com"/> <log message="got FTP file"/> <to uri="file:src/data/out"/> <log message="done"/> </route> Large files cause the following error message: 17:07:19.170 [Camel (MyCamel) thread #1 - ftp://ftp.dlptest.com] WARN o.a.c.c.file.remote.FtpConsumer - Error processing file RemoteFile [ftptest2.xml] due to File operation failed: 500 Unknown command Cannot change directory to: /. Code: 500. Caused by: [org.apache.camel.component.file.GenericFileOperationFailedException - File operation failed: 500 Unknown command Cannot change directory to: /. Code: 500] org.apache.camel.component.file.GenericFileOperationFailedException: File operation failed: 500 Unknown command Cannot change directory to: /. Code: 500 at org.apache.camel.component.file.remote.FtpOperations.doChangeDirectory(FtpOperations.java:885) at org.apache.camel.component.file.remote.FtpOperations.changeCurrentDirectory(FtpOperations.java:848) at org.apache.camel.component.file.remote.FtpOperations.retrieveFileToStreamInBody(FtpOperations.java:443) at org.apache.camel.component.file.remote.FtpOperations.retrieveFile(FtpOperations.java:371) at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:409) at org.apache.camel.component.file.remote.RemoteFileConsumer.processExchange(RemoteFileConsumer.java:132) at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:219) at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:183) at org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:174) at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:101) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)

    Description

      The FTP component seems to have 2 approaches to deal with very large files:
      1) use parameter 'localWorkDirectory' which copies the remote file to a temp folder first.
      2) use parameter streamDownload set to true.

      Approach 1) seems to work well, it first copy the file locally.

      Approach 2) seems to work well with small files, but the same process definition seems to fail for very large files.

      Attachments

        Issue Links

          Activity

            People

              jondruse@redhat.com Jiri Ondrusek
              bmesegue@redhat.com Bruno Meseguer
              Tomas Turek Tomas Turek
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: