Uploaded image for project: 'Red Hat Fuse'
  1. Red Hat Fuse
  2. ENTESB-6053

Camel splitter doesn't process the payload of the file when received via FTP

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Done
    • Icon: Major Major
    • None
    • jboss-fuse-6.2.1
    • Camel
    • None
    • % %
    • Hide

      Use two routes, the first of which retrieves the file and the second of which splits it.

      Show
      Use two routes, the first of which retrieves the file and the second of which splits it.
    • Hide
      • unzip attached camel-example-ftp-camel-6231.zip
      • mvn compile exec:java -Pfile to execute file variant
      • setup ftp server on localhost, place src/main/resources/file_to_tokenize.txt in ftp://127.0.0.1:21/mypath
      • mvn compile exec:java -Pftp to execute ftp variant
      Show
      unzip attached camel-example-ftp-camel-6231.zip mvn compile exec:java -Pfile to execute file variant setup ftp server on localhost, place src/main/resources/file_to_tokenize.txt in ftp://127.0.0.1:21/mypath mvn compile exec:java -Pftp to execute ftp variant

      Comparing two simple routes, one using file and one using ftp we see different splitting behaviour:

      file variant:

              from("{{file.server}}")
	    .log("Copied file ${file:name} about to split")
	    .split(body().tokenize("\n"))
	    .to("log:org.apache.camel.camel-example-ftp-camel-6231?level=INFO&showAll=true&multiline=true")
           .to("file:target/download")
           .log("Processing file ${file:name} complete.");

      output snippet

      2016-09-02 15:33:49,155 [leServer.main()] INFO  DefaultCamelContext            - Route: route1 started and consuming from: Endpoint[file://target/files?fileName=file_to_tokenize.txt&noop=true]
2016-09-02 15:33:49,156 [leServer.main()] INFO  DefaultCamelContext            - Total 1 routes, of which 1 is started.
2016-09-02 15:33:49,157 [leServer.main()] INFO  DefaultCamelContext            - Apache Camel 2.15.1.redhat-621084 (CamelContext: camel-1) started in 0.320 seconds
2016-09-02 15:33:50,175 [://target/files] INFO  route1                         - Copied file file_to_tokenize.txt about to split
2016-09-02 15:33:50,189 [://target/files] INFO  camel-example-ftp-camel-6231   - Exchange[
, Id: ID-i7t450s-44436-1472839428587-0-4
, ExchangePattern: InOnly
, Properties: {CamelAggregationStrategy={Splitter[on: tokenize(body, 
) to: Pipeline[[Channel[sendTo(Endpoint[log://org.apache.camel.camel-example-ftp-camel-6231?level=INFO&multiline=true&showAll=true])], Channel[sendTo(Endpoint[file://target/download])], Channel[Log(route1)[Processing file ${file:name} comp
lete.]]]] aggregate: null]=UseOriginalAggregationStrategy}, CamelBatchComplete=true, CamelBatchIndex=0, CamelBatchSize=1, CamelCorrelationId=ID-i7t450s-44436-1472839428587-0-2, CamelCreatedTimestamp=Fri Sep 02 15:33:50 NDT 2016, CamelFileE
xchangeFile=GenericFile[file_to_tokenize.txt], CamelFileLockFileAcquired=true, CamelFileLockFileName=/home/rkieley/LocalProjects/cases/01694861/jboss-fuse-6.2.1.redhat-084/extras/apache-camel-2.15.1.redhat-621084/examples/camel-example-ftp
-camel-6231/target/files/file_to_tokenize.txt.camelLock, CamelMessageHistory=[DefaultMessageHistory[routeId=route1, node=to1]], CamelSplitComplete=false, CamelSplitIndex=0, CamelSplitSize=4, CamelStreamCacheUnitOfWork=DefaultUnitOfWork, Ca
melToEndpoint=log://org.apache.camel.camel-example-ftp-camel-6231?level=INFO&multiline=true&showAll=true}
, Headers: {breadcrumbId=ID-i7t450s-44436-1472839428587-0-1, CamelFileAbsolute=false, CamelFileAbsolutePath=/home/rkieley/LocalProjects/cases/01694861/jboss-fuse-6.2.1.redhat-084/extras/apache-camel-2.15.1.redhat-621084/examples/camel-exam
ple-ftp-camel-6231/target/files/file_to_tokenize.txt, CamelFileContentType=text/plain, CamelFileLastModified=1472839410000, CamelFileLength=24, CamelFileName=file_to_tokenize.txt, CamelFileNameConsumed=file_to_tokenize.txt, CamelFileNameOn
ly=file_to_tokenize.txt, CamelFileParent=target/files, CamelFilePath=target/files/file_to_tokenize.txt, CamelFileRelativePath=file_to_tokenize.txt}
, BodyType: String
, Body: line0
, Out: null: 
]
2016-09-02 15:33:50,203 [://target/files] INFO  route1                         - Processing file file_to_tokenize.txt complete.
2016-09-02 15:33:50,204 [://target/files] INFO  camel-example-ftp-camel-6231   - Exchange[
, Id: ID-i7t450s-44436-1472839428587-0-6
, ExchangePattern: InOnly
, Properties: {CamelAggregationStrategy={Splitter[on: tokenize(body, 
) to: Pipeline[[Channel[sendTo(Endpoint[log://org.apache.camel.camel-example-ftp-camel-6231?level=INFO&multiline=true&showAll=true])], Channel[sendTo(Endpoint[file://target/download])], Channel[Log(route1)[Processing file ${file:name} comp
lete.]]]] aggregate: null]=UseOriginalAggregationStrategy}, CamelBatchComplete=true, CamelBatchIndex=0, CamelBatchSize=1, CamelCorrelationId=ID-i7t450s-44436-1472839428587-0-2, CamelCreatedTimestamp=Fri Sep 02 15:33:50 NDT 2016, CamelFileE
xchangeFile=GenericFile[file_to_tokenize.txt], CamelFileLockFileAcquired=true, CamelFileLockFileName=/home/rkieley/LocalProjects/cases/01694861/jboss-fuse-6.2.1.redhat-084/extras/apache-camel-2.15.1.redhat-621084/examples/camel-example-ftp
-camel-6231/target/files/file_to_tokenize.txt.camelLock, CamelMessageHistory=[DefaultMessageHistory[routeId=route1, node=to1]], CamelSplitComplete=false, CamelSplitIndex=1, CamelSplitSize=4, CamelStreamCacheUnitOfWork=DefaultUnitOfWork, Ca
melToEndpoint=log://org.apache.camel.camel-example-ftp-camel-6231?level=INFO&multiline=true&showAll=true}
, Headers: {breadcrumbId=ID-i7t450s-44436-1472839428587-0-1, CamelFileAbsolute=false, CamelFileAbsolutePath=/home/rkieley/LocalProjects/cases/01694861/jboss-fuse-6.2.1.redhat-084/extras/apache-camel-2.15.1.redhat-621084/examples/camel-exam
ple-ftp-camel-6231/target/files/file_to_tokenize.txt, CamelFileContentType=text/plain, CamelFileLastModified=1472839410000, CamelFileLength=24, CamelFileName=file_to_tokenize.txt, CamelFileNameConsumed=file_to_tokenize.txt, CamelFileNameOn
ly=file_to_tokenize.txt, CamelFileParent=target/files, CamelFilePath=target/files/file_to_tokenize.txt, CamelFileRelativePath=file_to_tokenize.txt}
, BodyType: String
, Body: line1
, Out: null: 
]
2016-09-02 15:33:50,214 [://target/files] INFO  route1                         - Processing file file_to_tokenize.txt complete.
.
.
.

      ftp variant:

              from("{{ftp.server}}")
	    .log("Received file ${file:name} about to split")
	    .split(body().tokenize("\n"))
	    .to("log:org.apache.camel.camel-example-ftp?level=INFO&showAll=true&multiline=true")
           .to("file:target/download")
           .log("Downloaded file ${file:name} complete.");

      output snippet:

      2016-09-02 15:34:02,705 [tpServer.main()] INFO  DefaultCamelContext            - Route: route1 started and consuming from: Endpoint[ftp://127.0.0.1:21/mypath?delay=2s&fileName=file_to_tokenize.txt&move=.done&password=xxxxxx&username=rhel-l
iveuser]
2016-09-02 15:34:02,705 [tpServer.main()] INFO  DefaultCamelContext            - Total 1 routes, of which 1 is started.
2016-09-02 15:34:02,707 [tpServer.main()] INFO  DefaultCamelContext            - Apache Camel 2.15.1.redhat-621084 (CamelContext: camel-1) started in 0.373 seconds
2016-09-02 15:34:03,732 [0.0.1:21/mypath] INFO  route1                         - Received file file_to_tokenize.txt about to split
2016-09-02 15:34:03,745 [0.0.1:21/mypath] INFO  camel-example-ftp              - Exchange[
, Id: ID-i7t450s-46125-1472839442093-0-4
, ExchangePattern: InOnly
, Properties: {CamelAggregationStrategy={Splitter[on: tokenize(body, 
) to: Pipeline[[Channel[sendTo(Endpoint[log://org.apache.camel.camel-example-ftp?level=INFO&multiline=true&showAll=true])], Channel[sendTo(Endpoint[file://target/download])], Channel[Log(route1)[Downloaded file ${file:name} complete.]]]] a
ggregate: null]=UseOriginalAggregationStrategy}, CamelBatchComplete=true, CamelBatchIndex=0, CamelBatchSize=1, CamelCorrelationId=ID-i7t450s-46125-1472839442093-0-2, CamelCreatedTimestamp=Fri Sep 02 15:34:03 NDT 2016, CamelFileExchangeFile
=RemoteFile[file_to_tokenize.txt], CamelMessageHistory=[DefaultMessageHistory[routeId=route1, node=to1]], CamelSplitComplete=true, CamelSplitIndex=0, CamelSplitSize=1, CamelStreamCacheUnitOfWork=DefaultUnitOfWork, CamelToEndpoint=log://org
.apache.camel.camel-example-ftp?level=INFO&multiline=true&showAll=true}
, Headers: {breadcrumbId=ID-i7t450s-46125-1472839442093-0-1, CamelFileAbsolute=false, CamelFileAbsolutePath=mypath/file_to_tokenize.txt, CamelFileHost=127.0.0.1, CamelFileLastModified=1472848380000, CamelFileLength=24, CamelFileName=file_t
o_tokenize.txt, CamelFileNameConsumed=file_to_tokenize.txt, CamelFileNameOnly=file_to_tokenize.txt, CamelFileParent=mypath, CamelFilePath=mypath/file_to_tokenize.txt, CamelFileRelativePath=file_to_tokenize.txt, CamelFtpReplyCode=226, Camel
FtpReplyString=226 Transfer complete
}
, BodyType: String
, Body: -rw-rw-r--   1 rhel-liveuser rhel-liveuser       24 Sep  2 18:03 file_to_tokenize.txt
, Out: null: 
]
2016-09-02 15:34:03,759 [0.0.1:21/mypath] INFO  route1                         - Downloaded file file_to_tokenize.txt complete.

      See attached for reproducer. Appears similar to CAMEL-6231 but without the obvious multi-threading/parallel processing.

        1. camel-example-ftp-camel-6231.zip
          53 kB

            pantinor@redhat.com Paolo Antinori
            rhn-support-rkieley Roderick Kieley
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: