Uploaded image for project: 'Red Hat build of Apache Camel'
  1. Red Hat build of Apache Camel
  2. RHBAC-7

camel-file - Can ant filter be optimized when using min/max depth

XMLWordPrintable

    • Icon: Story Story
    • Resolution: Duplicate
    • Icon: Major Major
    • None
    • None
    • None
    • None
    • False
    • Hide

      None

      Show
      None
    • False

      Using a combination of `recursive=true`, `antExclude=...`, and `readLockDeleteOrphanLockFiles=true` :

      from("file://inbox?delete=true&minDepth=1&maxDepth=1&readLock=changed&readLockCheckInterval=1000&readLockMinAge=1s&recursive=true&readLockDeleteOrphanLockFiles=true&antExclude=**/subdir/*").log("TEST2 TEST2 TEST2").to("file://outbox");
      

      With the following files present:

      $ tree inbox/
      inbox/
      ├── subdir
      │   ├── subsubdir
      │   │   ├── subsubsubdir
      │   │   │   └── test4.txt
      │   │   └── test3.txt
      │   └── test2.txt
      └── test.txt
      

      The following is observed on Camel start:

      13:19:47.655 [Camel (MyCamel) thread #1 - file://inbox] DEBUG o.a.c.c.f.s.MarkerFileExclusiveReadLockStrategy - Prepare on startup by deleting orphaned lock files from: inbox
      13:19:47.656 [Camel (MyCamel) thread #1 - file://inbox] TRACE o.a.c.c.f.AntPathMatcherFileFilter - Filtering file: subdir/subsubdir/subsubsubdir/test4.txt
      13:19:47.657 [Camel (MyCamel) thread #1 - file://inbox] TRACE o.a.c.c.f.AntPathMatcherFileFilter - Filtering file: subdir/subsubdir/test3.txt
      13:19:47.658 [Camel (MyCamel) thread #1 - file://inbox] TRACE o.a.c.c.f.AntPathMatcherFileFilter - Filtering file: subdir/test2.txt
      13:19:47.658 [Camel (MyCamel) thread #1 - file://inbox] TRACE o.a.c.c.f.AntPathMatcherFileFilter - File is excluded: subdir/test2.txt
      13:19:47.658 [Camel (MyCamel) thread #1 - file://inbox] TRACE o.a.c.c.f.AntPathMatcherFileFilter - Filtering file: test.txt
      

      In other words, all files/subfolders deeper than maxDepth=2 are scanned. This is a performance issue for customers that have a large number of files that need to be scanned on startup (can take hours).

      A workaround is to configure `readLockDeleteOrphanLockFiles=false`

              Unassigned Unassigned
              rhn-support-shiggs Stephen Higgs
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated:
                Resolved: