Uploaded image for project: 'Teiid'
  1. Teiid
  2. TEIID-6019

Normalize and expand file api and file glob search

    XMLWordPrintable

Details

    • Enhancement
    • Resolution: Done
    • Major
    • 16.0
    • None
    • Misc. Connectors
    • None
    • DV Sprint 66
    • 1
    • Hide
      "exception on file not found" will also happen on the return of an empty array - as we cannot quickly differentiate between a situation where the path is referring to a parent directory that does not exist and one where no files in a given directory match. At worst this may mean some users will need to update their file translator properties.
      Show
      "exception on file not found" will also happen on the return of an empty array - as we cannot quickly differentiate between a situation where the path is referring to a parent directory that does not exist and one where no files in a given directory match. At worst this may mean some users will need to update their file translator properties.

    Description

      The file sources need to have consistent implementation of getFiles and should provide additional information such as the full file path and if it's a directory. For example currently it's possible to return directories in the file source if you specify a non-glob directory path, but that information is not exposed.

      The glob paths should all support multiple, not just a single * in the file name.

      At a first glance we may not implement these changes for ftp, but just throw appropriate exceptions - it currently only knows about returning a single file. It would be possible to eventually add logic for that though as it supports similar file visiting logic as Java.

      So the focus will be on file, hdfs, and s3 - which invinci_1 is already looking into.

      Attachments

        Issue Links

          There are no Sub-Tasks for this issue.

          Activity

            People

              rhn-engineering-shawkins Steven Hawkins
              rhn-engineering-shawkins Steven Hawkins
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - 1 day Original Estimate - 1 day
                  1d
                  Remaining:
                  Remaining Estimate - 0 minutes
                  0m
                  Logged:
                  Time Spent - 1 day, 2 hours
                  1d 2h