Uploaded image for project: 'Teiid'
  1. Teiid
  2. TEIID-6019

Normalize and expand file api and file glob search

    Details

    • Type: Enhancement
    • Status: Resolved (View Workflow)
    • Priority: Major
    • Resolution: Done
    • Affects Version/s: None
    • Fix Version/s: 16.0
    • Component/s: Misc. Connectors
    • Labels:
      None
    • Sprint:
      DV Sprint 66
    • Story Points:
      1
    • Release Notes Text:
      Hide
      "exception on file not found" will also happen on the return of an empty array - as we cannot quickly differentiate between a situation where the path is referring to a parent directory that does not exist and one where no files in a given directory match. At worst this may mean some users will need to update their file translator properties.
      Show
      "exception on file not found" will also happen on the return of an empty array - as we cannot quickly differentiate between a situation where the path is referring to a parent directory that does not exist and one where no files in a given directory match. At worst this may mean some users will need to update their file translator properties.

      Description

      The file sources need to have consistent implementation of getFiles and should provide additional information such as the full file path and if it's a directory. For example currently it's possible to return directories in the file source if you specify a non-glob directory path, but that information is not exposed.

      The glob paths should all support multiple, not just a single * in the file name.

      At a first glance we may not implement these changes for ftp, but just throw appropriate exceptions - it currently only knows about returning a single file. It would be possible to eventually add logic for that though as it supports similar file visiting logic as Java.

      So the focus will be on file, hdfs, and s3 - which Aditya Manglam Sharma is already looking into.

        Gliffy Diagrams

          Attachments

            Issue Links

              Activity

                People

                • Assignee:
                  shawkins Steven Hawkins
                  Reporter:
                  shawkins Steven Hawkins
                • Votes:
                  0 Vote for this issue
                  Watchers:
                  1 Start watching this issue

                  Dates

                  • Created:
                    Updated:
                    Resolved:

                    Time Tracking

                    Estimated:
                    Original Estimate - 1 day Original Estimate - 1 day
                    1d
                    Remaining:
                    Remaining Estimate - 0 minutes
                    0m
                    Logged:
                    Time Spent - 1 day, 2 hours
                    1d 2h