Uploaded image for project: 'Teiid'
  1. Teiid
  2. TEIID-5796

Not able to read JSON data file kelpt on FTP/SFTP server using FTP resource adapter and file translator

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Obsolete
    • Icon: Major Major
    • None
    • 11.2.1
    • AdminApi
    • None
    • Hide

      Trying to connect to below file kept on Linux FTP server,please find data:

      {"employees": [ {"name":"Shyam", "email":"shyamjaiswal@gmail.com"}

      ,

      {"name":"Bob", "email":"bob32@gmail.com"}

      ,

      {"name":"Jai", "email":"jai87@gmail.com"}

      ]}
      Create resource adapter :

      <resource-adapter id="TESTDSJSON20190724125640107">
      <module slot="main" id="org.jboss.teiid.resource-adapter.ftp"/>
      <transaction-support>NoTransaction</transaction-support>
      <connection-definitions>
      <connection-definition class-name="org.teiid.resource.adapter.ftp.FtpManagedConnectionFactory" jndi-name="java:/TESTDSJSON20190724125640107" enabled="true" pool-name="TESTDSJSON20190724125640107">
      <config-property name="Host">
      <IP of Linux FTP server>
      </config-property>
      <config-property name="ParentDirectory">
      /pranita
      </config-property>
      <config-property name="Username">
      <username>
      </config-property>
      <config-property name="Password">
      <password>
      </config-property>
      <config-property name="Port">
      <Port>
      </config-property>
      </connection-definition>
      </connection-definitions>
      </resource-adapter>
      </resource-adapters>

      VDB details:
      <?xml version="1.0" ?>
      <vdb name="SvcSourceVdb_json08" version="1">
      <description>VDB for: SvcSourceVdb_json08, Version: 1</description>
      <connection-type>BY_VERSION</connection-type>
      <model name="json08" type="PHYSICAL" visible="true">
      <property name="importer.tableTypes" value="TABLE"></property>
      <property name="importer.importKeys" value="false"></property>
      <source name="json08" translator-name="file" connection-jndi-name="java:/DSJSON20190723144550004"></source>
      </model>
      <model name="json08_viewModel" type="VIRTUAL" visible="true">
      <description>json08</description>
      <metadata type="DDL">
      <![CDATA[CREATE VIEW emp AS
      SELECT x.*
      FROM (EXEC json08.getFiles('empFile.json')) AS f,
      XMLTABLE ('/emp/employees' PASSING JSONTOXML('emp', f.file)
      COLUMNS name string PATH '../name', email string PATH '../email') as x;]]>
      </metadata></model></vdb>

      Test connection gets successfull and VDB gets deployed.

      Try to execute query using deployed VDB:
      Select * from emp;

      When I am trying to run Query first time it is giving below error:
      2019-07-25 12:40:45,168 WARN [org.teiid.PROCESSOR] (Worker8_QueryProcessorQueue43) alTTE6th5rPw TEIID30020 Processing exception for request alTTE6th5rPw.0 'TEIID30504 json11: Can not open multiple ftp stream based file in one connection'. Originally TeiidProcessingException FtpFileConnectionImpl.java:81. Enable more detailed logging to see the entire stacktrace.
      2019-07-25 13:23:14,445 ERROR [org.teiid.PROCESSOR] (Worker10_QueryProcessorQueue46) XH6GMB1xbOX6 TEIID30019 Unexpected exception for request XH6GMB1xbOX6.0: java.lang.NullPointerException
      at org.teiid.core.types.BlobImpl.getBytes(BlobImpl.java:100)

      When I am again trying to run the query, we are continuously facing FTP connection issue and getting below error:
      2019-07-25 13:23:55,656 WARN [org.teiid.CONNECTOR] (Worker11_QueryProcessorQueue48) n/vcvzEKKWI0 Connector worker process failed for atomic-request=n/vcvzEKKWI0.0.3.4: org.teiid.translator.TranslatorException: Can not open multiple ftp stream based file in one connection
      at org.teiid.translator.file.FileExecutionFactory$VirtualFileProcedureExecution.execute(FileExecutionFactory.java:208)

      Please find attached teiid logs for more details.

      Show
      Trying to connect to below file kept on Linux FTP server,please find data: {"employees": [ {"name":"Shyam", "email":"shyamjaiswal@gmail.com"} , {"name":"Bob", "email":"bob32@gmail.com"} , {"name":"Jai", "email":"jai87@gmail.com"} ]} Create resource adapter : <resource-adapter id="TESTDSJSON20190724125640107"> <module slot="main" id="org.jboss.teiid.resource-adapter.ftp"/> <transaction-support>NoTransaction</transaction-support> <connection-definitions> <connection-definition class-name="org.teiid.resource.adapter.ftp.FtpManagedConnectionFactory" jndi-name="java:/TESTDSJSON20190724125640107" enabled="true" pool-name="TESTDSJSON20190724125640107"> <config-property name="Host"> <IP of Linux FTP server> </config-property> <config-property name="ParentDirectory"> /pranita </config-property> <config-property name="Username"> <username> </config-property> <config-property name="Password"> <password> </config-property> <config-property name="Port"> <Port> </config-property> </connection-definition> </connection-definitions> </resource-adapter> </resource-adapters> VDB details : <?xml version="1.0" ?> <vdb name="SvcSourceVdb_json08" version="1"> <description>VDB for: SvcSourceVdb_json08, Version: 1</description> <connection-type>BY_VERSION</connection-type> <model name="json08" type="PHYSICAL" visible="true"> <property name="importer.tableTypes" value="TABLE"></property> <property name="importer.importKeys" value="false"></property> <source name="json08" translator-name="file" connection-jndi-name="java:/DSJSON20190723144550004"></source> </model> <model name="json08_viewModel" type="VIRTUAL" visible="true"> <description>json08</description> <metadata type="DDL"> <![CDATA[CREATE VIEW emp AS SELECT x.* FROM (EXEC json08.getFiles('empFile.json')) AS f, XMLTABLE ('/emp/employees' PASSING JSONTOXML('emp', f.file) COLUMNS name string PATH '../name', email string PATH '../email') as x;]]> </metadata></model></vdb> Test connection gets successfull and VDB gets deployed. Try to execute query using deployed VDB: Select * from emp; When I am trying to run Query first time it is giving below error: 2019-07-25 12:40:45,168 WARN [org.teiid.PROCESSOR] (Worker8_QueryProcessorQueue43) alTTE6th5rPw TEIID30020 Processing exception for request alTTE6th5rPw.0 'TEIID30504 json11: Can not open multiple ftp stream based file in one connection'. Originally TeiidProcessingException FtpFileConnectionImpl.java:81. Enable more detailed logging to see the entire stacktrace. 2019-07-25 13:23:14,445 ERROR [org.teiid.PROCESSOR] (Worker10_QueryProcessorQueue46) XH6GMB1xbOX6 TEIID30019 Unexpected exception for request XH6GMB1xbOX6.0: java.lang.NullPointerException at org.teiid.core.types.BlobImpl.getBytes(BlobImpl.java:100) When I am again trying to run the query, we are continuously facing FTP connection issue and getting below error: 2019-07-25 13:23:55,656 WARN [org.teiid.CONNECTOR] (Worker11_QueryProcessorQueue48) n/vcvzEKKWI0 Connector worker process failed for atomic-request=n/vcvzEKKWI0.0.3.4: org.teiid.translator.TranslatorException: Can not open multiple ftp stream based file in one connection at org.teiid.translator.file.FileExecutionFactory$VirtualFileProcedureExecution.execute(FileExecutionFactory.java:208) Please find attached teiid logs for more details.
    • Compatibility/Configuration, User Experience

      Not able to read JSON data file kept on FTP/SFTP server using FTP resource adapter and file translator by parsing view using XMLTABLE.
      And As per Ramesh JSONTABLE do not support nested documents.
      For more details please find discussion :
      https://developer.jboss.org/message/990083#990083

      I am using Teiid 11.1.2 Admin API and wildfly server

              rhn-engineering-shawkins Steven Hawkins
              pranitag Pranita Gurav (Inactive)
              Pranita Gurav Pranita Gurav (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated:
                Resolved: