Uploaded image for project: 'Teiid'
  1. Teiid
  2. TEIID-3579

Hive 0.13.1 JDBC jars makes queries run slow in DV runtime

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Duplicate
    • Icon: Major Major
    • None
    • None
    • JDBC Driver
    • None
    • Mac OS X Yosemite 10.10.4, JBoss Developer Studio 8.1.0 GA, JBoss Data Virtualization 6.1.0

    • Hide

      1. Download the Hive JDBC driver modules folder as .zip from here: https://drive.google.com/file/d/0BxJhoZ1V34QHSmgzTlBRVktZaGM/
      2. Unzip the downloaded .zip file onto $DV_HOME
      3. Have HDP 2.1 running at a reachable IP and PORT with a username and password
      4. Ensure that the username has SELECT access to all the tables of the schema/database in the JDBC connection in later steps
      5. Add a driver definition as shown below to the standalone.xml file

      <driver name="hive" module="org.apache.hadoop.hive:0.13.1">
      <driver-class>org.apache.hive.jdbc.HiveDriver</driver-class>
      </driver>

      6. Start the DV runtime server
      7. Create a new Teiid Model project using JDBS 8.1.0 in Teiid Designer perspective
      8. Try importing a 'Teiid Connection >> Source Model', click on a 'New' button to create a new Datasource connection, choose 'hive' as the driver and translator for the connection and provide the JDBC URL in the format:
      jdbc:hive2://ip_address:10000/default . See if the importVDB.vdb gets successfully deployed and you can import the metadata of two tables sample_07 and sample_08 into HDP.xmi (for example) source model.
      9. After finishing step 8 successfully, right click on the one of the imported table and choose 'Preview Data'. Note how long it takes to get back 500 rows of data

      Show
      1. Download the Hive JDBC driver modules folder as .zip from here: https://drive.google.com/file/d/0BxJhoZ1V34QHSmgzTlBRVktZaGM/ 2. Unzip the downloaded .zip file onto $DV_HOME 3. Have HDP 2.1 running at a reachable IP and PORT with a username and password 4. Ensure that the username has SELECT access to all the tables of the schema/database in the JDBC connection in later steps 5. Add a driver definition as shown below to the standalone.xml file <driver name="hive" module="org.apache.hadoop.hive:0.13.1"> <driver-class>org.apache.hive.jdbc.HiveDriver</driver-class> </driver> 6. Start the DV runtime server 7. Create a new Teiid Model project using JDBS 8.1.0 in Teiid Designer perspective 8. Try importing a 'Teiid Connection >> Source Model', click on a 'New' button to create a new Datasource connection, choose 'hive' as the driver and translator for the connection and provide the JDBC URL in the format: jdbc:hive2://ip_address:10000/default . See if the importVDB.vdb gets successfully deployed and you can import the metadata of two tables sample_07 and sample_08 into HDP.xmi (for example) source model. 9. After finishing step 8 successfully, right click on the one of the imported table and choose 'Preview Data'. Note how long it takes to get back 500 rows of data
    • User Experience

      When using the JDBC jars for Hive 0.13.1 running on HDP 2.1, queries executed against table 'default.sample_07' takes approximately 20-30 seconds to return.

      The Hive JDBC jars for version 0.13.1 can be found here :
      https://github.com/vchintal/hive-jdbc-jars-archive

      Alternatively a ready-to-go modules can be downloaded from here for testing:
      https://drive.google.com/file/d/0BxJhoZ1V34QHSmgzTlBRVktZaGM/

      Use the following driver snippet when using the above mentioned module:

      <driver name="hive" module="org.apache.hadoop.hive:0.13.1">
      <driver-class>org.apache.hive.jdbc.HiveDriver</driver-class>
      </driver>

              rhn-engineering-shawkins Steven Hawkins
              vchintal@redhat.com Vijay Chintalapati (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated:
                Resolved: