Uploaded image for project: 'Teiid'
  1. Teiid
  2. TEIID-5686

NPE in MergeJoinStrategy when using joins with a particular limit value

    XMLWordPrintable

    Details

    • Type: Bug
    • Status: Resolved (View Workflow)
    • Priority: Major
    • Resolution: Done
    • Affects Version/s: 12.0
    • Fix Version/s: 12.2
    • Component/s: Query Engine
    • Labels:
      None
    • Steps to Reproduce:
      Hide

      1. In postgresql create the test_a and test_e tables using the following scripts:

      CREATE TABLE public.test_a
      (
        a integer,
        b integer
      );
      INSERT INTO public.test_a(a, b) VALUES (1, 1);
      INSERT INTO public.test_a(a, b) VALUES (1, 2);
      INSERT INTO public.test_a(a, b) VALUES (2, 1);
      INSERT INTO public.test_a(a, b) VALUES (2, 2);
      INSERT INTO public.test_a(a, b) VALUES (3, 2);
      INSERT INTO public.test_a(a, b) VALUES (3, 10);
      
      CREATE TABLE public.test_e
      (
        e character varying(254),
        f integer
      );
      INSERT INTO public.test_e(e, f) VALUES ('test', 1234);
      INSERT INTO public.test_e(e, f) VALUES ('abcde', 1111);
      INSERT INTO public.test_e(e, f) VALUES ('abtest', 2222);
      INSERT INTO public.test_e(e, f) VALUES ('testab', 3333);
      INSERT INTO public.test_e(e, f) VALUES ('abtestab', 4444);
      

      2. Add postgresql database configuration in standalone-teiid.xml:

                      <datasource jndi-name="java:/test_tables_pg" pool-name="test_tables_pg" enabled="true" use-java-context="true">
                          <connection-url>jdbc:postgresql://localhost:5432/test?charSet=utf8</connection-url>
                          <driver-class>org.postgresql.Driver</driver-class>
                          <driver>org.postgresql</driver>
                          <pool>
                              <min-pool-size>2</min-pool-size>
                              <max-pool-size>70</max-pool-size>
                              <prefill>false</prefill>
                              <use-strict-min>false</use-strict-min>
                              <flush-strategy>FailingConnectionOnly</flush-strategy>
                          </pool>
                          <security>
                              <user-name>XXXXX</user-name>
                              <password>XXXXX</password>
                          </security>
                          <validation>
                              <check-valid-connection-sql>select 0</check-valid-connection-sql>
                          </validation>
                          <timeout>
                              <blocking-timeout-millis>120000</blocking-timeout-millis>
                              <idle-timeout-minutes>5</idle-timeout-minutes>
                          </timeout>
                      </datasource>
      

      3. Add in test-vdb.xml java:/test_tables_pg configured in previous step as datasource:

          <model name="test_tables_pg">
              <property name="importer.useFullSchemaName" value="false"/>
              <property name="importer.tableTypes" value="TABLE,VIEW"/>
              <property name="importer.importKeys" value="false"/>
              <source name="test_tables_pg" translator-name="myPg" connection-jndi-name="java:/test_tables_pg"/>
          </model>
      

      4. Configure in the test-vdb.xml the following virtual view:

          <model visible = "true" type = "VIRTUAL" name = "views">
              <metadata type = "DDL"><![CDATA[
                   create view v as
      		select *
      		from(
      			select e
      			from "test_tables_pg.test_e" tb1, TEXTTABLE(tb1.e COLUMNS col1 string) x
         		) t 
      		join "test_tables_pg.test_a" tb2 
         		on true
              ]]>
              </metadata>
          </model>
      

      5. Run the following query:

      begin
          declare integer i=0;
          while (i < 2)
              begin
                  select * 
                  from "test_tables_pg.test_a" t0
                  JOIN "test_tables_pg.test_e" t1
                    ON true
                  JOIN "test_tables_pg.test_e" t2
                    ON true
                  JOIN views.v t3
                        on true
                  JOIN views.v t4
                        on true
                  limit 257 ;
                  i=i+1;
              end
      end ;;
      

      IMPORTANT: you shouldn't use row limit. I used SQuirrel and the bug was reproduced only when I removed row limit and provided limit clause with 257 value and above. If you provide limit clause with 256 value and below the query above will work correctly.

      Show
      1. In postgresql create the test_a and test_e tables using the following scripts: CREATE TABLE public .test_a ( a integer , b integer ); INSERT INTO public .test_a( a , b) VALUES (1, 1); INSERT INTO public .test_a( a , b) VALUES (1, 2); INSERT INTO public .test_a( a , b) VALUES (2, 1); INSERT INTO public .test_a( a , b) VALUES (2, 2); INSERT INTO public .test_a( a , b) VALUES (3, 2); INSERT INTO public .test_a( a , b) VALUES (3, 10); CREATE TABLE public .test_e ( e character varying (254), f integer ); INSERT INTO public .test_e(e, f) VALUES ( 'test' , 1234); INSERT INTO public .test_e(e, f) VALUES ( 'abcde' , 1111); INSERT INTO public .test_e(e, f) VALUES ( 'abtest' , 2222); INSERT INTO public .test_e(e, f) VALUES ( 'testab' , 3333); INSERT INTO public .test_e(e, f) VALUES ( 'abtestab' , 4444); 2. Add postgresql database configuration in standalone-teiid.xml: <datasource jndi-name= "java:/test_tables_pg" pool-name= "test_tables_pg" enabled= "true" use-java-context= "true" > <connection-url> jdbc:postgresql://localhost:5432/test?charSet=utf8 </connection-url> <driver-class> org.postgresql.Driver </driver-class> <driver> org.postgresql </driver> <pool> <min-pool-size> 2 </min-pool-size> <max-pool-size> 70 </max-pool-size> <prefill> false </prefill> <use-strict-min> false </use-strict-min> <flush-strategy> FailingConnectionOnly </flush-strategy> </pool> <security> <user-name> XXXXX </user-name> <password> XXXXX </password> </security> <validation> <check-valid-connection-sql> select 0 </check-valid-connection-sql> </validation> <timeout> <blocking-timeout-millis> 120000 </blocking-timeout-millis> <idle-timeout-minutes> 5 </idle-timeout-minutes> </timeout> </datasource> 3. Add in test-vdb.xml java:/test_tables_pg configured in previous step as datasource: <model name= "test_tables_pg" > <property name= "importer.useFullSchemaName" value= "false" /> <property name= "importer.tableTypes" value= "TABLE,VIEW" /> <property name= "importer.importKeys" value= "false" /> <source name= "test_tables_pg" translator-name= "myPg" connection-jndi-name= "java:/test_tables_pg" /> </model> 4. Configure in the test-vdb.xml the following virtual view: <model visible = "true" type = "VIRTUAL" name = "views" > <metadata type = "DDL" > <![CDATA[ create view v as select * from( select e from "test_tables_pg.test_e" tb1, TEXTTABLE(tb1.e COLUMNS col1 string) x ) t join "test_tables_pg.test_a" tb2 on true ]]> </metadata> </model> 5. Run the following query: begin declare integer i=0; while (i < 2) begin select * from "test_tables_pg.test_a" t0 JOIN "test_tables_pg.test_e" t1 ON true JOIN "test_tables_pg.test_e" t2 ON true JOIN views.v t3 on true JOIN views.v t4 on true limit 257 ; i=i+1; end end ;; IMPORTANT: you shouldn't use row limit. I used SQuirrel and the bug was reproduced only when I removed row limit and provided limit clause with 257 value and above. If you provide limit clause with 256 value and below the query above will work correctly.

      Description

      When running the following query (IMPORTANT: you shouldn't use row limit):

      begin
          declare integer i=0;
          while (i < 2)
              begin
                  select * 
                  from "test_tables_pg.test_a" t0
                  JOIN "test_tables_pg.test_e" t1
                    ON true
                  JOIN "test_tables_pg.test_e" t2
                    ON true
                  JOIN views.v t3
                        on true
                  JOIN views.v t4
                        on true
                  limit 257 ;
                  i=i+1;
              end
      end ;;
      

      teiid throws out the following NPE:

      Unable to find source-code formatter for language: noformat. Available languages are: actionscript, ada, applescript, bash, c, c#, c++, cpp, css, erlang, go, groovy, haskell, html, java, javascript, js, json, lua, none, nyan, objc, perl, php, python, r, rainbow, ruby, scala, sh, sql, swift, visualbasic, xml, yaml
      2019-03-13 13:50:28,582 ERROR [org.teiid.PROCESSOR] (Worker4_QueryProcessorQueue233) InwGU8fhfbja TEIID30019 Unexpected exception for request InwGU8fhfbja.22: java.lang.NullPointerEx
      ception
              at org.teiid.query.processor.relational.MergeJoinStrategy.compareToPrevious(MergeJoinStrategy.java:284)
              at org.teiid.query.processor.relational.MergeJoinStrategy.process(MergeJoinStrategy.java:238)
              at org.teiid.query.processor.relational.JoinNode.nextBatchDirectInternal(JoinNode.java:260)
              at org.teiid.query.processor.relational.JoinNode.nextBatchDirect(JoinNode.java:195)
              at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:277)
              at org.teiid.query.processor.relational.LimitNode.nextBatchDirect(LimitNode.java:98)
              at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:277)
              at org.teiid.query.processor.relational.ProjectNode.nextBatchDirect(ProjectNode.java:146)
              at org.teiid.query.processor.relational.RelationalNode.nextBatch(RelationalNode.java:277)
              at org.teiid.query.processor.relational.RelationalPlan.nextBatch(RelationalPlan.java:141)
              at org.teiid.query.processor.QueryProcessor.nextBatchDirect(QueryProcessor.java:148)
              at org.teiid.query.processor.QueryProcessor.nextBatch(QueryProcessor.java:111)
              at org.teiid.query.processor.BatchIterator.finalRow(BatchIterator.java:65)
              at org.teiid.common.buffer.AbstractTupleSource.getCurrentTuple(AbstractTupleSource.java:66)
              at org.teiid.query.processor.BatchIterator.getCurrentTuple(BatchIterator.java:80)
              at org.teiid.common.buffer.AbstractTupleSource.nextTuple(AbstractTupleSource.java:44)
              at org.teiid.query.processor.proc.ProcedurePlan.nextBatchDirect(ProcedurePlan.java:303)
              at org.teiid.query.processor.proc.ProcedurePlan.nextBatch(ProcedurePlan.java:269)
              at org.teiid.query.processor.QueryProcessor.nextBatchDirect(QueryProcessor.java:148)
              at org.teiid.query.processor.QueryProcessor.nextBatch(QueryProcessor.java:111)
              at org.teiid.query.processor.BatchCollector.collectTuples(BatchCollector.java:160)
              at org.teiid.query.processor.BatchCollector.collectTuples(BatchCollector.java:142)
              at org.teiid.dqp.internal.process.RequestWorkItem.processMore(RequestWorkItem.java:492)
              at org.teiid.dqp.internal.process.RequestWorkItem.process(RequestWorkItem.java:362)
              at org.teiid.dqp.internal.process.AbstractWorkItem.run(AbstractWorkItem.java:43)
              at org.teiid.dqp.internal.process.RequestWorkItem.run(RequestWorkItem.java:285)
              at org.teiid.dqp.internal.process.DQPWorkContext.runInContext(DQPWorkContext.java:281)
              at org.teiid.dqp.internal.process.ThreadReuseExecutor$RunnableWrapper.run(ThreadReuseExecutor.java:113)
              at org.teiid.dqp.internal.process.ThreadReuseExecutor$3.run(ThreadReuseExecutor.java:199)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
              at java.lang.Thread.run(Thread.java:745)
      

        Attachments

          Activity

            People

            Assignee:
            shawkins Steven Hawkins
            Reporter:
            dalex005 Dmitrii Pogorelov
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: