Uploaded image for project: 'JBeret'
  1. JBeret
  2. JBERET-158

Implement a property named autoCommit to JdbcItemReader to address PostgreSQL large dataset issue

XMLWordPrintable

    • Icon: Feature Request Feature Request
    • Resolution: Done
    • Icon: Minor Minor
    • 1.1.0.Final
    • 1.1.0.Beta1
    • jberet-support
    • None

      When autoCommit=true, PostgreSQL JDBC driver ignores the fetch size which set with Statement#setFetchSize().

      http://stackoverflow.com/questions/1468036/java-jdbc-ignores-setfetchsize
      https://jdbc.postgresql.org/documentation/94/query.html#query-with-cursor

      It seems like PostgreSQL specific but important to deal with large datasets. and we can't set it to false in connection URL with PostgreSQL JDBC driver. http://postgresql.nabble.com/Connection-Properties-for-FetchSize-and-Autocommit-td2175811.html

      I think typically JdbcItemReader will be used with a non-jta data source which is autoCommit=true by default so it would insufficient to work with large datasets. so I wish JdbcItemReader to implement a batch property named autoCommit.

      A workaround is that we can implement a subclass of JdbcItemReader and override getConnection() and invoke Connection#autoCommit(false) before returning it.

            cfang@redhat.com Cheng Fang
            xkylex Kohei Nozaki (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: