Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-2002

DB2/AS400 CDC using free jt400 library

XMLWordPrintable

    • Icon: Feature Request Feature Request
    • Resolution: Done
    • Icon: Optional Optional
    • 2.6.0.Beta1
    • Backlog
    • db2-connector
    • None

      The current DB2 connector requires a license for IIDR (IBM InfoSphere Data Replication)

      It seems that we should be able to access the journal using a direct program call. There is also a SQL function but that tends to be considerably slower.

      The program QjoRetrieveJournalEntries retrieves the journal log we would need to use the jt400 library to call it.
      There's some sample code here: https://stanleyvong.blogspot.com/ I used to test out the concept - Stanley is OK with licensing it as Apache.
      There is a limitation that the code retrieves the current structure of the table to decode the data returned - that might not be the structure at the time of the log entry. Also my prototype has some problems with some of journal entries which I've not fully investigated - it could be bad data e.g. text in numeric fields. I suspect there's quite a bit of work to decode the data.

      jt400 library http://jt400.sourceforge.net/
      jt400 licensed as https://opensource.org/licenses/ibmpl.php.

      Gunnar Morling suggests creating an alternative StreamingChangeEventSource implementation within the existing Db2 connector with an option for controlling which ones of the implementations to choose
      He also highlights the problem that testing requires a AS400 with DB2 and that's not available as a docker image - there is google cloud support but I imagine this will be expensive.

            Unassigned Unassigned
            msillencejhc Martin Sillence (Inactive)
            Votes:
            2 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: