Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-3

Ingest change data from PostgreSQL databases

    XMLWordPrintable

Description

    Add support for monitoring PostgreSQL databases by using logical decoding (which reads the database's write ahead log) and writing those changes to Kafka using Kafka Connect.

    This involves creating a Kafka Connect source connector for PostgreSQL that creates source tasks as needed. Each source task remotely connects to a PostgreSQL database and tails the oplog to extract and return the individual data change events that include, among other things, the position in the WAL where the event was read. When run as part of Kafka Connect, the resulting data change events will be recorded in Kafka logs, and if anything fails Kafka Connect will figure out what was last successfully written and restart a source task to start reading from the next point.

    The connector should ideally provide the same behavior, functionality, and options as the MySQL connector. Where possible, this connector should also use the same configuration fields as other connectors.

    Attachments

      Activity

        People

          hchiorean Horia Chiorean (Inactive)
          rhauch Randall Hauch (Inactive)
          Votes:
          1 Vote for this issue
          Watchers:
          5 Start watching this issue

          Dates

            Created:
            Updated:
            Resolved: