Uploaded image for project: 'AMQ Streams'
  1. AMQ Streams
  2. ENTMQST-5584

Include common plug-ins in Kafka Connect images

XMLWordPrintable

    • Icon: Story Story
    • Resolution: Done
    • Icon: Undefined Undefined
    • None
    • 2.5.0.GA
    • kafka-connect
    • None
    • False
    • None
    • False

      Kafka Connect on Streams is rather painful to use, for Debezium or anything else. Although it's possible to deploy a stock KafkaConnect custom resource, it will provide no useful plug-ins. In practice, every practical application of Kafka Connect on Streams requires creating a custom image.

      While the infrastructure does automate the process of custom image creation to some extent, it's time-consuming to set up, and time-consuming for the actual build. Integrators still have to locate the necessary plug-ins and make them available to the infrastructure. It isn't entirely clear what versions of these plug-ins Red Hat tests or supports for a specific Streams release, or what sub-dependencies might need to be included.

      Red Hat supports only a limited number of applications of Kafka Connect – essentially Debezium with a small number of databases. It would make KC much easier to use if the dependencies for these supported applications were included in the base image. I appreciate that that the custom build process would still be required, for those cases (e.g., Oracle) where we can't distribute proprietary binaries. However, mandating its use when there's no logical need for it is off-putting.

      Since the runtime size of the KC image is already 600Mb, adding all the connectors we support would not make the image significantly larger, compared to its base size. But it would make KC a lot easier to adopt.

       

       

            Unassigned Unassigned
            rhn-support-kboone Kevin Boone
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated:
              Resolved: