Uploaded image for project: 'AMQ Streams'
  1. AMQ Streams
  2. ENTMQST-33

Create Topic Controller for managing Kafka topics "the OpenShift way"

    XMLWordPrintable

Details

    • Epic
    • Resolution: Done
    • Major
    • None
    • None
    • topic-operator
    • Topic Management
    • Done
    • 100
    • 100% 100%

    Description

      When running Apache Kafka on OpenShift it can be expected that many of the applications connecting to it will run on OpenShift as well. In such environment it would be great if also the Kafka topics can be managed in "Kubernetes / OpenShift way". For that purpose we should implement a Topic Controller which will work on the operator principle and use Config Maps or Custom Resource Definitions to manage topics.

      • When a new Config Map / CRD is created, the corresponding Kafka Topic should be created by the controller
      • When the Config Map / CRD is updated, the topic should be updated as well (configuration, replication, partitioning)
      • When the Config Map / CRD is deleted the Kafka Topic should be deleted as well

      Since many tools create topics also directly in Kafka (Kafka Streams API, Kafka Connect etc.) it would be good if the operator can work in both directions and also (optrionally) create Config Maps / CRDs for topics created through Kafka or update the Config Maps / CRDs when topic configuration changes directly in Kafka.

      The Topic controller also needs to be able to deal with the differences in allowed characters for names of ConfigMaps / CRDs and Kafka topics.

      User stories:

      • As a Administrator, I need to add and configure a topic on an AMQ Streams cluster to allow applications to produce and consume data streams.
      • As a Administrator, I need to modify the configuration parameters of an existing topic so that it meets the changing needs of the applications that produce or consume streams to or from it respectively

      Attachments

        Activity

          People

            Unassigned Unassigned
            scholzj JAkub Scholz
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: