Uploaded image for project: 'Managed Service - Streams'
  1. Managed Service - Streams
  2. MGDSTRM-10905

[multitenant mvp] tenant group isolation

XMLWordPrintable

    • False
    • None
    • False
    • No
    • MGDSRVS-373 - Kafka Proxy
    • ---
    • ---
    • MK - Sprint 233

      *Is your feature request related to a problem? Please describe.*

      The multi-tenant POC implemented the introduced limited multi tenant support for *topics*. Topic names are prefixed using a tenant prefix (currently derived from the SNI used that the tenant uses to connect). For effect is that tenants A and B can have a topic `mytopic` without collision.

      This issue concerned with extending the POC to support Kafka groups too. Each RPC that carries a `groupid` must be intercepted and rewritten: for requests this means appending the prefix, for responses it means removing the prefix.

      This [discussion](https://github.com/kroxylicious/design/discussions/17) sets of the list of RPCs that need to mapped for multi-tenancy. This issue is concern with those RPCs that carry a `groupid`. The scope of this work is limited to non-transaction use-cases, so RPCs such as `AddOffsetsToTxn` are out of scope.

      *Describe the solution you'd like*

      As tenant A, my application must be able to consume from topic `foo` using groupid `bar`, independently of any other tenant, who happens to chose the same group id.

      The consumer API supports two consume gestures: manual assignment (`KafkaConsumer#assign`) and subscription (`KafkaConsumer#subscribe`). The admin API supports a number of APIs for maintenance of a consumer group (`#listConsumerGroups`, `#describeConsumerGroups`, `#deleteConsumerGroup`, these are all in scope.

      *Describe alternatives you've considered*
      N/A

      *Additional context*

      This task is large so it should be delivered incrementally (i.e. several PRs). An obstacle for getting starting is that in Kafka groups are created by side effect, so several RPCs will need to be implemented before a group is actually created on the broker).

      A suggested way to chunk the work:

      1. Manual assignment / list groups (RPCs: FindCoordinator, OffsetFetch, OffsetCommit, ListGroups)
      2. Subscription (RPCs: JoinGroup, SyncGroup, LeaveGroup, Heartbeat)
      3. DescribeGroups (RPCs: DescribeGroup)
      4. alterConsumerGroupOffsets and deleteConsumerGroupOffsets (RPCs: OffsetCommit, . OffsetDelete)

              raryan@redhat.com Rachel Ryan
              keithbwall Keith Wall
              Kafka Integrations
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated:
                Resolved: