Uploaded image for project: 'Managed Service - Connectors'
  1. Managed Service - Connectors
  2. MGDCTRS-1737

Debezium connectors are not failing (instead stuck in `provisioning` state) when RHOSAK partition count quota exceeded

This issue belongs to an archived project. You can view it, but you can't modify it. Learn more

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Won't Do
    • Icon: Major Major
    • None
    • None
    • None
    • False
    • Hide

      None

      Show
      None
    • False
    • Very Likely
    • 0

      Debezium connectors are not failing but instead get stuck in `provisioning` state when RHOSAK partition count quota exceeded (on PolicyViolationException):

      2022-11-29 16:39:24,149 ERROR [Worker clientId=connect-1, groupId=mctr-ce31md5ckvjrfn14ibp0] Uncaught exception in herder work thread, exiting: (org.apache.kafka.connect.runtime.distributed.DistributedHerder) [DistributedHerder-connect-1-1]
      org.apache.kafka.connect.errors.ConnectException: Error while attempting to create/find topic(s) 'mctr-ce31md5ckvjrfn14ibp0-offset'
      at org.apache.kafka.connect.util.TopicAdmin.createOrFindTopics(TopicAdmin.java:427)
      at org.apache.kafka.connect.util.TopicAdmin.createTopics(TopicAdmin.java:332)
      at org.apache.kafka.connect.storage.KafkaOffsetBackingStore.lambda$createKafkaBasedLog$0(KafkaOffsetBackingStore.java:137)
      at org.apache.kafka.connect.util.KafkaBasedLog.start(KafkaBasedLog.java:170)
      at org.apache.kafka.connect.storage.KafkaOffsetBackingStore.start(KafkaOffsetBackingStore.java:151)
      at org.apache.kafka.connect.runtime.Worker.start(Worker.java:187)
      at org.apache.kafka.connect.runtime.AbstractHerder.startServices(AbstractHerder.java:130)
      at org.apache.kafka.connect.runtime.distributed.DistributedHerder.run(DistributedHerder.java:320)
      at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
      at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
      at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
      at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
      at java.base/java.lang.Thread.run(Thread.java:829)
      Caused by: java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.PolicyViolationException: Topic mctr-ce31md5ckvjrfn14ibp0-offset with 25 partitions would exceed the cluster partition limit of 100 (current count is 94)
      at java.base/java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:395)
      at java.base/java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1999)
      at org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:165)
      at org.apache.kafka.connect.util.TopicAdmin.createOrFindTopics(TopicAdmin.java:387)
      ... 12 more
      Caused by: org.apache.kafka.common.errors.PolicyViolationException: Topic mctr-ce31md5ckvjrfn14ibp0-offset with 25 partitions would exceed the cluster partition limit of 100 (current count is 94)
      Nov 29, 2022 4:39:24 PM org.glassfish.jersey.internal.inject.Providers checkProviderRuntime
      WARNING: A provider org.apache.kafka.connect.runtime.rest.resources.LoggingResource registered in SERVER runtime does not implement any provider interfaces applicable in the SERVER runtime. Due to constraint configuration problems the provider org.apache.kafka.connect.runtime.rest.resources.LoggingResource will be ignored.
      2022-11-29 16:39:24,155 INFO Stopped http_8083@49122b8f{HTTP/1.1, (http/1.1)}{0.0.0.0:8083} (org.eclipse.jetty.server.AbstractConnector) [JettyShutdownThread]
      2022-11-29 16:39:24,155 INFO node0 Stopped scavenging (org.eclipse.jetty.server.session) [JettyShutdownThread]
      2022-11-29 16:39:24,765 INFO Started o.e.j.s.ServletContextHandler@34b87182{/,null,SHUTDOWN} (org.eclipse.jetty.server.handler.ContextHandler) [main]
      2022-11-29 16:39:24,765 INFO REST resources initialized; server is started and ready to handle requests (org.apache.kafka.connect.runtime.rest.RestServer) [main]
      2022-11-29 16:39:24,765 INFO Kafka Connect started (org.apache.kafka.connect.runtime.Connect) [main]
      2022-11-29 16:39:24,765 INFO Kafka Connect stopping (org.apache.kafka.connect.runtime.Connect) [connect-shutdown-hook]
      2022-11-29 16:39:24,765 INFO Stopping REST server (org.apache.kafka.connect.runtime.rest.RestServer) [connect-shutdown-hook]
      2022-11-29 16:39:24,765 INFO REST server stopped (org.apache.kafka.connect.runtime.rest.RestServer) [connect-shutdown-hook]
      2022-11-29 16:39:24,765 INFO [Worker clientId=connect-1, groupId=mctr-ce31md5ckvjrfn14ibp0] Herder stopping (org.apache.kafka.connect.runtime.distributed.DistributedHerder) [connect-shutdown-hook]
      2022-11-29 16:39:29,766 INFO [Worker clientId=connect-1, groupId=mctr-ce31md5ckvjrfn14ibp0] Herder stopped (org.apache.kafka.connect.runtime.distributed.DistributedHerder) [connect-shutdown-hook]
      2022-11-29 16:39:29,766 INFO Kafka Connect stopped (org.apache.kafka.connect.runtime.Connect) [connect-shutdown-hook]
      

              Unassigned Unassigned
              rk3rn3r René Kerner
              Archiver:
              ranumula@redhat.com Raju Anumula

                Created:
                Updated:
                Resolved:
                Archived: