Uploaded image for project: 'AMQ Streams'
  1. AMQ Streams
  2. ENTMQST-4707

Make connector task backoff configurable in Kafka Connect

    XMLWordPrintable

Details

    • Story
    • Resolution: Done
    • Major
    • 2.5.0.GA
    • 2.3.0.GA
    • None
    • None
    • False
    • None
    • False

    Description

      It appears that variable RECONFIGURE_CONNECTOR_TASKS_BACKOFF_MS is fixed and hard-coded in Connect:

      ref : https://github.com/a0x8o/kafka/blob/master/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/distributed/DistributedHerder.java

            private static final long RECONFIGURE_CONNECTOR_TASKS_BACKOFF_MS = 250;
      private void reconfigureConnectorTasksWithRetry(long initialRequestTime, final String connName) {
              reconfigureConnector(connName, (error, result) -> {
                  // If we encountered an error, we don't have much choice but to just retry. If we don't, we could get
                  // stuck with a connector that thinks it has generated tasks, but wasn't actually successful and therefore
                  // never makes progress. The retry has to run through a DistributedHerderRequest since this callback could be happening
                  // from the HTTP request forwarding thread.
                  if (error != null) {
                      if (isPossibleExpiredKeyException(initialRequestTime, error)) {
                          log.debug("Failed to reconfigure connector's tasks ({}), possibly due to expired session key. Retrying after backoff", connName);
                      } else {
                          log.error("Failed to reconfigure connector's tasks ({}), retrying after backoff:", connName, error);
                      }
                      addRequest(RECONFIGURE_CONNECTOR_TASKS_BACKOFF_MS,
                          () -> {
                              reconfigureConnectorTasksWithRetry(initialRequestTime, connName);
                              return null;
                          }, (err, res) -> {
                              if (err != null) {
                                  log.error("Unexpected error during connector task reconfiguration: ", err);
                                  log.error("Task reconfiguration for {} failed unexpectedly, this connector will not be properly reconfigured unless manually triggered.", connName);
                              }
                          }
                      );
                  }
              });
          }
      

      This enhancement would expose this property generically for all connector types and make it configurable via property or environment driven configuration.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              rhn-support-dhawkins Duane Hawkins
              Henrich Zrncik Henrich Zrncik
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: