Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-8735

Add support for Google Cloud Pub/Sub locational endpoints

XMLWordPrintable

    • Icon: Feature Request Feature Request
    • Resolution: Done
    • Icon: Minor Minor
    • 3.1.0.Beta1
    • None
    • debezium-server
    • None
    • False
    • None
    • False

      Feature request or enhancement

      Which use case/requirement will be addressed by the proposed feature?

      Currently, the Debezium Server Pub/Sub connector does not provide a way to specify Google Cloud Pub/Sub locational endpoints in production environments. While `debezium.sink.pubsub.address` can be specified, this is intended for development or test environments with emulators and is not suitable for production use with regional endpoints as it uses plaintext communication without encryption. 

      This feature request aims to address the following use cases:

      • When using ordering keys with Pub/Sub, messages must be processed in the same region to guarantee order. The current global endpoint connection relies on automatic routing to the nearest region, which cannot be 100% guaranteed.
      • Organizations with specific compliance, data residency requirements, or message storage policies with `enforceInTransit=True` need direct control over which regional endpoints process their Pub/Sub messages. Without explicit endpoint specification, requests may be rejected if initially routed to non-allowed regions.

       

      refs:

      https://cloud.google.com/pubsub/docs/ordering

      https://cloud.google.com/pubsub/docs/resource-location-restriction

       

      Adding locational endpoint support would allow direct connections to regional endpoints (e.g., us-central1), bypassing the uncertainties of global endpoint routing.

      Implementation ideas (optional)

      Add a new config parameter `debezium.sink.pubsub.locational.endpoint` that accepts a Google Cloud region name (e.g., `us-central1`). When specified, Debezium Server would construct and use the appropriate regional endpoint URL (e.g., https://us-central1-pubsub.googleapis.com/) instead of the global endpoint.

              Unassigned Unassigned
              kesompochy Kento Someya
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated:
                Resolved: