• Icon: Feature Request Feature Request
    • Resolution: Unresolved
    • Icon: Major Major
    • 7.later
    • 5.12.5.Final
    • Demonstrator
    • None

      Kafka supports transactional producers for sending messages between topics [1]:

      > "Transactional delivery allows producers to send data to multiple partitions such that either all messages are successfully delivered, or none of them are."

      However, the user cannot include other types of data (such as an SQL database update) within the transaction because Kafka does not support XA resources - this RFE is to explore ways of achieving the same effect.

      For example, if the other type of data is accessed via JDBC then Narayana provides LRCO and CMR [2,3]. This use case should already be supported so the task would be to produce a blog, or otherwise, explaining how to use it with Kafka. It is a variation of a more general strategy where Kafka messages are stored in a database during the transaction (e.g. using a producer interceptor [4]); it's more general because it means other XA resource types can be included as part of the JTA transaction.

      If full ACID is not required then another approach would be to use a compensation based model such as LRA [5].

      [1] https://kafka.apache.org/26/javadoc/org/apache/kafka/clients/producer/KafkaProducer.html
      [2] https://access.redhat.com/documentation/en-us/red_hat_jboss_enterprise_application_platform/7.3/html/development_guide/java_transaction_api#transaction_optimizations
      [3] https://narayana.io//docs/project/index.html#d0e1859
      [4] https://kafka.apache.org/0100/javadoc/org/apache/kafka/clients/producer/ProducerInterceptor.html
      [5] https://github.com/eclipse/microprofile-lra#long-running-actions-for-microprofile

            rh-ee-msappegr Marco Sappe Griot
            rhn-engineering-mmusgrov Michael Musgrove
            0 Vote for this issue
            1 Start watching this issue