While we attempt to save batch events, there may be errors. This card's purpose is to automatically create a new topic for the unsuccessful batch. Only those events will be included in the topic that were successfully parsed but failed to save in the table.

• This can be probably achieved by SpringRetriabletopics or DeadLetterQueue.

Some places that uses DeadLetterQueue: https://www.kai-waehner.de/blog/2022/05/30/error-handling-via-dead-letter-queue-in-apache-kafka/
Example of DeadLetterQueue: https://blog.jdriven.com/2022/01/kafka-dead-letter-publishing/

Example of SpringRetriabletopics: https://www.lydtechconsulting.com/blog-kafka-spring-retry-topics.html

Note: Mostly we will go with DeadLetterQueue since it will be supported both in spring and quarkus

• Add the new topic in SRE github.

Done: Events failed before saving to the table should be put in a dead letter queue for further investigation.
Question should we retry in the same flow?

• For now 1 retry, then DLQ

Log warn when there is a retry and error when we are in DLQ

Note: Work with QE if there is a way to test this easily, leaving off QE label for now.