Uploaded image for project: 'AMQ Streams'
  1. AMQ Streams
  2. ENTMQST-1550

[DOC OCP] TO: When recovering cluster from PVs, all topics might be deleted

    XMLWordPrintable

    Details

      Description

      Users / customer might need to recover a deleted cluster from the PVs only - for example after loss of a cluster of namespace. That seems to work fine in general:

      • You take the PVs
      • You create matching PVCs with the right names and bind the correct PV into them
      • You start the cluster, it recovers the data from disk as with normal restart and continues the operations.

      This was confirmed by several users to work with on exception and that is the Topic Operator. The Topic operator does the following:

      • If the KafkaTopic resources were not recovered before the cluster was started, they will end up being deleted.
      • The Zookeeper PVs contain the internal TO state as well. so as a result the TO will start and see following information: 1) The topics exist in Kafka, 2) The KafkaTopics do not exist and 3) The topics existed in the past.
      • So it mades a seemingly valid decision to delete the topics assuming the KafkaTopic resources were deleted.

      This is how it normally works during regular operations, so it is not a bug per se. However, it is quite dangerous and undesired during the recovery.

      I guess the options how to fix this are quite limited:

      • Fix it only in Docs with warning?
      • Recommend to disable topic deletion in Kafka?

      Or do we have anything else?

        Attachments

          Activity

            People

            Assignee:
            pmellor Paul Mellor
            Reporter:
            scholzj JAkub Scholz
            Tester:
            Jakub Stejskal Jakub Stejskal
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: