Uploaded image for project: 'Hybrid Cloud Console'
  1. Hybrid Cloud Console
  2. RHCLOUD-21074

Set up reaper job for managed kafka (Ephemeral Environment)

XMLWordPrintable

    • False
    • Hide

      None

      Show
      None
    • False
    • Hide

      The script will need to

      • Talk to managed kafka to get a list of all topics within the whitelisted pattern.
      • Talk to the clusters that are using it (dev/ephem) to get a list of namespaces
      • If managed kafka has topics that exist for namespaces that don't on the clusters, remove them.

      It needs to happen in this order to prevent the destruction of topics that are new. i.e. if you first talk to the clusters and then talk to managed kafka, a namespace could have been provisioned in the time it has taken inbetween. Doing it this way lessens the risk, and will simply yield some extra topics that are missed until the next cleanup.

      It is also important to collect ALL namespaces from both / multiple clusters. Ideally we should have a managed kafka instance for each cluster and be careful about naming.

      Show
      The script will need to Talk to managed kafka to get a list of all topics within the whitelisted pattern. Talk to the clusters that are using it (dev/ephem) to get a list of namespaces If managed kafka has topics that exist for namespaces that don't on the clusters, remove them. It needs to happen in this order to prevent the destruction of topics that are new. i.e. if you first talk to the clusters and then talk to managed kafka, a namespace could have been provisioned in the time it has taken inbetween. Doing it this way lessens the risk, and will simply yield some extra topics that are missed until the next cleanup. It is also important to collect ALL namespaces from both / multiple clusters. Ideally we should have a managed kafka instance for each cluster and be careful about naming.
    • Unset
    • CRCPLAN-132 - HCC | Managed Instance of Kafka
    • None
    • Major

      Though we do our best to clean up envs - There are going to be instances where this doesn't happen. We need to clean up all other managed kafka topics to ensure that we have enough free topic/partitions for new envs.

              rhn-support-cmitchel Chris Mitchell
              psavage@redhat.com Peter Savage
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated:
                Resolved: