Uploaded image for project: 'Hybrid Cloud Console'
  1. Hybrid Cloud Console
  2. RHCLOUD-43679

Safely batch cleanup records in Inventory

XMLWordPrintable

    • Product / Portfolio Work
    • False
    • Hide

      None

      Show
      None
    • False
    • None
    • Unset
    • None

      Like SpiceDB bulk delete, we need a mass cleanup mechanism for Inventory. We cannot rely on manual psql access or the Kessel Debug Pod for bulk deletion in Prod during an emergency. 

      One potential solution is utilizing a kube job and custom cobra command built on top of inventory's existing gorm usage

      • The job must perform deletes in batches (e.g., 5,000 records at a time) with short sleeps in between to allow the database to breathe and autovacuum to catch up.
      • We can’t use OFFSET/LIMIT for deletion loops (it gets slower as it gets to later records). We should use keyset pagination.
      • Package this logic into a cobra command, and then a Kubernetes Job that we/AppSRE can trigger.

              rh-ee-tcreller Tyler Creller
              rh-ee-tcreller Tyler Creller
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated: