Uploaded image for project: 'Hybrid Cloud Console'
  1. Hybrid Cloud Console
  2. RHCLOUD-43678

Stress test relations checks during bulk delete

XMLWordPrintable

    • Product / Portfolio Work
    • 8
    • False
    • Hide

      None

      Show
      None
    • False
    • None
    • Unset
    • None

      If we need to do bulk delete on millions of records in prod against spicedb, we need to make sure it does not cause the spicedb postgres database to lock or cause significant strain on our resources when other service providers are making check calls.

      Ideally we would test this by inserting millions of host records into an ephemeral or perf setup, executing a bulk delete, and then simultaneously making check calls against non-host resources. This will help verify whether bulk delete is a prod safe operation and does not bring down authz.

      Kessel has some tools to help perform this test, it can be done as part of cleaning up a migration test

      Basically, the high level process would look like this:
      0. Complete a HBI Host migration process
      1. Seed SpiceDB with data using the Data Generator tool
      2. Start the Relations Load Generator tool to simulate customer traffic
      3. Perform a bulk-delete of all hbi/host relationships using zed cli

      zed relationship bulk-delete hbi/host --force

      5. Capture metrics on Inventory, Relations, SpiceDB, and Database
      6. When bulk delete is done, stop the relations load generator job by deleting it

              anatale.openshift Antony Natale
              rh-ee-tcreller Tyler Creller
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated: