Uploaded image for project: 'Subscription Watch'
  1. Subscription Watch
  2. SWATCH-2327

Measure performance/limit of entities we can export

XMLWordPrintable

    • Icon: Task Task
    • Resolution: Duplicate
    • Icon: Undefined Undefined
    • None
    • None
    • None
    • False
    • Hide

      None

      Show
      None
    • False
    • BIZ-356 - Export of Subscription Watch data

      The idea is to measure how many data we can export using the Export API in JSON and CSV format.

      The general use case is that an user requests to download either subscriptions or instances data from UI. This interaction happens between the user <> UI <> export service (TBD: to analyze how we can simulate this).
      Then, the export-service asks for the data to swatch and swatch uploads a file in either JSON or CSV to export-service, so the user can download it.

      There are two kind of entities we need to measure:

      • subscriptions (in stage, we have up to 50k entities)
      • instances where there could be hundreds of thousands of systems reporting in

      After SWATCH-2186, we found out that we have a size limit that we can send of 50MB, this means that using mock data we could send up to 200k subscriptions. See more about this analysis in https://github.com/RedHatInsights/rhsm-subscriptions/pull/3145.

      In the export service side, there is no limit, but it strongly depends on the disk size of the container that is handling the upload request. The export service team is going to increase the limit to 500MB as part of RHCLOUD-31487 and they will request also to run some performance testing.

      Acceptance Criteria

      • Analyse what amount of subscriptions we can upload using the current limit of 50MB in JSON and CSV.
      • Analyse what amount of instances we can upload using the current limit of 50MB in JSON and CSV.

      If the number are not sufficient to cope with the SWATCH data, we might need to increase the 50 MB limit using the system property "dev.resteasy.entity.file.threshold".

              Unassigned Unassigned
              jcarvaja@redhat.com Jose Carvajal Hilario
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated:
                Resolved: