-
Task
-
Resolution: Duplicate
-
Undefined
-
None
-
None
-
None
The idea is to measure how many data we can export using the Export API in JSON and CSV format.
The general use case is that an user requests to download either subscriptions or instances data from UI. This interaction happens between the user <> UI <> export service (TBD: to analyze how we can simulate this).
Then, the export-service asks for the data to swatch and swatch uploads a file in either JSON or CSV to export-service, so the user can download it.
There are two kind of entities we need to measure:
- subscriptions (in stage, we have up to 50k entities)
- instances where there could be hundreds of thousands of systems reporting in
After SWATCH-2186, we found out that we have a size limit that we can send of 50MB, this means that using mock data we could send up to 200k subscriptions. See more about this analysis in https://github.com/RedHatInsights/rhsm-subscriptions/pull/3145.
In the export service side, there is no limit, but it strongly depends on the disk size of the container that is handling the upload request. The export service team is going to increase the limit to 500MB as part of RHCLOUD-31487 and they will request also to run some performance testing.
Acceptance Criteria
- Analyse what amount of subscriptions we can upload using the current limit of 50MB in JSON and CSV.
- Analyse what amount of instances we can upload using the current limit of 50MB in JSON and CSV.
If the number are not sufficient to cope with the SWATCH data, we might need to increase the 50 MB limit using the system property "dev.resteasy.entity.file.threshold".
- depends on
-
SWATCH-2020 Add CSV support to instances export
- Closed
-
SWATCH-2021 Report filtered subscription data to Export Service in JSON
- Closed
-
SWATCH-2023 Add CSV support subscriptions export
- Closed
-
SWATCH-2018 Report filtered instance data to Export Service in JSON
- Closed
- is related to
-
SWATCH-2186 Investigate Performance Optimization For Export Fetching
- Closed