Uploaded image for project: 'Cost Management'
  1. Cost Management
  2. COST-4481

Update datatype on report downloader to improve memory usage

XMLWordPrintable

    • Icon: Task Task
    • Resolution: Done
    • Icon: Minor Minor
    • 2023Q4
    • None
    • None
    • None

      QA had detected high memory usage on some columns when executing the report downloader (e.g. https://github.com/project-koku/koku/blob/46cd8b2f0626a9fa0012f7a835b5a5a6d980be4b/koku/masu/external/downloader/aws/aws_report_downloader.py#L121).

       

      The idea to solve that is to change the str data type to StringDtype, which probably would consume less memory, as well as follows Pandas best practices (https://pandas.pydata.org/pandas-docs/stable/user_guide/basics.html#dtypes).

      It is needed to perform this action in all _report_downloader.py files, including the _local_report_downloder.py, to make it consistent.

              mskarbek Michael Skarbek
              rh-ee-lbacciot Lucas Bacciotti
              Pedro Lopez Perez Pedro Lopez Perez
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

                Created:
                Updated:
                Resolved: