Uploaded image for project: 'Subscription Watch'
  1. Subscription Watch
  2. SWATCH-2186

Investigate Performance Optimization For Export Fetching

XMLWordPrintable

    • Icon: Story Story
    • Resolution: Done
    • Icon: Normal Normal
    • None
    • None
    • None

      For this card, we want to investigate how much impact on memory will be use if we send export JSON as a stream to export service for upload.

      Goal:
      We want memory usage for export usage to be the same for both scenarios we want constant memory :
      1. Single subscription 
      2. Pick a large number 10,000 + subscriptions 

      Requirements:

      Look at the generated class to add the `existingJavaType` yaml field for the export model class :

      Example: java.util.stream.Stream<org.candlepin.subscriptions.json.Subscription>
      Keep in mind that we may have to refactor yaml to properly generate streams class for Json body 

      • Leverage profiler tools to track how java Stream API would effect memory usage and ways to optimize implementation.
      • Create a unit test to track the memory performance for streams of POJO's  with jsonschema2pojo leverage existing Types
      • We want to use jackson to write to a tmp file and than upload file to export API client. May need to tweak here and to watch out for client causing memory to be consumed
      • Use File as the body (and it does upload properly), seems to have a 50MB cap, we can probably live with that for the short term at least.

      Testing:

      Verify Memory usage performance using profiler for memory benchmark.
      Do we want to bring perf (optional)? - Work with performance team to set up Continuous Performance Test (CPT) in this area. Yes: SWATCH-2123

              jcarvaja@redhat.com Jose Carvajal Hilario
              ksynvrit Kenny Synvrit (Inactive)
              Sanket Jagtap Sanket Jagtap
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

                Created:
                Updated:
                Resolved: