Epic Goal

• Perform performance and scale testing for Data Science Pipelines v2 release
• Created from request PRFRSR-66

Why is this important?

• In the past we've seen that data science pipelines are sensitive to scale issues. We need to continue to ensure that reasonable configurations and stress that DSP's might be used with do not cause an unreasonable performance penalty.

Scenarios

1. How many pipelines can be deployed before performance breaks down?
2. Can a reasonable number of pipelines be deployed (~100) without significant performance degradation?

Acceptance Criteria

• DSPv2 code merged and passing CI tests in TOPSAIL
• Tests succesfully run on ScaleLab allocation
• Results agregated and reported concisely for the DSPv2 team (prepare the results)

Dependencies (internal and external)

1. TOPSAIL - pipelines

Previous Work (Optional):

1. Pipeline testing report from DSPv1

Open questions::

1. RHOAIENG-1701 Are we postponing this related task?
2. RHOAIENG-2838 What do we consider the cluster breaking down? Does a critical performance degradation count?

Done Checklist

• CI - CI is running, tests are automated and merged.{}
• DEV - Upstream code and tests merged: <link to meaningful PR or GitHub Issue>