During regression testing for version 1.19, we observed significant inconsistencies in the signing duration times across multiple runs of the same test scenarios.

• For 10,000 PR with HA:

• • Run 1: ~1400s

• • Run 2: ~2000s

• For 5,000 PR:

• • Run 1: ~390s

• • Run 2: ~600s

While minor variation (±5-10%) is expected due to environmental factors, the observed discrepancies (up to ~60%) are abnormal and not acceptable. This indicates a possible underlying issue affecting the consistency and reproducibility of results.

Expected Result:

• Signing duration should remain relatively consistent across repeated runs (only minor acceptable variations).

Actual Result:

• Large fluctuations in signing duration were observed (difference ranging from 300s to 700s for identical test cases).

Impact:

• Inconsistent performance metrics make it difficult to benchmark and validate system performance.

Action Required:

• Investigate the root cause of the inconsistency in test results.

• Confirm whether the variation is due to environment, configuration, or code changes.

• Provide a fix or mitigation to ensure stable and reproducible performance results.