Value Statement

The search-indexer component requires >1GB of memory when managing large clusters. The current solution is to increase the memory limits, which isn't ideal.

Details

The increased memory is caused by the initial sync from a managed cluster. This initial sync contains the full state, which gets larger as the cluster has more resources.

The problem is compounded when multiple managed clusters need to do a full sync around the same time (ie. database deleted)

Possible solutions:

1. Update the indexer to parse the incoming payload in small chunks. Code here.
2. Update the collector to send smaller chunks.
3. Improve the logic in requestLimiter.go to account for the request content size and available memory.

Definition of Done for Engineering Story Owner (Checklist)

• ...

Development Complete

• The code is complete.
• Functionality is working.
• Any required downstream Docker file changes are made.

Tests Automated

• [ ] Unit/function tests have been automated and incorporated into the
  build.
• [ ] 100% automated unit/function test coverage for new or changed APIs.

Secure Design

• [ ] Security has been assessed and incorporated into your threat model.

Multidisciplinary Teams Readiness

• [ ] Create an informative documentation issue using the Customer

Portal Doc template that you can access from [The Playbook](

https://docs.google.com/document/d/1YTqpZRH54Bnn4WJ2nZmjaCoiRtqmrc2w6DdQxe_yLZ8/edit#heading=h.9fvyr2rdriby),

and ensure doc acceptance criteria is met.

• Call out this sentence as it's own action:

• [ ] Link the development issue to the doc issue.

Support Readiness

• [ ] The must-gather script has been updated.