-
Bug
-
Resolution: Unresolved
-
Critical
-
None
-
None
-
False
-
None
-
False
-
-
-
Pipelines Sprint Tekshift 26, Pipelines Sprint Tekshift 27
-
Customer Facing, Customer Reported
Description of problem:
When referencing a Remote pipeline using the following parameters, the git resolver uses an excessive amount of memory resulting in the Remote Resolver pod getting OOM killed, and the Remote Resolution Request timing out.
```yaml
pipelineRef:
resolver: git
params:
- name: url
value: https://github.com/red-hat-data-services/RHOAI-Build-Config.git
- name: revision
value: pipeline
- name: pathInRepo
value: pipeline/fbc-fragment-build-pipeline.yaml
```
The repository referenced is only ~145M when cloned with `git clone`, however in my local tests this remote resolution caused a pod with a memory limit of 4G to be OOM killed, and when I wrote a unit test to run `gitresolution.ResolveAnonymousGit()` using these parameters (with -memoryprof, which may add some memory overhead) the unit test consumed over 25G of memory before my computer froze.
The customer was provided the workaround of using the http resolver instead specifying the URL as `https://raw.githubusercontent.com/red-hat-data-services/RHOAI-Build-Config/refs/heads/pipeline/pipeline/fbc-fragment-build-pipeline.yaml`
Upstream Issue, since the direction here requires some community discussion: https://github.com/tektoncd/pipeline/issues/8652
Prerequisites (if any, like setup, operators/versions):
Steps to Reproduce
- Download the attached patch `memory-usage-test.diff`
- In a copy of the tektoncd/pipeline repository clean on origin/main, apply the attached diff memory-usage-test.diff with `git apply /path/to/memory-usage-test.diff`
- Run the test with `go test -v -timeout 20m ./pkg/remoteresolution/resolver/git/... -run TestCrazyMemoryUse`
- Separately, monitor the memory usage of the test using `top` or your preferred memory monitoring utility
Actual results:
The test will use an obscene amount of memory (and it remains to be seen if it even completes)
Expected results:
The test should complete in a timely manner and should not use much memory given the size of the repo is less than 200M
Reproducibility (Always/Intermittent/Only Once):
Always
Acceptance criteria:
Definition of Done:
Build Details:
Additional info (Such as Logs, Screenshots, etc):