-
Bug
-
Resolution: Done
-
Critical
-
3.10.0.GA
-
None
-
False
-
None
-
False
-
-
Description of problem:
Pluginregistry-plugins job has been failing lately.
Some plugins are simply requiring changes in Dockerfile, or configuration in plugin-config.json in redhat-developer/devspaces-vscode-extensions
Another issue exits, that the job crashes on Jenkins once plugin is built successfully, and the crash seems up to be related to `podman system prune` command invoked in Jenkinsfile: https://gitlab.cee.redhat.com/codeready-workspaces/crw-jenkins/-/blob/master/jobs/DS_CI/pluginregistry-plugins_3.x.jenkinsfile#L83
Error message would look like:
22:23:53 [Pipeline] sh*22:23:53* 22:23:53 + podman system prune -a -f*22:23:55* 22:23:55 Deleted Containers*22:23:55* 22:23:55 584f086e19a2767f68e141c4d8a7c5a819d3fd959e4f7970c2eeeb5780cd25ea*22:23:55* 22:23:55 Error: remove ms-vscode.js-debug-builder-id: no such file or directory
While it seemed originally that the problem would be the failure of `rm` command removing the cid file ->
https://github.com/redhat-developer/devspaces-vscode-extensions/blob/devspaces-3-rhel-8/build/build.sh#L96
so the if block was added for it, however it didn't help.
then after running this job with various "debug" modification to see additional info.
like placing 'ls -l' around this line https://github.com/redhat-developer/devspaces-vscode-extensions/commit/d273743bada3d08aa890cabc86dff012b14ac11f#diff-67021cdb17a230c48a35b503841c8893957f146914bc66749180c04ecb60481aR96 and seeing that indeed, cid file gets removed every time properly. Then at some point in jenkins configuration I edited out the prune command, and then the job completed without the error of such kind.
I don't know wha can cause this, unless I'm missing something super obvious, as I couldn't find any similar errors, or anything about prune command having issues with cid files, adding `--log-level=debug` didn't reveal much of useful information, compared to successful run of prune on other occasions.
But prune command surely doesn't complete, as it won't produce in the output how many free space was reclaimed (also image not getting deleted, just one container and then error)
If it's some obscure bug, we should try to get it replicated outside of jenkins, and reach out to it's developers, in the meantime, implement some workaround for it, so that job wouldn't crash for it, and ideally, prune still executing in some way.
Workaround:
on Jenkins instance, edit the live configuration of the job, and disable podman prune command. this temporary fix would allow successfully built plugins to be published to rcm-tools, however proper "long term" workaround would have to be implemented, especially if building lots of plugins (lot's of images) that we would want to prune properly
Prerequisites (if any, like setup, operators/versions):
Steps to Reproduce
Run https://main-jenkins-csb-crwqe.apps.ocp-c1.prod.psi.redhat.com/job/DS_CI/job/pluginregistry-plugins_3.x/ with default parameters, for fast builds, enter a single working plugin like "atlassian.atlascode" as a parameter to reproduce podman error
Actual results:
Job fails to build and publish plugins
Expected results:
Job fails either to build plugins, or fails to publish built plugins due to podman error
Reproducibility (Always/Intermittent/Only Once):
Always
Acceptance criteria:
Definition of Done:
Build Details:
Additional info (Such as Logs, Screenshots, etc):
*
- is related to
-
CRW-3160 vscode :: migrate from pkgs.devel lookaside asset files to cachito w/ yarn
- Resolved
- mentioned on