Uploaded image for project: 'Red Hat OpenShift Dev Spaces (formerly CodeReady Workspaces) '
  1. Red Hat OpenShift Dev Spaces (formerly CodeReady Workspaces)
  2. CRW-4859

pluginregistry-plugins_3.x job fails to publish vscode plugins

XMLWordPrintable

    • False
    • None
    • False

      Description of problem:

      Pluginregistry-plugins job has been failing lately.
      Some plugins are simply requiring changes in Dockerfile, or configuration in plugin-config.json in redhat-developer/devspaces-vscode-extensions

      Another issue exits, that the job crashes on Jenkins once plugin is built successfully, and the crash seems up to be related to `podman system prune` command invoked in Jenkinsfile: https://gitlab.cee.redhat.com/codeready-workspaces/crw-jenkins/-/blob/master/jobs/DS_CI/pluginregistry-plugins_3.x.jenkinsfile#L83

      Error message would look like:

      22:23:53 [Pipeline] sh*22:23:53* 22:23:53 + podman system prune -a -f*22:23:55* 22:23:55 Deleted Containers*22:23:55* 22:23:55 584f086e19a2767f68e141c4d8a7c5a819d3fd959e4f7970c2eeeb5780cd25ea*22:23:55* 22:23:55 Error: remove ms-vscode.js-debug-builder-id: no such file or directory

      While it seemed originally that the problem would be the failure of `rm` command removing the cid file ->
      https://github.com/redhat-developer/devspaces-vscode-extensions/blob/devspaces-3-rhel-8/build/build.sh#L96
      so the if block was added for it, however it didn't help.

      then after running this job with various "debug" modification to see additional info.
      like placing 'ls -l' around this line https://github.com/redhat-developer/devspaces-vscode-extensions/commit/d273743bada3d08aa890cabc86dff012b14ac11f#diff-67021cdb17a230c48a35b503841c8893957f146914bc66749180c04ecb60481aR96 and seeing that indeed, cid file gets removed every time properly.  Then at some point in jenkins configuration I edited out the prune command, and then the job completed without the error of such kind.

      I don't know wha can cause this, unless I'm missing something super obvious, as I couldn't find any similar errors, or anything about prune command having issues with cid files, adding `--log-level=debug` didn't reveal much of useful information, compared to successful run of prune on other occasions.

      But prune command surely doesn't complete, as it won't produce in the output how many free space was reclaimed (also image not getting deleted, just one container and then error)

      If it's some obscure bug, we should try to get it replicated outside of jenkins, and reach out to it's developers, in the meantime, implement some workaround for it, so that job wouldn't crash for it, and ideally, prune still executing in some way.

      Workaround:

      on Jenkins instance, edit the live configuration of the job, and disable podman prune command. this temporary fix would allow successfully built plugins to be published to rcm-tools, however proper "long term" workaround would have to be implemented, especially if building lots of plugins (lot's of images) that we would want to prune properly

      Prerequisites (if any, like setup, operators/versions):

      Steps to Reproduce

      Run https://main-jenkins-csb-crwqe.apps.ocp-c1.prod.psi.redhat.com/job/DS_CI/job/pluginregistry-plugins_3.x/ with default parameters, for fast builds, enter a single working plugin like "atlassian.atlascode" as a parameter to reproduce podman error

      Actual results:
      Job fails to build and publish plugins

      Expected results:

      Job fails either to build plugins, or fails to publish built plugins due to podman error

      Reproducibility (Always/Intermittent/Only Once):

      Always

      Acceptance criteria: 

       

      Definition of Done:

      Build Details:

      Additional info (Such as Logs, Screenshots, etc):

       

       *

            mkuznets Mykhailo Kuznietsov
            mkuznets Mykhailo Kuznietsov
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated:
              Resolved: