-
Bug
-
Resolution: Unresolved
-
Critical
-
3.1.0.GA
-
None
-
False
-
None
-
False
-
Not Required
devfile registry requires a bootstrapping stage to collect some files for use in brew.
https://github.com/redhat-developer/devspaces-images/blob/devspaces-3-rhel-8/devspaces-devfileregistry/get-sources.sh
https://github.com/redhat-developer/devspaces-images/blob/devspaces-3-rhel-8/devspaces-operator/Dockerfile
So... we would switch to adding a fetch-artifacts-url.yaml file to the mid and downstream repos, per the docs? if not, a fallback method might be to commit the contents of the devspaces-samples project repos directly into the midstream GH repo, and sync it downstream to pkgs.devel repos.
https://osbs.readthedocs.io/en/latest/users.html#fetch-artifacts-url-yaml
Could also use cachito w/o a package manager to simply fetch sources from the dozens of devspaces-samples GH projects (and a commit SHA) so that the sources are available in the downstream repo at build time.
Contents of the sources file is right now:
SHA512 (root-local.tgz) = 4ce440fb02375aa5981cad3175003651a20bd91ada1383779bc8ffb9ff20d115705ca0cfdf298ba32cab3db64b944385e0dcba4d351f4f8d21d335576e945cad SHA512 (resources.tgz) = a42df4c30e4d1bd75eb5b61dde8b80d3f691665023bc1ba02d246998987809080292da09438fa06c25fcb29f8b5ad8216d79237eba1b704c534c476e24c15cfc
These are created from bootstrap.Dockerfile, then we collect these files/folders:
- root-local.tgz :: /opt/app-root/src/.local/bin/
- root-local.tgz :: /opt/app-root/src/.local/lib/
- resources.tgz :: /build/devfiles
- resources.tgz :: /build/resources
Note that we have two similar dockerfiles:
- clones
-
-
- Closed
-
- links to
- mentioned on
(1 links to, 1 mentioned on)
[CRW-3196] devfileregistry :: migrate to cachito using devspaces-samples repos in container.yaml
Build OK in https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=51869873 - only takes about 20 mins to run in OSBS (when there's no queue)
Refactored/cleaned up some stuff, checking if I broke anything in:
https://brewweb.engineering.redhat.com/brew/taskinfo?taskID=51877850
==> https://pkgs.devel.redhat.com/cgit/containers/devspaces-devfileregistry/commit/?h=private-nboldt-cachito&id=2ad40055b253c532b912b46d2a659b60d49871f0
==> registry-proxy.engineering.redhat.com/rh-osbs/devspaces-devfileregistry-rhel8:devspaces-3-rhel-8-containers-candidate-78866-20230407012722
--------
Next steps:
- merge jenkins PR; verify container.yaml updater works in Jenkins (next week?)
- add logic to sync.sh so that 3.y tag is updated in brew.Dockerfile to align to correct version - `FROM registry-proxy.engineering.redhat.com/rh-osbs/devspaces-pluginregistry-rhel8:3.6 as pluginregistrybuilder`
- remove https://github.com/redhat-developer/devspaces/blob/devspaces-3-rhel-8/dependencies/che-devfile-registry/versions.json#L2
- port brew.Dockerfle into upstream/midstream, keeping with CRW-4043 goals, so we have a working rhel.Dockerfile too
- remove @eclipse-che/plugin-registry-generator (we don't consume it, we build it now) from
- job-config.json
- ./updateVersionAndRegistryTags.sh
- ./devspaces-pluginregistry/build.sh
GitLab CEE Bot
added a comment - Nick Boldt mentioned this issue in a merge request of codeready-workspaces / crw-jenkins on branch CRW-3196_updateContainerYamlSHAs.sh:
[ci skip] update 3.6/3.x jobs to use new updateContainerYamlSHAs.sh script for...
GitLab CEE Bot
added a comment - Nick Boldt mentioned this issue in a merge request of codeready-workspaces / crw-jenkins on branch CRW-3196_updateContainerYamlSHAs.sh : [ci skip] update 3.6/3.x jobs to use new updateContainerYamlSHAs.sh script for... Removed both these flags:
- --plugin-registry-url:file:///build/ \
- --editor-entry:che-incubator/che-code/latest \
keeping only:
--editor-path:"/build/plugins/che-incubator/che-code/latest/devfile.yaml" \
Aaaaand... we have a container:
- registry-proxy.engineering.redhat.com/rh-osbs/devspaces-devfileregistry-rhel8:devspaces-3-rhel-8-containers-candidate-99798-20230406135044
However...
+ che-devworkspace-generator --devfile-path:/remote-source/dotnet-web-simple/app/devfile.yaml --editor-path:/build/plugins/che-incubator/che-idea/latest/devfile.yaml --output-file:./devfiles/TP__dotnet__dotnet-web-simple//devworkspace-che-idea-latest.yaml '--project.dotnet-web-simple={{_INTERNAL_URL_}}/resources/v2/dotnet-web-simple.zip' No plug-in registry url. Setting to https://eclipse-che.github.io/che-plugin-registry/main/v3 DevWorkspace che-idea-dotnet was generated. + echo '[brew.generate_devworkspace-templates.sh] Zip /remote-source/dotnet-web-simple/app' + echo '[brew.generate_devworkspace-templates.sh] to /build/resources/v2/dotnet-web-simple.zip ... ' + pushd /remote-source/dotnet-web-simple/app/ + zip -9r /build/resources/v2/dotnet-web-simple.zip . [brew.generate_devworkspace-templates.sh] Zip /remote-source/dotnet-web-simple/app [brew.generate_devworkspace-templates.sh] to /build/resources/v2/dotnet-web-simple.zip ... adding: .gitignore (stored 0%) ... adding: devfile.yaml (deflated 61%) adding: web.csproj (deflated 37%) + popd
Strange to be setting a default value for plugin-registry-url, if it's not going to be used (because no editor-entry is set). But if the build works, I'm happy from this end.
Tried using both file:/ and file:/// which didn't work either:
+ che-devworkspace-generator --devfile-path:/remote-source/lombok-project-sample/app/devfile.yaml --plugin-registry-url:file:/build/ --editor-path:/build/plugins/che-incubator/che-code/latest/devfile.yaml --editor-entry:che-incubator/che-code/latest --output-file:./devfiles/java11-maven-lombok__lombok-project-sample/devworkspace-che-code-latest.yaml '--project.lombok-project-sample={{_INTERNAL_URL_}}/resources/v2/lombok-project-sample.zip' stack=Error: connect ECONNREFUSED ::1:80 at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1159:16) Unable to start Error: connect ECONNREFUSED ::1:80 at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1159:16) { errno: -111, code: 'ECONNREFUSED', syscall: 'connect', address: '::1', port: 80, config: { url: 'file:/build//plugins/che-incubator/che-code/latest/devfile.yaml',
and
+ che-devworkspace-generator --devfile-path:/remote-source/lombok-project-sample/app/devfile.yaml --plugin-registry-url:file:///build/ --editor-path:/build/plugins/che-incubator/che-code/latest/devfile.yaml --editor-entry:che-incubator/che-code/latest --output-file:./devfiles/java11-maven-lombok__lombok-project-sample/devworkspace-che-code-latest.yaml '--project.lombok-project-sample={{_INTERNAL_URL_}}/resources/v2/lombok-project-sample.zip' stack=Error: connect ECONNREFUSED 127.0.0.1:80 at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1159:16) Unable to start Error: connect ECONNREFUSED 127.0.0.1:80 at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1159:16) { errno: -111, code: 'ECONNREFUSED', syscall: 'connect', address: '127.0.0.1', port: 80, config: { url: 'file:///build//plugins/che-incubator/che-code/latest/devfile.yaml',
[2/3] STEP 29/36: RUN ls -la plugins/*/*/*/devfile.y*ml; ./brew.generate_devworkspace_templates.sh && chmod -R g+rwX /build/resources && rm -fr /build/plugins/ -rw-rwxr--. 1 root root 2134 Apr 5 20:57 plugins/che-incubator/che-code/insiders/devfile.yaml -rw-rwxr--. 1 root root 2134 Apr 5 20:57 plugins/che-incubator/che-code/latest/devfile.yaml -rw-rwxr--. 1 root root 2785 Apr 5 20:57 plugins/che-incubator/che-idea/latest/devfile.yaml +++ mktemp -d -u ++ TEMP_REPO=/tmp/tmp.Kp10kClz9K ++ find . -name job-config.json + jobconfigjson=./job-config.json + [[ -f ./job-config.json ]] ++ yq -r .Version ./job-config.json + VERSION=3.6 + [[ -z 3.6 ]] + echo '[brew.generate_devworkspace-templates.sh] Generating devworkspace templates for DS 3.6 from locally cached content ... ' + che_devworkspace_generator=che-devworkspace-generator + mkdir -p ./resources/v2/ [brew.generate_devworkspace-templates.sh] Generating devworkspace templates for DS 3.6 from locally cached content ... ++ pwd + WORKDIR=/build + for dir in ./devfiles/*/ ++ grep v2: ./devfiles/java11-maven-lombok__lombok-project-sample/meta.yaml + devfile_url=' v2: https://github.com/devspaces-samples/lombok-project-sample/tree/devspaces-3-rhel-8' + '[' -n ' v2: https://github.com/devspaces-samples/lombok-project-sample/tree/devspaces-3-rhel-8' ']' + devfile_url=https://github.com/devspaces-samples/lombok-project-sample/tree/devspaces-3-rhel-8 + devfile_url=https://github.com/devspaces-samples/lombok-project-sample/tree/devspaces-3-rhel-8 + devfile_url=https://github.com/devspaces-samples/lombok-project-sample/tree/devspaces-3-rhel-8 + devfile_repo=https://github.com/devspaces-samples/lombok-project-sample ++ basename https://github.com/devspaces-samples/lombok-project-sample + name=lombok-project-sample + project='lombok-project-sample={{_INTERNAL_URL_}}/resources/v2/lombok-project-sample.zip' + npm_config_yes=true + che-devworkspace-generator --devfile-path:/remote-source/lombok-project-sample/app/devfile.yaml --editor-path:/build/plugins/che-incubator/che-code/latest/devfile.yaml --editor-entry:che-incubator/che-code/latest --output-file:./devfiles/java11-maven-lombok__lombok-project-sample/devworkspace-che-code-latest.yaml '--project.lombok-project-sample={{_INTERNAL_URL_}}/resources/v2/lombok-project-sample.zip' No plug-in registry url. Setting to https://eclipse-che.github.io/che-plugin-registry/main/v3 stack=Error: connect ECONNREFUSED 185.199.109.153:443 at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1159:16) Unable to start Error: connect ECONNREFUSED 185.199.109.153:443 at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1159:16) { errno: -111, code: 'ECONNREFUSED', syscall: 'connect', address: '185.199.109.153', port: 443, config: { url: 'https://eclipse-che.github.io/che-plugin-registry/main/v3/plugins/che-incubator/che-code/latest/devfile.yaml',
+ che-devworkspace-generator --devfile-url:https://github.com/devspaces-samples/lombok-project-sample/tree/devspaces-3-rhel-8 --editor-entry:che-incubator/che-code/latest --plugin-registry-url:https://redhat-developer.github.io/devspaces/che-plugin-registry/3.6/x86_64/v3 --output-file:./devfiles/java11-maven-lombok__lombok-project-sample/devworkspace-che-code-latest.yaml '--project.lombok-project-sample={{_INTERNAL_URL_}}/resources/v2/lombok-project-sample.zip' stack=Error: connect ECONNREFUSED 185.199.111.133:443 at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1159:16) Unable to start Error: connect ECONNREFUSED 185.199.111.133:443 at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1159:16) { errno: -111, code: 'ECONNREFUSED', syscall: 'connect', address: '185.199.111.133', port: 443, config: { url: 'https://raw.githubusercontent.com/devspaces-samples/lombok-project-sample/devspaces-3-rhel-8/devfile.yaml', method: 'get', headers: { Accept: 'application/json, text/plain, */*', 'User-Agent': 'axios/0.21.2' }, transformRequest: [ [Function: transformRequest] ], transformResponse: [ [Function: transformResponse] ], timeout: 0, adapter: [Function: httpAdapter], responseType: 'text', xsrfCookieName: 'XSRF-TOKEN', xsrfHeaderName: 'X-XSRF-TOKEN', maxContentLength: -1, maxBodyLength: -1, validateStatus: [Function: validateStatus], transitional: { silentJSONParsing: true, forcedJSONParsing: true, clarifyTimeoutError: false }, data: undefined },
trying a workaround for adding ./bin to the PATH:
(waiting 48 mins for build to start...)
Disabled lint, format, and test: just running compile:
[1/2] STEP 26/38: WORKDIR $REMOTE_SOURCES_DIR/devworkspace-generator/app/tools/devworkspace-generator
[1/2] STEP 27/38: RUN yarn compile
yarn run v1.22.19
$ tsc --declaration --project .
/bin/sh: tsc: command not found
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
error Command failed with exit code 127.
All tests passed with Valerii's PR to remove theia/sidecars!
repo: https://github.com/eclipse-che/che-plugin-registry.git
ref: 6b013202eee97c3a2ed6ad7701559643af4cd8f3
Next problem:
+ echo 'Generating devworkspace templates for DS 3.6 ... ' ++ uname -m + arch=x86_64 + che_devworkspace_generator=che-devworkspace-generator ++ jq -r --arg v che-devworkspace-generator '.[$v]' versions.json + npm install @eclipse-che/che-devworkspace-generator@0.0.1-c8bd5c0 npm ERR! code ECONNREFUSED npm ERR! errno ECONNREFUSED npm ERR! FetchError: request to https://registry.npmjs.org/@eclipse-che%2fche-devworkspace-generator failed, reason: connect ECONNREFUSED 104.16.21.35:443
New attempt to build w/ devfile registry's devworkspace-generator:
[1/2] STEP 26/38: WORKDIR $REMOTE_SOURCES_DIR/devworkspace-generator/app/tools/devworkspace-generator [1/2] STEP 27/38: RUN yarn build yarn run v1.22.19 $ yarn run format && yarn run compile && yarn run lint && yarn run test $ if-env SKIP_FORMAT=true && echo 'skip format check' || prettier --check '{src,tests}/**/*.ts' package.json /bin/sh: if-env: command not found /bin/sh: prettier: command not found error Command failed with exit code 127.
Seems like we have another "backport to yarn v1" problem here.
> Is there anything needed from QE side on this one?
No, the task here is to complete the migration of "grab stuff from GH into a tarball" to "grab stuff from GH via Cachito configuration". It's been on my TODO list for a while, but CVEs and more urgent GA dates have conspired to push this to the back burner.
And on days where I could work on this, Brew is having another outage, so... can't really do much