-
Task
-
Resolution: Done
-
Major
-
1.6.0
-
None
As exposed in RHIDP-6288 sometimes we get benign warnings that still result in an exit code of 0 for yarn processes, which can obfuscate real problems with excessive log.
To reduce the size of the haystack and make needle-finding easier, we could
- redirect verbose yarn processes to a file instead of the console, then ONLY include the contents of that file in the overall build/test log if something fails
- verify that the default behaviour of subprocesses is to log to file (eg., node-gyp) instead of logging to console to make things quieter by default (perhaps a Yarn env var is being used in the GH action runner?
Some useful sample code to play with exists in the WIP cachi2 onboarding MRs attached to RHIDP-4014 (https://gitlab.cee.redhat.com/rhidp/rhdh/-/merge_requests/184):
RUN echo "=== YARN INSTALL ==="; "$YARN" install --immutable >/tmp/yarn.install.log.txt 2>&1 RUN echo "=== YARN BUILD ==="; "$YARN" build --filter=backend >/tmp/yarn.build.log.txt 2>&1 RUN echo "=== EXPORT DYNAMIC PLUGINS (with --no-install) ==="; "$YARN" export-dynamic --filter=./dynamic-plugins/wrappers/*; \ for d in $(find . -name yarn-install.log); do echo; echo "===== EXPORT DYNAMIC PLUGINS: $d =====>"; cat ${d}; echo "<===== EXPORT DYNAMIC PLUGINS: $d ====="; done; RUN echo "=== DIST-DYNAMIC YARN INSTALLS ==="; for f in $(find ./dynamic-plugins/wrappers/ -maxdepth 2 -type d -name dist-dynamic); do g=${f#*/wrappers/}; echo " > $g"; g=${g//\//_}; \ pushd $f >/dev/null; "$YARN" install --immutable >"/tmp/yarn.install.${g}.log.txt" 2>&1; popd >/dev/null; done ... # dump console logs if they have errors RUN hadFail=0; for d in $(find /tmp -name "yarn*log*" || true) $(find /tmp/xfs-* -name build.log || true); do \ failsCheck="$(grep -E "Failed with errors|Failed to execute command|Failed to install|Failed to download|exit code 1" "$d")"; \ if [[ $failsCheck != "" ]]; then \ echo "==== $d SUMMARY ===="; echo "$failsCheck"; \ echo "==== $d FULL ===="; cat "$d"; echo "==== $d ===="; \ (( hadFail = hadFail +1 )); \ fi; \ done; if [[ $hadFail -gt 0 ]]; then exit $hadFail; fi
Definition of done:
review https://github.com/redhat-developer/rhdh/blob/86d8ca667edd63f154a662086b3d689fed106bf8/.ibm/pipelines/utils.sh#L636 (e2e-test env. setup)
implement quieter logging / log-on-failure-only for the yarn install step (as per ideas above)
Also consider that https://github.com/redhat-developer/rhdh/blame/main/.ibm/images/Dockerfile#L27 installs yarn@latest instead of the version we want from the yarn rc file
can we pin the install to use the same version? (could copy the .yarnrc.yml into the docker context, then read its contents)
and trigger a respin when the yarnrc or .ibm/images/Dockerfile is updated? https://github.com/redhat-developer/rhdh/blob/main/.github/workflows/push-e2e-runner.yaml#L8