EPIC Goal

What are we trying to solve here?

Today it is impossible to export a single dynamic plugin npm package without validating the entire workspace can be built. While this is handy to ensure all the cross-package dependencies are met, it can also result in a lot of wasted time, debugging, yarn lock updates, etc.

It also means that when we want to release a single plugin like in RHDHBUGS-2735 we have to rebuild all 5 orchestrator plugins, and can get errors like this that prevent releasing one package due to problems in another:

FAILED_EXPORTS
plugins/orchestrator-backend
plugins/orchestrator-backend-module-loki
plugins/scaffolder-backend-module-orchestrator

PUBLISHED_EXPORTS
quay.io/rhdh/red-hat-developer-hub-backstage-plugin-orchestrator:5.4.1
quay.io/rhdh/red-hat-developer-hub-backstage-plugin-orchestrator-form-widgets:1.6.2

Possible implementation:

Instead of a single yarn.lock at the monorepo root, we could switch to this sort of structure:

$➔ tree orchestrator-wrappers/
orchestrator-wrappers/
├── metadata
│   ├── redhat-backstage-plugin-orchestrator-backend-module-loki.yaml
│   ├── redhat-backstage-plugin-orchestrator-backend.yaml
│   ├── redhat-backstage-plugin-orchestrator-form-widgets.yaml
│   ├── redhat-backstage-plugin-orchestrator.yaml
│   └── redhat-backstage-plugin-scaffolder-backend-module-orchestrator.yaml
├── redhat-backstage-plugin-orchestrator
│   ├── package.json
│   └── yarn.lock
├── redhat-backstage-plugin-orchestrator-backend
│   ├── package.json
│   └── yarn.lock
├── redhat-backstage-plugin-orchestrator-backend-module-loki
│   ├── package.json
│   └── yarn.lock
├── redhat-backstage-plugin-orchestrator-form-widgets
│   ├── package.json
│   └── yarn.lock
└── redhat-backstage-plugin-scaffolder-backend-module-orchestrator
    ├── package.json
    └── yarn.lock

This would also allow us to patch, use resolutions at the single package level, and version the wrappers separately from the upstream npm packages.

Today we use <RHDH-version>-<plugin-version> for OCI artifact versioning, eg., 1.9.0-1.0.1; under this new scheme we could keep the plugin versions aligned to the upstream (eg., loki plugin 1.0.2 and wrapper loki plugin 1.0.2) while the OCI artifact retains the double-versioning approach.

A future versioning scheme could be to use a semver suffix like 1.0.3-redhat-001 for the wrapper version of 1.0.2, if we also explore publishing these wrappers onto packages.redhat.com. (Since semver considers 1.0.2-suffix to be a pre-release version of 1.0.2, we would of course have to bump the z-digit every time we release this way.)