Description

We need to automate the validation of plugins and packages metadata to ensure consistency and completeness across the repository. This feature aims to create a
script to enforce these rules programmatically and a CI workflow to run it on Pull Requests.

Requirements

Validation Script

A CLI-executable script (e.g., Python, Node.js, or Shell) that can be easily run locally.

Scope

The script must scan:

• workspaces/<workspace_name>/ for workspace structure and package metadata.
• catalog-entities/extensions/plugins/ for Plugin entity metadata.

Validation Rules:

• Workspace Structure: Verify existence of source.json, plugins-list.yaml, and metadata/*.yaml for each workspace.
• Field Presence: Check for required fields like apiVersion, kind, metadata.name, spec.packageName, spec.version, etc.
• Enumerated Field Validation: Validate that specific fields contain only allowed values.
  • metadata.namespace: rhdh, community
  • spec.backstage.role: frontend-plugin, backend-plugin, scaffolder-module, api-report, common-library
  • spec.support: community, redhat
  • spec.lifecycle: active, experimental, deprecated
  • spec.support.level (Plugin entities): community, certified, supported
• Format Validation: Validate the format of specific fields.
  • spec.version: Semantic Versioning (SemVer) format (e.g., 1.0.0, 1.2.3-alpha.1).
  • spec.backstage.supportedVersions: SemVer range or specific version (e.g., ^1.0.0, 1.2.3).
  • spec.history.added: Date format YYYY-MM-DD.
  • spec.icon: Base64 encoded string (starts with data:image/svg+xml;base64, or similar).
• Reference Integrity:
  • Verify that all packages listed in spec.packages[] in Plugin entities (catalog-entities/extensions/plugins/*.yaml) correspond to existing package
    metadata files in workspaces/*/metadata/*.yaml.
  • Verify that all plugins listed in workspaces/*/plugins-list.yaml have corresponding metadata files in the respective workspaces/*/metadata/ directory.
• YAML Schema Validation:
  • Validate metadata files against the existing JSON schema located at workspaces/extensions/json-schema/plugins.json.

Extensibility: The script should be designed to allow easy addition of new validation checks (e.g., new enumerated values, additional schema constraints) in the
future.

CI Integration

• Create a GitHub Actions workflow (e.g., .github/workflows/validate-metadata.yaml).
• The workflow should trigger on Pull Requests that modify relevant files (workspaces/, catalog-entities/).
• The workflow should run the validation script and fail the build if any checks fail, reporting the specific errors.

Acceptance Criteria

• A script is available in the repository that runs all specified validation checks (structure, enum, format, integrity, schema) and outputs clear error messages.
• A new GitHub Action workflow is created that successfully runs the script on PRs.
• The script can be easily updated to include more fields, allowed values, or format regexes.
• The workflow fails when any validation rule is violated in a PR.