Uploaded image for project: 'Operator Runtime'
  1. Operator Runtime
  2. OPRUN-3992

Define a Development-Friendly Strategy for OTE Test Metadata and update Usage

XMLWordPrintable

    • Icon: Story Story
    • Resolution: Done
    • Icon: Undefined Undefined
    • None
    • None
    • None
    • None
    • None
    • Nidoran Sprint 274

      Context:
      This issue stems from the discussion in PR #383 of operator-framework-operator-controller, where the update command was suggested to be added in the build target.

      Summary:
      Currently, the update command (olmv1-tests-ext update) is required after building the external test binary to ensure unique TestIDs and generate test metadata. However, this poses significant friction for local development.

      Problem Statement:
      When update is executed locally, it generates metadata with hardcoded developer-specific paths (e.g., /Users/username/...). This breaks workflows where:

      • Test authors want to iterate quickly
      • Specs are renamed or restructured
      • Local development shouldn’t fail due to "missing test" metadata enforcement

      Running update as part of make build causes builds to fail even on small changes — blocking progress and making local iteration cumbersome.

      Example Metadata Output:

      FROM: .openshift-tests-extension

      [
        {
          "name": "[sig-olmv1] OLMv1 should pass a trivial sanity check",
          "labels": {},
          "resources": {
            "isolation": {}
          },
          "source": "openshift:payload:olmv1",
          "codeLocations": [
            "/Users/camilam/go/src/github/operator-framework-operator-controller/openshift/tests-extension/test/olmv1.go:12",
            "/Users/camilam/go/src/github/operator-framework-operator-controller/openshift/tests-extension/test/olmv1.go:13"
          ],
          "lifecycle": "blocking",
          "environmentSelector": {}
        }
      ] 

      Goals / Request:
      Define a strategy that supports both:

      Test ID uniqueness and traceability (required for OTE integration)
      Flexible, error-free local development and iteration

      References:

      NOTES:

      by @Kui Wang:

       
      by the way, FYI. OTE is not good to support
      1, allow case owner to change the case's name
      2,'update' pass to ensure test id unique.
      So I raise the issue with https://redhat-internal.slack.com/archives/C07RDCVEYJG/p1750982390938499.
      the current solution:
      https://redhat-internal.slack.com/archives/C07RDCVEYJG/p1751246356715809?thread_ts=1750982390.938499&cid=C07RDCVEYJG
      the related code and HOW:
      https://redhat-internal.slack.com/archives/C07RDCVEYJG/p1751358584817999?thread_ts=1750982390.938499&cid=C07RDCVEYJG

      currently you do not need to add the code into main function in case there is better solution.
      if there is no better solution, we could add these code into main function late.
      but now need to add update after building binary as the above suggested.
       

              rh-ee-cmacedo Camila Macedo
              rh-ee-cmacedo Camila Macedo
              None
              None
              None
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Created:
                Updated:
                Resolved: