-
Story
-
Resolution: Unresolved
-
Normal
-
None
-
None
-
None
-
Quality / Stability / Reliability
-
False
-
-
False
-
None
-
None
-
None
-
None
Description
By following the doc: https://docs.google.com/document/d/1cFZj9QdzW8hbHc3H0Nce-2xrJMtpDJrwAse9H7hLiWk/edit?tab=t.0#heading=h.66y4kqbj468a and creating a binary, we have the update option to generate metadata to ensure unique IDs.
However, this metadata will be created using the full user machine path instead of the relative path such as:
```
[ { "name": "...", ... "codeLocations": [ "/Users/camilam/go/src/github/operator-framework-operator-controller/openshift/tests-extension/test/olmv1.go:12" ] } ]
```
Then, it does not allow us to properly use it since 2 or more people using the command will not provide an idempotent result. Therefore, we cannot commit the file, do checks and validations to ensure that it was merged, etc.
More info: https://redhat-internal.slack.com/archives/C07RDCVEYJG/p1751637388362499
The same also occur when we call OTE list bin, example:
$ ./bin/olmv1-tests-ext list [ { "name": "[sig-olmv1] OLMv1 should pass a trivial sanity check", "labels": {}, "resources": { "isolation": {} }, "source": "openshift:payload:olmv1", "codeLocations": [ "/Users/camilam/go/src/github/operator-framework-operator-controller/openshift/tests-extension/test/olmv1.go:12", "/Users/camilam/go/src/github/operator-framework-operator-controller/openshift/tests-extension/test/olmv1.go:13" ], "lifecycle": "blocking", "environmentSelector": {} } ]
Impact
According to: https://github.com/openshift/enhancements/blob/master/enhancements/testing/openshift-tests-extension.md#test-id, the TestID must be unique, and we cannot only rename or remove tests, we must ensure traceability.
What are the problem faced if we don’t enforce unique ID names over time? Specifically, what issues might arise if we change the name or the it("description") and don’t track this information anywhere? What kinds of problems could we face as a result?
Suggestion
Use the relative path instead
- blocks
-
OPRUN-3992 Define a Development-Friendly Strategy for OTE Test Metadata and update Usage
-
- Closed
-
- links to