As a developer I would like to run a small set of tests without needing to rebuild the binary, so I can increase my developer experience/productivity.

DESCRIPTION

To use the DEV_MODE env var on plugins, we need to change the plugins manifests and rebuild the CLI. It is very useful when I am trying other features on CLI or plugins, to make sure the complete flow is running, but with a small set of e2e (sometimes we don't need to check the e2e results, we want to know the flow will work).

But it's kindy boring to change the manifests everytime and rebuild the CLI, it could be more useful when setting a CLI flag to run in "DEV MODE".

The quick implementation was done when running the upgrade feature, but a final change need a more understanding of the impact of allowing this by default.

Some points to review:

• can we keep this flag hidden for users?
• how we will invalidate the results (through the support case) when ran in dev mode?

ACCEPTANCE CRITERIA:

• Make sure the Flag is hidden on CLI
• Make sure we can get this information on the 'process' (review subcommand)
•  

ENGINEERING DETAILS

Reference implementation used on https://github.com/redhat-openshift-ecosystem/provider-certification-tool/pull/33:

1) Create the flag

cmd.Flags().StringVar(&o.devCount, "dev-count", "0", "Developer Mode only: run small random set of tests. Default: 0 (disabled)") 

2) set it as hidden

cmd.Flags().MarkHidden("dev-count")

3) Add the flag value to the Plugins Config

    if err := r.createConfigMap(kclient, sclient, &v1.ConfigMap{
        ObjectMeta: metav1.ObjectMeta{
            Name:      pkg.PluginsVarsConfigMapName,
            Namespace: pkg.CertificationNamespace,
        },
        Data: map[string]string{
(..)
            "dev-count":             r.devCount,
        },
    }); err != nil {
        return err
    } 

4) Define the env var using this config maps on each conformance plugin manifest (container 'plugin')

    - name: DEV_MODE_COUNT
      valueFrom:
        configMapKeyRef:
          name: plugins-config
          key: dev-count
  

5) Run the CLI - the plugin already support the DEV_MODE env var since v0.1, and it assumes != 0 is the regular execution:

./opct run -w --dev-count 10