Epic Goal

...

The application-manager long lived secret is created by app addon on OCP >=4.16 managed clusters for the ArgoCD push model usage.

, we are planning to enhance it in next 2.13 release. The proposal is
app addon detects if there is token for the application-manager SA on the managed cluster
If no token found, it won't create the long lived secret on the managed cluster. Instead. It will create a managedServiceAccount with 7 days expiration in the hub cluster NS

apiVersion: authentication.open-cluster-management.io/v1beta1
kind: ManagedServiceAccount
metadata:
  name: application-manager
  namespace: <cluster name>
spec:
  rotation:
    enabled: true
    validity: 168h0m0s            ===========> 7 days expiration

managedServiceAccount agent on the managed cluster will be reconciled to generate a short lived token (7 days per managedServiceAccount rotation spec) though k8s tokenRequest api, the token is saved as a Opaque type secret (not the kubernetes.io/service-account-token type) in the hub cluster namespace, and it is rotated when its remaining lifetime is less than 20% of its rotation validity.
After the SA token is rotated, the ArgoCD cluster secret on the hub cluster will be synced up
Also the app addon agent will delete the existing long lived token if found on the managed cluster

Why is this important?

...

Scenarios

Goal test the ArgoCD push model cluster secret(secret used to connect hub to managed cluster)
Prerequiste

• ArgoCD operator installed on hub
• Create Gitopscluster CR
• Create a managed cluster with OCP version >= 4.16

Test #1
1. Import managed cluster to hub.
2. Verify a ManagedServiceAccount CR named application-manager is created in the managed cluster namespace.
3. Look at the ManagedServiceAccount YAML status.tokenSecretRef.name, verify a Secret CR with this name is created in the managed cluster namespace.
4. Look at the Secret YAML data.token from step 3, verify the token is the same as the data.config bearerToken used in the openshift-gitops Secret with name <clister_name>-application-manager-cluster-secret.
Note: base64 decode data.config into JSON format to see the bearerToken. The secret token in step 3 also needs to be base64 decoded
5. Deploy a ArgoCD push model application to the managed cluster, verify deploy is successful.

Test #2
1. Using the same ManagedServiceAccount CR application-manager from Test #1, delete this ManagedServiceAccount CR.
2. Create a new ManagedServiceAccount CR with the same name in the same namespace but with a spec.rotation.validity of only 10 mins.
apiVersion: authentication.open-cluster-management.io/v1beta1
kind: ManagedServiceAccount
metadata:
 name: application-manager
 namespace: <cluster_name>
spec:
 rotation:
   enabled: true
   validity: 0h10m0s
3. The ManagedServiceAccount will rotate the secret when there is ~20% time left. Verify the status.tokenSecretRef.lastRefreshTimestamp when this happens.
4. Do the same verification as in Test #1 step 4 to make sure the tokens match.
5. Make sure the ArgoCD application is still working when the rotation happens.
Note: in step 1 when you delete the ManagedServiceAccount CR all the ArgoCD Application CRs targeting this cluster will be deleted on the hub but the application resources on the managed cluster should still exist and when the Application CR is created again it will take ownership of the resources automatically

Test #3 - Upgrade test
1. Create a managed cluster with OCP version 4.15.
2. Deploy an ArgoCD push model application to the managed cluster.
2. Upgrade the cluster to 4.16.
3. Verify using Test #1 steps 2-4.
4. Verify the ArgoCD application from step 2 is still working.
Note: This is because OCP <= 4.15, the managed cluster still generates a token for the ServiceAccount to use but OCP >= 4.16 stopped doing this. So for OCP >= 4.16 we used to generate a long lived token(ie. never expires) but in this feature we changed the token to a short lived one(expires every 7 days)

Acceptance Criteria

...

Dependencies (internal and external)

1. ...

Previous Work (Optional):

1. ...

Open questions:

1. …

Done Checklist

• CI - CI is running, tests are automated and merged.
• Release Enablement <link to Feature Enablement Presentation>
• DEV - Upstream code and tests merged: <link to meaningful PR or GitHub
  Issue>
• DEV - Upstream documentation merged: <link to meaningful PR or GitHub
  Issue>
• DEV - Downstream build attached to advisory: <link to errata>
• QE - Test plans in Polarion: <link or reference to Polarion>
• QE - Automated tests merged: <link or reference to automated tests>
• DOC - Doc issue opened with a completed template. Separate doc issue
  opened for any deprecation, removal, or any current known
  issue/troubleshooting removal from the doc, if applicable.