Sync of application may fail in rare condition as openshift-gitops-application-controller-0 is using /dev/shm as temporary directory. Since /dev/shm is limited to 64M in space (can't be changed in OpenShift Container Platform 4) it may eventually fill up and report and error similar to the below (which will cause application sync to fail).

Failed sync attempt to 306419123d21df53cbeeef7c289gf4531d579j7h: failed to initialize sync context: failed to write kubeconfig: write /dev/shm/3227598765: no space left on device

The view of the filesystem usage within the pod was as following.

overlay         120G   56G   65G  47% /
tmpfs            64M     0   64M   0% /dev
tmpfs            40G     0   40G   0% /sys/fs/cgroup
shm              64M   64M     0 100% /dev/shm                   <<-- is full by 100%
tmpfs            40G  121M   40G   1% /etc/passwd
/dev/sda4       120G   56G   65G  47% /etc/hosts
tmpfs            40G     0   40G   0% /app/config/controller/tls
tmpfs            40G   44K   40G   1% /run/secrets/kubernetes.io/serviceaccount

Once the pod openshift-gitops-application-controller-0 is re-created/restarted, /dev/shm is also reset and therefore application sync may start to work again. But there are chances that /dev/shm will fill-up again and hence cause the same issue over and over again.

It's therefore required to change the temporary directory from /dev/shm to something more feasible that potentially can even be changed in size.