keda-operator is missing files in /usr/share/zoneinfo from tzdata causing cron triggers with timezone set to fail

custom-metrics-autoscaler.v2.10.1-253 is failing with cron triggers when they have a timezone set as the keda-operator is missing the necessary tzdata files under /usr/share/zoneinfo. Those are being removed in the ubi8-minimal and only UTC is left there.

So the below scaledobject will cause an error as reported in keda-operator

#
$ oc get scaledobject -n project-2 -o yaml cron-scaledobject
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  labels:
    scaledobject.keda.sh/name: cron-scaledobject
  name: cron-scaledobject
  namespace: project-2
spec:
  scaleTargetRef:
    name: curl-ip
  triggers:
  - metadata:
      desiredReplicas: "2"
      end: '*/10 * * * *'
      start: '*/5 * * * *'
      timezone: Europe/Zurich
    type: cron
#

The error reported is as below:

2023-06-21T11:47:52Z	ERROR	scale_handler	error getting scale decision	{"scaledObject.Namespace": "project-2", "scaledObject.Name": "cron-scaledobject", "scaler": "cronScaler", "error": "unable to load timezone. Error: unknown time zone Europe/Zurich"}

When rebuilding the keda-operator image and re-install tzdata to have the files in /usr/share/zoneinfo the scaledobject can be successfully reconciled.

#
$ cat Dockerfile 
FROM registry.redhat.io/custom-metrics-autoscaler/custom-metrics-autoscaler-rhel8:latest

USER root
RUN microdnf reinstall -y tzdata
#

The image is available on quay.io/rhn_support_sreber/cma:latest and when running the same the logs in keda-operator are as following:

2023-06-21T14:58:38Z	INFO	Reconciling ScaledObject	{"controller": "scaledobject", "controllerGroup": "keda.sh", "controllerKind": "ScaledObject", "ScaledObject": {"name":"cron-scaledobject","namespace":"project-2"}, "namespace": "project-2", "name": "cron-scaledobject", "reconcileID": "6d3ae923-d3d1-4a16-9061-4601d07a2061"}

 - custom-metrics-autoscaler.v2.10.1-253
 - OpenShift Container Platform 4.12

 - Always

1. Install OpenShift Container Platform 4.12 and Custom Metrics Autoscaler as per documentation (does not matter platform and install method)
2. Create a ScaledObject with a cron trigger as shown above
3. Check keda-operator logs in openshift-keda namespace

2023-06-21T11:47:52Z	ERROR	scale_handler	error getting scale decision	{"scaledObject.Namespace": "project-2", "scaledObject.Name": "cron-scaledobject", "scaler": "cronScaler", "error": "unable to load timezone. Error: unknown time zone Europe/Zurich"}

2023-06-21T14:58:38Z	INFO	Reconciling ScaledObject	{"controller": "scaledobject", "controllerGroup": "keda.sh", "controllerKind": "ScaledObject", "ScaledObject": {"name":"cron-scaledobject","namespace":"project-2"}, "namespace": "project-2", "name": "cron-scaledobject", "reconcileID": "6d3ae923-d3d1-4a16-9061-4601d07a2061"}