Follow-up of https://web-rca.devshift.net/incident/ITN-2024-00175

We currently rely on an SSL client certificate used for authentication purposes when we call the IT Users Service. That certificate is stored in the Vault and renewed manually.

As suggested by nmalik-srej the ITN-2024-00175 retrospective doc, we should automate the alerting and renewal process of the certificate. That kind of automation already exists in many areas. We need to examine the existing solutions and possibly leverage them in Notifications.

Things to consider

• Since the certificates are generated in PEM format, we need to have an init container that wraps those in a JWKs container for the Java applications to be able to consume them.
• There will be no need to set up Catchpoint alerts, because AppSRE will be monitoring those certificates and will be creating tickets and sending notifications for them.