During the addition of AWS EC2 Brew workers, it turned out that the current Splunk logging depends on worker hostnames. This means that the Splunk index that we log into needs to be modified each time we add or remove workers. In addition, our EC2 workers don't have hostname stable in case of re-provisioning.

After some investigation done by Diaa, the Splunk client is now available for all architectures and should be used instead of forwarding rsyslog to Splunk. In addition, there is also an Ansible role to setup Splunk client.

Links:

• https://redhat-internal.slack.com/archives/C04LP3BKBU3/p1691081167256739
• https://gitlab.corp.redhat.com/it-ansible/playbooks/splunk/splunk_client