Summary

We need a supported way to disable glibc rseq when the process runs with AT_SECURE (e.g. when Linux capabilities are set), so that tcmalloc's per-CPU caches can use rseq on RHEL 9.6.

Problem

After upgrading to RHEL 9.6, Google tcmalloc's per-CPU caches stopped working. This matches known issues (e.g. https://github.com/google/tcmalloc/issues/144 and Red Hat article https://developers.redhat.com/articles/2022/12/22/restartable-sequences-support-glibc-rhel-9#): glibc registers its own rseq, which conflicts with tcmalloc's use of rseq. The suggested workaround is to disable glibc rseq via the glibc.pthread.rseq GLIBC_TUNABLES. However, GLIBC_TUNABLES are ignored when the process runs in secure mode (AT_SECURE), which is the case when our process has capabilities. We therefore cannot use this workaround.

Impact

Our application has a lot of threads which benefits (in terms of performance) from per-CPU caches in Google tcmalloc v/s using per-thread caches.

Upstream thread: Need a mechanism to disable glibc rseq for binaries with capabilities