We have an OpenShift test that fails in about 50% cases when executed on EAP QE Jenkins pipelines.

The mentioned test deploys:

• a PostgreSql service to store timer expirations metadata
• an EAP application service that exposes endpoints to handle such persistence operations
• an EAP application service that exposes two EJB timer beans, one which is transactionally calling the persistence APIs to record its expirations and another one that just logs messages, without storing any expiration metadata. This application service also exposes endpoints for the test class to be able to create and delete timers, and to retrieve information about them. The EJB timer beans are called remotely by such endpoints.

The timer persistence is delegated to the Infinispan subsystem, as per EAP7-1417.

After deploying the scenario, several tests are run, e.g. to verify a timer can be created or deleted successfully, and then a couple of fail-over scenario tests are executed.

The one which is failing is about an high frequency (.5 seconds) distributed timer that is created and executed by a pod which is stopped after some time.
The timer expirations are recorded by the persistence mechanism and we are facing the case where we the actual count of recorded expirations in a period of time is less than the expected one (i.e. 95% of expected timeouts are recorded).

The failures hasn't been noticed when running the same test/configuration locally so far.

Links to internal resources documenting the test behavior are reported as part of the first comments.