-
Task
-
Resolution: Done
-
Blocker
-
None
-
None
-
None
The following splunk alert has been triggered for swatch-contracts in Prod:
index=rh_rhsm namespace=rhsm-prod "TooManyMessagesWithoutAckException" | rex field=message ".+has waited for (?<seconds>\d+).*" | table host, seconds, _time
The problem seems to be happening because the Subscription API takes more than 30 seconds to reply:
ERROR [com.red.swa.con.ser.SubscriptionService] (executor-thread-2) Api exception from subscription service: The timeout period of 30000ms has been exceeded while executing GET /svcrest/subscription/v5/search/criteria;web_customer_id=12492279/options;products=ALL;showExternalReferences=true;firstResultIndex=0;maxResults=1000/ for server null
Search:
index=rh_rhsm namespace=rhsm-prod host=swatch-contracts-service-569d744b65-9l9xb "error" NOT "VertxGrpcExporter"
since 3 september and last 24 hours.