-
Bug
-
Resolution: Done
-
Critical
-
None
-
None
-
3
-
False
-
-
False
-
subs-swatch-lightning
-
-
-
Swatch Lightning Sprint 9
The swatch-tally export consumer (swatch-tally-export consumer group) stopped processing messages from the platform.export.requests Kafka topic after the production deployment on ~Feb 12, 2026.
The Kafka consumer lag for this consumer group is growing continuously and never decreases, which means export requests from users are being queued but never processed. This directly impacts any customer trying to export their subscription data through the export service.
The metric spring_kafka_listener_seconds_count
{container="swatch-tally-service", name="swatch-tally-export-0"}confirms the consumer was working fine until the deployment ~4 days ago, at which point it flatlined (no new messages consumed):
Note that in stage, this is working fine:
The issue seems to be caused by an incompatible issue with resteasy:
2026-02-12T15:25:33.077+0000 [thread=swatch-tally-export-0-C-1] [level=ERROR] [category=org.springframework.kafka.listener.KafkaMessageListenerContainer] - Error while stopping the container java.util.concurrent.CompletionException: java.lang.IllegalAccessError: failed to access class org.jboss.resteasy.spi.EntityOutputStream$EntityInputStream from class org.jboss.resteasy.client.jaxrs.engines.ClientEntityOutputStream$PathHttpEntity (org.jboss.resteasy.spi.EntityOutputStream$EntityInputStream and org.jboss.resteasy.client.jaxrs.engines.ClientEntityOutputStream$PathHttpEntity are in unnamed module of loader 'app') at java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:315) at java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:320) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1807) at java.base/java.lang.Thread.run(Thread.java:1583) Caused by: java.lang.IllegalAccessError: failed to access class org.jboss.resteasy.spi.EntityOutputStream$EntityInputStream from class org.jboss.resteasy.client.jaxrs.engines.ClientEntityOutputStream$PathHttpEntity (org.jboss.resteasy.spi.EntityOutputStream$EntityInputStream and org.jboss.resteasy.client.jaxrs.engines.ClientEntityOutputStream$PathHttpEntity are in unnamed module of loader 'app') at org.jboss.resteasy.client.jaxrs.engines.ClientEntityOutputStream$PathHttpEntity.<init>(ClientEntityOutputStream.java:80) at org.jboss.resteasy.client.jaxrs.engines.ClientEntityOutputStream.toEntity(ClientEntityOutputStream.java:68) at org.jboss.resteasy.client.jaxrs.engines.ManualClosingApacheHttpClient43Engine.buildEntity(ManualClosingApacheHttpClient43Engine.java:519) at org.jboss.resteasy.client.jaxrs.engines.ManualClosingApacheHttpClient43Engine.loadHttpMethod(ManualClosingApacheHttpClient43Engine.java:454) at org.jboss.resteasy.client.jaxrs.engines.ManualClosingApacheHttpClient43Engine.invoke(ManualClosingApacheHttpClient43Engine.java:330) at org.jboss.resteasy.client.jaxrs.internal.ClientInvocation.invoke(ClientInvocation.java:427) at org.jboss.resteasy.client.jaxrs.internal.ClientInvocation.invoke(ClientInvocation.java:71) at org.jboss.resteasy.client.jaxrs.internal.ClientInvocationBuilder.post(ClientInvocationBuilder.java:205) at com.redhat.swatch.clients.export.api.client.ApiClient.invoke(ApiClient.java:703) at com.redhat.swatch.clients.export.api.client.ApiClient.invokeAPI(ApiClient.java:666) at com.redhat.swatch.clients.export.api.resources.ExportApi.downloadExportUpload(ExportApi.java:163) at org.candlepin.subscriptions.export.ExportDelegateImpl.upload(ExportDelegateImpl.java:62) at com.redhat.swatch.export.ExportRequestHandler.uploadData(ExportRequestHandler.java:107) at com.redhat.swatch.export.ExportRequestHandler.handle(ExportRequestHandler.java:70) at org.candlepin.subscriptions.export.ExportRequestConsumer.receive(ExportRequestConsumer.java:60) at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) at java.base/java.lang.reflect.Method.invoke(Method.java:580) at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:360) at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:196) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:89) at io.micrometer.core.aop.TimedAspect.processWithTimer(TimedAspect.java:262) at io.micrometer.core.aop.TimedAspect.perform(TimedAspect.java:227) at io.micrometer.core.aop.TimedAspect.ajc$inlineAccessMethod$io_micrometer_core_aop_TimedAspect$io_micrometer_core_aop_TimedAspect$perform(TimedAspect.java:1) at io.micrometer.core.aop.TimedAspect.timedMethod(TimedAspect.java:218) at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) at java.base/java.lang.reflect.Method.invoke(Method.java:580) at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:649) at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:631) at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:71) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184) at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:380) at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:184) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:728) at org.candlepin.subscriptions.export.ExportRequestConsumer$$SpringCGLIB$$0.receive(<generated>) at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103) at java.base/java.lang.reflect.Method.invoke(Method.java:580) at org.springframework.messaging.handler.invocation.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:169) at org.springframework.kafka.listener.adapter.KotlinAwareInvocableHandlerMethod.doInvoke(KotlinAwareInvocableHandlerMethod.java:45) at org.springframework.messaging.handler.invocation.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:119) at org.springframework.kafka.listener.adapter.HandlerAdapter.invoke(HandlerAdapter.java:78) at org.springframework.kafka.listener.adapter.MessagingMessageListenerAdapter.invokeHandler(MessagingMessageListenerAdapter.java:475) at org.springframework.kafka.listener.adapter.MessagingMessageListenerAdapter.invoke(MessagingMessageListenerAdapter.java:421) at org.springframework.kafka.listener.adapter.RecordMessagingMessageListenerAdapter.onMessage(RecordMessagingMessageListenerAdapter.java:85) at org.springframework.kafka.listener.adapter.RecordMessagingMessageListenerAdapter.onMessage(RecordMessagingMessageListenerAdapter.java:50) at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.doInvokeOnMessage(KafkaMessageListenerContainer.java:2894) at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.invokeOnMessage(KafkaMessageListenerContainer.java:2872) at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.doInvokeRecordListener(KafkaMessageListenerContainer.java:2784) at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.doInvokeWithRecords(KafkaMessageListenerContainer.java:2621) at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.invokeRecordListener(KafkaMessageListenerContainer.java:2510) at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.invokeListener(KafkaMessageListenerContainer.java:2159) at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.invokeIfHaveRecords(KafkaMessageListenerContainer.java:1535) at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.pollAndInvoke(KafkaMessageListenerContainer.java:1473) at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.run(KafkaMessageListenerContainer.java:1342) at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1804)
Acceptance Criteria
- Investigate why the consumer is not working and fix it
- is caused by
-
-
- Closed
-
- is triggering
-
-
- Open
-