-
Bug
-
Resolution: Done
-
Undefined
-
None
-
False
-
-
False
-
-
Before reporting an issue
[X] I have searched existing issues
[X] I have reproduced the issue with the latest release
Area
storage
Describe the bug
One of our realms contains 8754 clients.
On this realm, we are unable to display the Authentication/Flows screen.
Version
21.1.1
Expected behavior
The Authentication/Flows screen should be usable even when the database contains many clients.
Actual behavior
When accessing Authentication/Flows, the http request on https://identity-test.cosium.com/auth/admin/realms/general/ui-ext/authentication-management/flows hangs for 5 minutes then fails with a 500 error.
On the server side, we see that the request has been killed by the TransactionReaper.
How to Reproduce?
Create a realm holding ~4000 clients, then try to load the authentication flows screen.
Anything else?
Here is the stacktrace that always comes up from thread dumps made while the request is hanging:
```
"executor-thread-3" - Thread t@154
java.lang.Thread.State: RUNNABLE
at org.hibernate.engine.spi.CollectionEntry.preFlush(CollectionEntry.java:172)
at org.hibernate.event.internal.AbstractFlushingEventListener.lambda$prepareCollectionFlushes$0(AbstractFlushingEventListener.java:195)
at org.hibernate.event.internal.AbstractFlushingEventListener$$Lambda$1619/0x0000000800e02440.accept(Unknown Source)
at org.hibernate.engine.internal.StatefulPersistenceContext.forEachCollectionEntry(StatefulPersistenceContext.java:1140)
at org.hibernate.event.internal.AbstractFlushingEventListener.prepareCollectionFlushes(AbstractFlushingEventListener.java:194)
at org.hibernate.event.internal.AbstractFlushingEventListener.flushEverythingToExecutions(AbstractFlushingEventListener.java:86)
at org.hibernate.event.internal.DefaultAutoFlushEventListener.onAutoFlush(DefaultAutoFlushEventListener.java:50)
at org.hibernate.internal.SessionImpl$$Lambda$1560/0x0000000800d89840.accept(Unknown Source)
at org.hibernate.event.service.internal.EventListenerGroupImpl.fireEventOnEachListener(EventListenerGroupImpl.java:107)
at org.hibernate.internal.SessionImpl.autoFlushIfRequired(SessionImpl.java:1372)
at org.hibernate.internal.SessionImpl.scroll(SessionImpl.java:1597)
at org.hibernate.query.internal.AbstractProducedQuery.doScroll(AbstractProducedQuery.java:1588)
at org.hibernate.query.internal.AbstractProducedQuery.scroll(AbstractProducedQuery.java:1574)
at org.hibernate.query.internal.AbstractProducedQuery.stream(AbstractProducedQuery.java:1598)
at org.hibernate.query.Query.getResultStream(Query.java:1140)
at org.keycloak.models.jpa.JpaRealmProvider.getClientScopes(JpaRealmProvider.java:986)
at org.keycloak.storage.ClientStorageManager.getClientScopes(ClientStorageManager.java:212)
at org.keycloak.models.cache.infinispan.RealmCacheSession.getClientScopes(RealmCacheSession.java:1355)
at org.keycloak.models.jpa.ClientAdapter.getClientScopes(ClientAdapter.java:366)
at org.keycloak.models.cache.infinispan.entities.CachedClient.<init>(CachedClient.java:112)
at org.keycloak.models.cache.infinispan.RealmCacheSession.cacheClient(RealmCacheSession.java:1152)
at org.keycloak.models.cache.infinispan.RealmCacheSession.getClientById(RealmCacheSession.java:1114)
at org.keycloak.models.delegate.ClientModelLazyDelegate$WithId.lambda$new$0(ClientModelLazyDelegate.java:53)
at org.keycloak.models.delegate.ClientModelLazyDelegate$WithId$$Lambda$2226/0x00000008011db840.get(Unknown Source)
at org.keycloak.models.delegate.ClientModelLazyDelegate.getDelegate(ClientModelLazyDelegate.java:83)
at org.keycloak.models.delegate.ClientModelLazyDelegate.getAuthenticationFlowBindingOverrides(ClientModelLazyDelegate.java:324)
at org.keycloak.admin.ui.rest.model.AuthenticationMapper.lambda$convertToModel$1(AuthenticationMapper.java:32)
at org.keycloak.admin.ui.rest.model.AuthenticationMapper$$Lambda$2224/0x00000008011db040.test(Unknown Source)
at java.base@11.0.18/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:176)
at java.base@11.0.18/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
at java.base@11.0.18/java.util.Spliterators$IteratorSpliterator.tryAdvance(Spliterators.java:1812)
at java.base@11.0.18/java.util.stream.ReferencePipeline.forEachWithCancel(ReferencePipeline.java:127)
at java.base@11.0.18/java.util.stream.AbstractPipeline.copyIntoWithCancel(AbstractPipeline.java:502)
at java.base@11.0.18/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:488)
at java.base@11.0.18/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
at java.base@11.0.18/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913)
at java.base@11.0.18/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.base@11.0.18/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578)
at org.hibernate.query.spi.StreamDecorator.collect(StreamDecorator.java:209)
at org.keycloak.utils.ClosingStream.collect(ClosingStream.java:182)
at org.keycloak.admin.ui.rest.model.AuthenticationMapper.convertToModel(AuthenticationMapper.java:35)
at org.keycloak.admin.ui.rest.AuthenticationManagementResource.lambda$listIdentityProviders$1(AuthenticationManagementResource.java:66)
at org.keycloak.admin.ui.rest.AuthenticationManagementResource$$Lambda$2217/0x00000008011d9440.apply(Unknown Source)
at java.base@11.0.18/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
at java.base@11.0.18/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:177)
at java.base@11.0.18/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1655)
at java.base@11.0.18/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
at java.base@11.0.18/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
at java.base@11.0.18/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913)
at java.base@11.0.18/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.base@11.0.18/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578)
at org.keycloak.admin.ui.rest.AuthenticationManagementResource.listIdentityProviders(AuthenticationManagementResource.java:67)
at java.base@11.0.18/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base@11.0.18/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base@11.0.18/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base@11.0.18/java.lang.reflect.Method.invoke(Method.java:566)
at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:170)
at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:130)
at org.jboss.resteasy.core.ResourceMethodInvoker.internalInvokeOnTarget(ResourceMethodInvoker.java:660)
at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTargetAfterFilter(ResourceMethodInvoker.java:524)
at org.jboss.resteasy.core.ResourceMethodInvoker.lambda$invokeOnTarget$2(ResourceMethodInvoker.java:474
```
According to my investigation, the request process spends a lot of time in https://github.com/keycloak/keycloak/blob/21.1.1/rest/admin-ui-ext/src/main/java/org/keycloak/admin/ui/rest/model/AuthenticationMapper.java#L35 . The MAX_USED_BY==9 limit is useless, because none of our clients overrides the authentication flow.
Keycloak iterates on the full list of clients. Consequently, the full list of clients loads up in Hibernate 1st level cache leading to big performance degradation of the Hibernate autoflush process.
- links to
