Uploaded image for project: 'Kogito'
  1. Kogito
  2. KOGITO-943

Jobs Service: Random failure on startup for Infinispan and load of scheduled jobs

XMLWordPrintable

    • Hide

      Repeat as long as you don't have the issuer, as it is random ...

      Start infinispan-server (don't forget to restart each time to clean the cache)

      docker run -p 11222:11222 -e USER=dev -e PASS=dev jboss/infinispan-server:10.0.0.Final-1

      In kogito-runtimes/addons/jobs/jobs-service:

      1. Set application properties 
        kogito.jobs-service.persistence=infinispan
quarkus.infinispan-client.server-list=172.17.0.2:11222
quarkus.infinispan-client.auth-username=dev
quarkus.infinispan-client.auth-password=dev
quarkus.infinispan-client.use-auth=true
quarkus.infinispan-client.auth-realm=default
quarkus.infinispan-client.sasl-mechanism=PLAIN
      1. Launch command 
        mvn clean compile quarkus:dev
      Show
      Repeat as long as you don't have the issuer, as it is random ... Start infinispan-server (don't forget to restart each time to clean the cache) docker run -p 11222:11222 -e USER=dev -e PASS=dev jboss/infinispan-server:10.0.0.Final-1 In kogito-runtimes/addons/jobs/jobs-service: Set application properties kogito.jobs-service.persistence=infinispan quarkus.infinispan-client.server-list=172.17.0.2:11222 quarkus.infinispan-client.auth-username=dev quarkus.infinispan-client.auth-password=dev quarkus.infinispan-client.use-auth=true quarkus.infinispan-client.auth-realm=default quarkus.infinispan-client.sasl-mechanism=PLAIN Launch command mvn clean compile quarkus:dev
    • 2020 Week 04-06 (from Jan 20)

      When starting the Job Service, we get random failures on startup.
      Error logs:

      2020-01-22 13:31:42,231 jobs-service-6849b858d7-wc8qw INFO  [io.smallrye.reactive.messaging.extension.MediatorManager:205] (main) Attempt to resolve org.kie.kogito.jobs.service.scheduler.impl.VertxJobScheduler#jobSuccessProcessor
2020-01-22 13:31:42,232 jobs-service-6849b858d7-wc8qw INFO  [io.smallrye.reactive.messaging.extension.MediatorManager:211] (main) Connecting org.kie.kogito.jobs.service.scheduler.impl.VertxJobScheduler#jobSuccessProcessor to `job-success-events` (org.eclipse.microprofile.reactive.streams.operators.core.PublisherBuilderImpl@36b0fcd5)
2020-01-23 10:05:19,555 tradisso INFO  [org.kie.kogito.jobs.service.repository.impl.JobRepositoryDelegate:54] (main) JobRepository selected class org.kie.kogito.jobs.service.repository.infinispan.InfinispanJobRepository_ClientProxy
2020-01-23 10:05:20,055 tradisso INFO  [org.infinispan.HOTROD:359] (main) ISPN004021: Infinispan version: Infinispan 'Chupacabra' 10.0.0.Final
2020-01-23 10:05:20,460 tradisso WARN  [org.infinispan.HOTROD:335] (HotRod-client-async-pool-1-1) ISPN004005: Error received from the server: org.infinispan.server.hotrod.CacheNotFoundException: Cache with name 'SCHEDULED_JOBS' not found amongst the configured caches
2020-01-23 10:05:20,462 tradisso ERROR [io.quarkus.application:295] (main) Failed to start application: java.lang.IllegalArgumentException: cache parameter cannot be null

      Correct logs:

      2020-01-23 10:12:51,576 tradisso INFO  [io.smallrye.reactive.messaging.extension.MediatorManager:205] (main) Attempt to resolve org.kie.kogito.jobs.service.scheduler.impl.VertxJobScheduler#jobSuccessProcessor
2020-01-23 10:12:51,576 tradisso INFO  [io.smallrye.reactive.messaging.extension.MediatorManager:211] (main) Connecting org.kie.kogito.jobs.service.scheduler.impl.VertxJobScheduler#jobSuccessProcessor to `job-success-events` (org.eclipse.microprofile.reactive.streams.operators.core.PublisherBuilderImpl@7c75db8b)
2020-01-23 10:12:51,601 tradisso INFO  [org.kie.kogito.jobs.service.repository.infinispan.InfinispanConfiguration:66] (main) Persistence config Optional[infinispan]
2020-01-23 10:12:51,963 tradisso INFO  [org.infinispan.HOTROD:359] (main) ISPN004021: Infinispan version: Infinispan 'Chupacabra' 10.0.0.Final
2020-01-23 10:12:52,272 tradisso INFO  [org.kie.kogito.jobs.service.repository.infinispan.InfinispanConfiguration:78] (main) Executed Infinispan configuration
2020-01-23 10:12:52,273 tradisso INFO  [org.kie.kogito.jobs.service.scheduler.JobSchedulerManager:45] (main) Loading scheduled jobs
2020-01-23 10:12:52,274 tradisso INFO  [org.kie.kogito.jobs.service.repository.impl.JobRepositoryDelegate:54] (main) JobRepository selected class org.kie.kogito.jobs.service.repository.infinispan.InfinispanJobRepository_ClientProxy
2020-01-23 10:12:52,558 tradisso INFO  [org.kie.kogito.jobs.service.scheduler.JobSchedulerManager:53] (main) Loading scheduled jobs completed !

      It seems that, sometimes, the 'onStart' of InfinispanConfiguration is not done before the InfinispanJobRepository initialization. And so we get the error of cache is null (full logs in attachments)

      This might be caused by the update of Quarkus to 1.1.0 => https://github.com/kiegroup/kogito-bom/commit/deee85f77ce68de19844806203d58b36b8c9a96f

        1. correct.log
          6 kB
          Tristan Radisson
        2. error.log
          11 kB
          Tristan Radisson

              tdolphin-1 Tiago Dolphine (Inactive)
              tradisso@redhat.com Tristan Radisson
              Tristan Radisson Tristan Radisson
              Tristan Radisson Tristan Radisson
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated:
                Resolved: