Uploaded image for project: 'WildFly Core'
  1. WildFly Core
  2. WFCORE-4136

Booting a slave HC fails if the content repository entry for a rollout plan is not present

    XMLWordPrintable

Details

    • Bug
    • Resolution: Done
    • Major
    • 7.0.0.Alpha2, 7.0.0.Final
    • 6.0.2.Final, 7.0.0.Final
    • Management
    • None
    • Hide

      Originally reported reproducer:

      • Create a server-group containing two jvms spanned across two hosts.
      • Shutdown the HostController on one of the hosts
      • Create a new rollout-plan that targets the server-group.
      • Attempt to bring up the HostController that was shutdown.

      This is what I did:

      • Unzip the appserver in two different dirs, 'master' and 'slave'
      • In master:
        • bin/jboss-cli.sh
        • embed-host-controller --host-config=host-master.xml
        • rollout-plan add --name=test --content= {rollout main-server-group(rolling-to-servers=true) rollback-across-groups}
        • quit
        • bin/domain.sh --host-config=host-master.xml
      • In 'slave'
        • bin/domain.sh --host-config=host-slave.xml -Djboss.domain.master.address=127.0.0.0 -Djboss.domain.management.port=19999

      Booting the slave fails

      Show
      Originally reported reproducer: Create a server-group containing two jvms spanned across two hosts. Shutdown the HostController on one of the hosts Create a new rollout-plan that targets the server-group. Attempt to bring up the HostController that was shutdown. This is what I did: Unzip the appserver in two different dirs, 'master' and 'slave' In master: bin/jboss-cli.sh embed-host-controller --host-config=host-master.xml rollout-plan add --name=test --content= {rollout main-server-group(rolling-to-servers=true) rollback-across-groups} quit bin/domain.sh --host-config=host-master.xml In 'slave' bin/domain.sh --host-config=host-slave.xml -Djboss.domain.master.address=127.0.0.0 -Djboss.domain.management.port=19999 Booting the slave fails
    • Workaround Exists
    • Hide

      Add a 'ignored-resources' element to the domain-controller/remote element in the slave's host xml file to ignore management-client-content resources. For example:

          <domain-controller>
        <remote security-realm="ManagementRealm">
	    <ignored-resources type="management-client-content" wildcard="true"/>
            <discovery-options>
                <static-discovery name="primary" protocol="${jboss.domain.master.protocol:remote}" host="${jboss.domain.master.address}" port="${jboss.domain.master.port:9999}"/>
            </discovery-options>
        </remote>
    </domain-controller>

      This allows the slave to boot. However it removes the ability for that slave to use pre-stored rollout plans when deciding how to rollout changes targeted at it to its servers (i.e. operations targeted at addresses in the /host=slave resource tree.) However, most operations involving rolling plans are targeted at the DC, which will have access to the rollout plans.

      Show
      Add a 'ignored-resources' element to the domain-controller/remote element in the slave's host xml file to ignore management-client-content resources. For example: <domain-controller> <remote security-realm= "ManagementRealm" > <ignored-resources type= "management-client-content" wildcard= " true " /> <discovery-options> < static -discovery name= "primary" protocol= "${jboss.domain.master.protocol:remote}" host= "${jboss.domain.master.address}" port= "${jboss.domain.master.port:9999}" /> </discovery-options> </remote> </domain-controller> This allows the slave to boot. However it removes the ability for that slave to use pre-stored rollout plans when deciding how to rollout changes targeted at it to its servers (i.e. operations targeted at addresses in the /host=slave resource tree.) However, most operations involving rolling plans are targeted at the DC, which will have access to the rollout plans.

    Description

      Slave HC fails to boot if it connects to a DC that has a rollout-plan configured when the slave was shut down.

      See Steps to Reproduce.

      Failure is: 

      [Host Controller] 14:51:04,208 ERROR [org.jboss.as.controller.management-operation] (Host Controller Service Threads - 4) WFLYCTL0013: Operation ("add") failed - address: ([("management-client-content" => "rollout-plans")]): org.jboss.as.management.client.content.ContentStorageException: java.io.FileNotFoundException: /Users/bstansberry/tmp/support/02184219/slave/domain/data/content/2d/56893cc4da1c67a79f4845a57ae90c04a969b4/content (No such file or directory)
[Host Controller] 	at org.jboss.as.management.client.content.ManagedDMRContentTypeResource.loadContent(ManagedDMRContentTypeResource.java:306)
[Host Controller] 	at org.jboss.as.management.client.content.ManagedDMRContentTypeResource.<init>(ManagedDMRContentTypeResource.java:91)
[Host Controller] 	at org.jboss.as.management.client.content.ManagedDMRContentTypeAddHandler.execute(ManagedDMRContentTypeAddHandler.java:58)
[Host Controller] 	at org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:982)
[Host Controller] 	at org.jboss.as.controller.AbstractOperationContext.processStages(AbstractOperationContext.java:726)
[Host Controller] 	at org.jboss.as.controller.AbstractOperationContext.executeOperation(AbstractOperationContext.java:450)
[Host Controller] 	at org.jboss.as.controller.OperationContextImpl.executeOperation(OperationContextImpl.java:1402)
[Host Controller] 	at org.jboss.as.controller.ModelControllerImpl.internalExecute(ModelControllerImpl.java:418)
[Host Controller] 	at org.jboss.as.controller.AbstractControllerService.internalExecute(AbstractControllerService.java:490)
[Host Controller] 	at org.jboss.as.host.controller.DomainModelControllerService.access$1100(DomainModelControllerService.java:200)
[Host Controller] 	at org.jboss.as.host.controller.DomainModelControllerService$InternalExecutor$1.lambda$apply$0(DomainModelControllerService.java:1372)
[Host Controller] 	at org.jboss.as.controller.access.InVmAccess.runInVm(InVmAccess.java:63)
[Host Controller] 	at org.jboss.as.host.controller.DomainModelControllerService$InternalExecutor$1.apply(DomainModelControllerService.java:1372)
[Host Controller] 	at org.jboss.as.host.controller.DomainModelControllerService$InternalExecutor$1.apply(DomainModelControllerService.java:1369)
[Host Controller] 	at org.jboss.as.host.controller.SecurityActions$Execution$1.execute(SecurityActions.java:77)
[Host Controller] 	at org.jboss.as.host.controller.SecurityActions.privilegedExecution(SecurityActions.java:47)
[Host Controller] 	at org.jboss.as.host.controller.DomainModelControllerService$InternalExecutor.execute(DomainModelControllerService.java:1375)
[Host Controller] 	at org.jboss.as.host.controller.RemoteDomainConnectionService.applyRemoteDomainModel(RemoteDomainConnectionService.java:603)
[Host Controller] 	at org.jboss.as.host.controller.RemoteDomainConnectionService.access$1200(RemoteDomainConnectionService.java:138)
[Host Controller] 	at org.jboss.as.host.controller.RemoteDomainConnectionService$3.applyDomainModel(RemoteDomainConnectionService.java:546)
[Host Controller] 	at org.jboss.as.host.controller.RemoteDomainConnection.applyDomainModel(RemoteDomainConnection.java:325)
[Host Controller] 	at org.jboss.as.host.controller.RemoteDomainConnection$RegisterSubsystemsRequest$1.execute(RemoteDomainConnection.java:468)
[Host Controller] 	at org.jboss.as.protocol.mgmt.ManagementRequestContextImpl$1.doExecute(ManagementRequestContextImpl.java:70)
[Host Controller] 	at org.jboss.as.protocol.mgmt.ManagementRequestContextImpl$AsyncTaskRunner.run(ManagementRequestContextImpl.java:160)
[Host Controller] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[Host Controller] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[Host Controller] 	at java.lang.Thread.run(Thread.java:748)
[Host Controller] 	at org.jboss.threads.JBossThread.run(JBossThread.java:320)
[Host Controller] Caused by: java.io.FileNotFoundException: /Users/bstansberry/tmp/support/02184219/slave/domain/data/content/2d/56893cc4da1c67a79f4845a57ae90c04a969b4/content (No such file or directory)
[Host Controller] 	at java.io.FileInputStream.open0(Native Method)
[Host Controller] 	at java.io.FileInputStream.open(FileInputStream.java:195)
[Host Controller] 	at java.io.FileInputStream.<init>(FileInputStream.java:138)
[Host Controller] 	at org.jboss.vfs.spi.RootFileSystem.openInputStream(RootFileSystem.java:51)
[Host Controller] 	at org.jboss.vfs.VirtualFile.openStream(VirtualFile.java:318)
[Host Controller] 	at org.jboss.as.management.client.content.ManagedDMRContentTypeResource.loadContent(ManagedDMRContentTypeResource.java:292)
[Host Controller] 	... 27 more
[Host Controller] 
[Host Controller] 14:51:04,212 ERROR [org.jboss.as.host.controller] (Host Controller Service Threads - 4) WFLYHC0143: Failed to apply domain-wide configuration from master host controller. Operation outcome: failed. Failure description {"WFLYCTL0062: Composite operation failed and was rolled back. Steps that failed:" => {"Operation step-552" => "WFLYCTL0158: Operation handler failed: org.jboss.as.management.client.content.ContentStorageException: java.io.FileNotFoundException: /Users/bstansberry/tmp/support/02184219/slave/domain/data/content/2d/56893cc4da1c67a79f4845a57ae90c04a969b4/content (No such file or directory)"}}
[Host Controller] 14:51:04,218 WARN  [org.jboss.as.host.controller] (Controller Boot Thread) WFLYHC0001: Could not connect to remote domain controller remote://127.0.0.1:9999: 1-$-
[Host Controller] 	at org.jboss.as.host.controller.RemoteDomainConnection$RegisterSubsystemsRequest$1.execute(RemoteDomainConnection.java:472)
...
[Host Controller] 14:51:04,219 WARN  [org.jboss.as.host.controller] (Controller Boot Thread) WFLYHC0147: No domain controller discovery options remain.
[Host Controller] 14:51:04,220 ERROR [org.jboss.as.host.controller] (Controller Boot Thread) WFLYHC0002: Could not connect to master. Error was: java.lang.IllegalStateException: WFLYHC0120: Tried all domain controller discovery option(s) but unable to connect
[Host Controller] 14:51:04,221 FATAL [org.jboss.as.host.controller] (Controller Boot Thread) WFLYHC0178: Aborting with exit code 99

      Attachments

        Issue Links

          is cloned by

          Bug - A problem that impairs or prevents the functions of the product. JBEAP-15530 [GSS](7.2.z) Booting a slave HC fails if the content repository entry for a rollout plan is not present

          • Major - To be worked after higher priority work items are addressed. When the severity is high and the effort to change it is low to moderate, this term is chosen. Issues in this category likely have an existing workaround but implementation or execution may be non-trivial.
          • Verified

          Activity

            People

              bstansbe@redhat.com Brian Stansberry
              bstansbe@redhat.com Brian Stansberry
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: