Bartosz Spyrko-Smietanko added a comment - 2021/02/19 11:35 AM I found there's one more problem - as soon as the FAILURE_DESCRIPTION is set in the first failing ServiceVerificationHelper, the rollback flag will be set in https://github.com/wildfly/wildfly-core/blob/master/controller/src/main/java/org/jboss/as/controller/AbstractOperationContext.java#L1020  which will prevent any further ServiceVerificationHelpers from running. Maybe in this case SVH could attach something like DEFERED_ROLLBACK flag to the context and if it's present, canContinueProcessing() would allow the VERIFY stage to finish?

Brian Stansberry added a comment - 2021/02/15 4:55 PM Thanks. So AIUI the issue is that steps that add services result in adding a step the executes a ServiceVerificationHelper that monitors those particular services. And the first ServiceVerificationHelper that fails prevents any later ServiceVerificationHelper steps running, because the first one triggers rollback. One solution to this is to change this: https://github.com/wildfly/wildfly-core/blob/master/controller/src/main/java/org/jboss/as/controller/ServiceVerificationHelper.java#L123 Instead of immediately calling context.setRollbackOnly, instead add another Stage.VERIFY step that calls it. That step will be added to the end of the queue for Stage.VERIFY, so it will execute after all the previously registered ServiceVerificationHelper steps.

Bartosz Spyrko-Smietanko added a comment - 2021/02/15 3:51 AM Thank you  bstansbe@redhat.com , Below is the output of standalone boot. In this case the boot operation is rolled back via https://github.com/wildfly/wildfly-core/blob/master/controller/src/main/java/org/jboss/as/controller/management/ManagementInterfaceAddStepHandler.java#L106-L116   08:05:17,630 ERROR [org.jboss.as.controller.management-operation] (Controller Boot Thread) WFLYCTL0013: Operation ("add") failed - address: ([ ("core-service" => "management"), ("management-interface" => "http-interface") ]) - failure description: { "WFLYCTL0412: Required services that are not installed:" => ["jboss.http-upgrade-registry.http-management"], "WFLYCTL0180: Services with missing/unavailable dependencies" => ["jboss.remoting.remoting-http-upgrade-service.http-management is missing [jboss.http-upgrade-registry.http-management]"] } 08:05:17,683 ERROR [org.jboss.as.controller.management-operation] (Controller Boot Thread) WFLYCTL0013: Operation ("add") failed - address: ([ ("core-service" => "management"), ("security-realm" => "ldap_security_realm") ]) - failure description: { "WFLYCTL0412: Required services that are not installed:" => ["jboss.server.path.\"jboss.server.config.dir22\""], "WFLYCTL0180: Services with missing/unavailable dependencies" => ["org.wildfly.core.management.security.realm.ldap_security_realm.key-manager is missing [jboss.server.path.\"jboss.server.config.dir22\"]"] } 08:05:17,684 ERROR [org.jboss.as.controller.management-operation] (Controller Boot Thread) WFLYCTL0013: Operation ("add") failed - address: ([ ("core-service" => "management"), ("security-realm" => "ldap_security_realm") ]) - failure description: { "WFLYCTL0412: Required services that are not installed:" => ["jboss.server.path.\"jboss.server.config.dir22\""], "WFLYCTL0180: Services with missing/unavailable dependencies" => ["org.wildfly.core.management.security.realm.ldap_security_realm.key-manager is missing [jboss.server.path.\"jboss.server.config.dir22\"]"] } 08:05:17,684 ERROR [org.jboss.as.controller] (Controller Boot Thread) WFLYCTL0459: Triggering roll back due to missing management services. 08:05:17,685 ERROR [org.jboss.as.controller.management-operation] (Controller Boot Thread) WFLYCTL0013: Operation ("add") failed - address: ([ ("core-service" => "management"), ("management-interface" => "http-interface") ]) - failure description: { "WFLYCTL0412: Required services that are not installed:" => ["jboss.http-upgrade-registry.http-management"], "WFLYCTL0180: Services with missing/unavailable dependencies" => ["jboss.remoting.remoting-http-upgrade-service.http-management is missing [jboss.http-upgrade-registry.http-management]"] }   Another option could be having the ServiceVerificationHelper report indirect missing services if there are no direct dependencies missing rather then silently failing. That might add some extra logging in some cases though.

Brian Stansberry added a comment - 2021/02/12 6:48 PM spyrkob It's not ok for a Host Controller to not fail if there is any error during boot. A broken HC can mess up the entire domain. So, no that can't be changed. (FWIW I'd like to see the default behavior or a standalone server changed as well, as allowing a process to continue on in the presence of boot errors is not a good default IMO. That's an RFE though.) What's the failure in a standalone server?

Bartosz Spyrko-Smietanko added a comment - 2021/02/04 4:03 AM The same case in standalone mode works correctly. It seems the difference is that domain mode uses ROLLBACK_ON_FAIL header during boot [1] [2] . That means that the boot fails on first operation (adding the management interface) and doesn't process following steps that would display the errors.   jmesnil1@redhat.com  do you know if there is a reason why the domain module sets ROLLBACK_ON_FAIL flag? Can we set it to work like standalone mode?   [1]   https://github.com/wildfly/wildfly-core/blob/master/host-controller/src/main/java/org/jboss/as/host/controller/DomainModelControllerService.java#L679 [2]   https://github.com/wildfly/wildfly-core/blob/master/server/src/main/java/org/jboss/as/server/ServerService.java#L392-L404