Uploaded image for project: 'Red Hat Fuse'
  1. Red Hat Fuse
  2. ENTESB-5868

Joined container does not have provision status sometimes

    XMLWordPrintable

Details

    • Bug
    • Resolution: Done
    • Major
    • jboss-fuse-6.3
    • jboss-fuse-6.3
    • Fabric8 v1
    • None
    • % %
    • Hide

      2 containers: "root", "xx"
      "root":

      • uncomment default admin user

      "xx":

      • uncomment default admin user
      • change karaf.name in etc/system.properties

      "root"

      • fabric:create

      "xx"

      • fabric:join --zookeeper-password admin <root_ip>:2181
      Show
      2 containers: "root", "xx" "root": uncomment default admin user "xx": uncomment default admin user change karaf.name in etc/system.properties "root" fabric:create "xx" fabric:join --zookeeper-password admin <root_ip>:2181

    Description

      When you join a container into fabric, it does not have provision status sometimes. It is probably related to ENTESB-5206 , as it can't connect to ZK initially (see "Unable to set provisioning result"):

      after fabric:join:
      
      2016-07-19 03:32:48,304 | INFO  | l Console Thread | CuratorFrameworkImpl             | 178 - io.fabric8.fabric-zookeeper - 1.2.0.redhat-630133 | Starting
      2016-07-19 03:32:48,313 | INFO  | l Console Thread | ZooKeeper                        | 178 - io.fabric8.fabric-zookeeper - 1.2.0.redhat-630133 | Client environment:zookeeper.version=3.4.7-1713338, built on 11/09/2015 04:32 GMT
      2016-07-19 03:32:48,313 | INFO  | l Console Thread | ZooKeeper                        | 178 - io.fabric8.fabric-zookeeper - 1.2.0.redhat-630133 | Client environment:host.name=host-172-16-117-126.openstacklocal
      2016-07-19 03:32:48,314 | INFO  | l Console Thread | ZooKeeper                        | 178 - io.fabric8.fabric-zookeeper - 1.2.0.redhat-630133 | Client environment:java.version=1.7.0_75
      2016-07-19 03:32:48,314 | INFO  | l Console Thread | ZooKeeper                        | 178 - io.fabric8.fabric-zookeeper - 1.2.0.redhat-630133 | Client environment:java.vendor=Oracle Corporation
      2016-07-19 03:32:48,314 | INFO  | l Console Thread | ZooKeeper                        | 178 - io.fabric8.fabric-zookeeper - 1.2.0.redhat-630133 | Client environment:java.home=/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.75-2.5.4.2.el7_0.x86_64/jre
      2016-07-19 03:32:48,314 | INFO  | l Console Thread | ZooKeeper                        | 178 - io.fabric8.fabric-zookeeper - 1.2.0.redhat-630133 | Client environment:java.class.path=/home/fuse/jboss-fuse-6.3.0.redhat-133/lib/karaf-jaas-boot.jar:/home/fuse/jboss-fuse-6.3.0.redhat-133/lib/karaf.jar:/home/fuse/jboss-fuse-6.3.0.redhat-133/lib/karaf-jmx-boot.jar
      2016-07-19 03:32:48,314 | INFO  | l Console Thread | ZooKeeper                        | 178 - io.fabric8.fabric-zookeeper - 1.2.0.redhat-630133 | Client environment:java.library.path=:/home/fuse/jboss-fuse-6.3.0.redhat-133/lib:/home/fuse/jboss-fuse-6.3.0.redhat-133/lib:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
      2016-07-19 03:32:48,314 | INFO  | l Console Thread | ZooKeeper                        | 178 - io.fabric8.fabric-zookeeper - 1.2.0.redhat-630133 | Client environment:java.io.tmpdir=/home/fuse/jboss-fuse-6.3.0.redhat-133/data/tmp
      2016-07-19 03:32:48,314 | INFO  | l Console Thread | ZooKeeper                        | 178 - io.fabric8.fabric-zookeeper - 1.2.0.redhat-630133 | Client environment:java.compiler=<NA>
      2016-07-19 03:32:48,314 | INFO  | l Console Thread | ZooKeeper                        | 178 - io.fabric8.fabric-zookeeper - 1.2.0.redhat-630133 | Client environment:os.name=Linux
      2016-07-19 03:32:48,314 | INFO  | l Console Thread | ZooKeeper                        | 178 - io.fabric8.fabric-zookeeper - 1.2.0.redhat-630133 | Client environment:os.arch=amd64
      2016-07-19 03:32:48,314 | INFO  | l Console Thread | ZooKeeper                        | 178 - io.fabric8.fabric-zookeeper - 1.2.0.redhat-630133 | Client environment:os.version=3.10.0-229.el7.x86_64
      2016-07-19 03:32:48,314 | INFO  | l Console Thread | ZooKeeper                        | 178 - io.fabric8.fabric-zookeeper - 1.2.0.redhat-630133 | Client environment:user.name=fuse
      2016-07-19 03:32:48,314 | INFO  | l Console Thread | ZooKeeper                        | 178 - io.fabric8.fabric-zookeeper - 1.2.0.redhat-630133 | Client environment:user.home=/home/fuse
      2016-07-19 03:32:48,314 | INFO  | l Console Thread | ZooKeeper                        | 178 - io.fabric8.fabric-zookeeper - 1.2.0.redhat-630133 | Client environment:user.dir=/home/fuse/jboss-fuse-6.3.0.redhat-133
      2016-07-19 03:32:48,315 | INFO  | l Console Thread | ZooKeeper                        | 178 - io.fabric8.fabric-zookeeper - 1.2.0.redhat-630133 | Initiating client connection, connectString=10.8.180.238:2181 sessionTimeout=60000 watcher=org.apache.curator.ConnectionState@735f1326
      2016-07-19 03:32:48,341 | INFO  | .8.180.238:2181) | ClientCnxn                       | 178 - io.fabric8.fabric-zookeeper - 1.2.0.redhat-630133 | Opening socket connection to server 10.8.180.238/10.8.180.238:2181
      2016-07-19 03:32:48,344 | INFO  | .8.180.238:2181) | ClientCnxn                       | 178 - io.fabric8.fabric-zookeeper - 1.2.0.redhat-630133 | Socket connection established to 10.8.180.238/10.8.180.238:2181, initiating session
      2016-07-19 03:32:48,349 | INFO  | .8.180.238:2181) | ClientCnxn                       | 178 - io.fabric8.fabric-zookeeper - 1.2.0.redhat-630133 | Session establishment complete on server 10.8.180.238/10.8.180.238:2181, sessionid = 0x15602103ac20002, negotiated timeout = 40000
      2016-07-19 03:32:48,352 | INFO  | read-EventThread | ConnectionStateManager           | 178 - io.fabric8.fabric-zookeeper - 1.2.0.redhat-630133 | State change: CONNECTED
      2016-07-19 03:32:48,380 | INFO  | l Console Thread | ZooKeeper                        | 178 - io.fabric8.fabric-zookeeper - 1.2.0.redhat-630133 | Session: 0x15602103ac20002 closed
      2016-07-19 03:32:48,380 | INFO  | read-EventThread | ClientCnxn                       | 178 - io.fabric8.fabric-zookeeper - 1.2.0.redhat-630133 | EventThread shut down for session: 0x15602103ac20002
      2016-07-19 03:32:48,385 | INFO  | MCF-1-thread-1   | CuratorFrameworkImpl             | 178 - io.fabric8.fabric-zookeeper - 1.2.0.redhat-630133 | Starting
      2016-07-19 03:32:48,385 | INFO  | MCF-1-thread-1   | ZooKeeper                        | 178 - io.fabric8.fabric-zookeeper - 1.2.0.redhat-630133 | Initiating client connection, connectString=10.8.180.238:2181 sessionTimeout=60000 watcher=org.apache.curator.ConnectionState@30ef67f2
      2016-07-19 03:32:48,385 | INFO  | .8.180.238:2181) | ClientCnxn                       | 178 - io.fabric8.fabric-zookeeper - 1.2.0.redhat-630133 | Opening socket connection to server 10.8.180.238/10.8.180.238:2181
      2016-07-19 03:32:48,387 | INFO  | .8.180.238:2181) | ClientCnxn                       | 178 - io.fabric8.fabric-zookeeper - 1.2.0.redhat-630133 | Socket connection established to 10.8.180.238/10.8.180.238:2181, initiating session
      2016-07-19 03:32:48,389 | INFO  | .8.180.238:2181) | ClientCnxn                       | 178 - io.fabric8.fabric-zookeeper - 1.2.0.redhat-630133 | Session establishment complete on server 10.8.180.238/10.8.180.238:2181, sessionid = 0x15602103ac20003, negotiated timeout = 40000
      2016-07-19 03:32:48,390 | INFO  | ad-1-EventThread | ConnectionStateManager           | 178 - io.fabric8.fabric-zookeeper - 1.2.0.redhat-630133 | State change: CONNECTED
      2016-07-19 03:32:48,416 | INFO  | MCF-1-thread-1   | ZooKeeperGroup                   | 172 - io.fabric8.fabric-groups - 1.2.0.redhat-630133 | Creating ZK Group for path "/fabric/registry/clusters/git"
      2016-07-19 03:32:48,416 | INFO  | MCF-1-thread-1   | ZooKeeperGroup                   | 172 - io.fabric8.fabric-groups - 1.2.0.redhat-630133 | Starting ZK Group for path "/fabric/registry/clusters/git"
      2016-07-19 03:32:48,659 | INFO  | MCF-1-thread-1   | GitDataStoreImpl                 | 173 - io.fabric8.fabric-git - 1.2.0.redhat-630133 | Starting up GitDataStore io.fabric8.git.internal.GitDataStoreImpl@41e02c1
      2016-07-19 03:32:48,661 | INFO  | MCF-1-thread-1   | GitDataStoreImpl                 | 173 - io.fabric8.fabric-git - 1.2.0.redhat-630133 | Remote url change from: null to: http://host-172-16-117-125:8181/git/fabric/
      2016-07-19 03:32:48,666 | INFO  | MCF-1-thread-1   | DefaultPullPushPolicy            | 173 - io.fabric8.fabric-git - 1.2.0.redhat-630133 | Performing a pull on remote URL: http://host-172-16-117-125:8181/git/fabric/
      2016-07-19 03:32:48,738 | INFO  | cache-1-thread-1 | ZkDataStoreImpl                  | 171 - io.fabric8.fabric-core - 1.2.0.redhat-630133 | Event CHILD_ADDED detected on /fabric/configs/versions/1.0/containers/xx with data fabric. Sending notification.
      2016-07-19 03:32:48,738 | INFO  | cache-1-thread-1 | ZkDataStoreImpl                  | 171 - io.fabric8.fabric-core - 1.2.0.redhat-630133 | Event CHILD_ADDED detected on /fabric/configs/ensemble with data 0000. Sending notification.
      2016-07-19 03:32:48,739 | INFO  | cache-1-thread-1 | ZkDataStoreImpl                  | 171 - io.fabric8.fabric-core - 1.2.0.redhat-630133 | Event CHILD_ADDED detected on /fabric/configs/containers/xx with data 1.0. Sending notification.
      2016-07-19 03:32:48,739 | INFO  | cache-1-thread-1 | ZkDataStoreImpl                  | 171 - io.fabric8.fabric-core - 1.2.0.redhat-630133 | Event CHILD_ADDED detected on /fabric/configs/ensemble/url with data ${zk:root/ip}:2181. Sending notification.
      2016-07-19 03:32:50,017 | INFO  | MCF-1-thread-1   | DefaultPullPushPolicy            | 173 - io.fabric8.fabric-git - 1.2.0.redhat-630133 | Pull result: [localUpdate=[patches-ssh-fabric8, patches-root-fuse, patches-root-amq, patches-ssh-fuse, 1.0, master, patches-child],remoteUpdate=false,versions=[1.0, master, patches-child, patches-root-amq, patches-root-fuse, patches-ssh-fabric8, patches-ssh-fuse],error=null]
      2016-07-19 03:32:50,019 | INFO  | MCF-1-thread-1   | GitDataStoreImpl                 | 173 - io.fabric8.fabric-git - 1.2.0.redhat-630133 | Using ZooKeeper SharedCount to react when master git repo is changed, so we can do a git pull to the local git repo.
      2016-07-19 03:32:50,027 | INFO  | MCF-1-thread-1   | DefaultPullPushPolicy            | 173 - io.fabric8.fabric-git - 1.2.0.redhat-630133 | Performing a pull on remote URL: http://host-172-16-117-125:8181/git/fabric/
      2016-07-19 03:32:50,039 | INFO  | MCF-1-thread-1   | DefaultPullPushPolicy            | 173 - io.fabric8.fabric-git - 1.2.0.redhat-630133 | Pull result: [localUpdate=[],remoteUpdate=false,versions=[1.0, master, patches-child, patches-root-amq, patches-root-fuse, patches-ssh-fabric8, patches-ssh-fuse],error=null]
      2016-07-19 03:32:50,044 | INFO  | MCF-1-thread-1   | HttpServiceFactoryImpl           | 116 - org.ops4j.pax.web.pax-web-runtime - 4.2.7 | Binding bundle: [io.fabric8.fabric-git-server [185]] to http service
      2016-07-19 03:32:50,053 | INFO  | MCF-1-thread-1   | ZooKeeperGroup                   | 172 - io.fabric8.fabric-groups - 1.2.0.redhat-630133 | Creating ZK Group for path "/fabric/registry/clusters/git"
      2016-07-19 03:32:50,059 | INFO  | MCF-1-thread-1   | ZooKeeperGroup                   | 172 - io.fabric8.fabric-groups - 1.2.0.redhat-630133 | Starting ZK Group for path "/fabric/registry/clusters/git"
      2016-07-19 03:32:50,244 | INFO  | MCF-1-thread-1   | HttpServiceFactoryImpl           | 116 - org.ops4j.pax.web.pax-web-runtime - 4.2.7 | Binding bundle: [io.fabric8.fabric-maven-proxy [192]] to http service
      2016-07-19 03:32:50,280 | INFO  | MCF-1-thread-1   | JettyServerWrapper               | 117 - org.ops4j.pax.web.pax-web-jetty - 4.2.7 | will add org.eclipse.jetty.websocket.jsr356.server.deploy.WebSocketServerContainerInitializer to ServletContainerInitializers
      2016-07-19 03:32:50,419 | INFO  | MCF-1-thread-1   | JettyServerWrapper               | 117 - org.ops4j.pax.web.pax-web-jetty - 4.2.7 | added ServletContainerInitializer: org.eclipse.jetty.websocket.jsr356.server.deploy.WebSocketServerContainerInitializer
      2016-07-19 03:32:50,419 | INFO  | MCF-1-thread-1   | JettyServerWrapper               | 117 - org.ops4j.pax.web.pax-web-jetty - 4.2.7 | will add org.apache.jasper.servlet.JasperInitializer to ServletContainerInitializers
      2016-07-19 03:32:50,419 | INFO  | MCF-1-thread-1   | JettyServerWrapper               | 117 - org.ops4j.pax.web.pax-web-jetty - 4.2.7 | Skipt org.apache.jasper.servlet.JasperInitializer, because specialized handler will be present
      2016-07-19 03:32:50,422 | INFO  | MCF-1-thread-1   | HttpServiceContext               | 117 - org.ops4j.pax.web.pax-web-jetty - 4.2.7 | registering context DefaultHttpContext [bundle=io.fabric8.fabric-maven-proxy [192], contextID=default], with context-name: 
      2016-07-19 03:32:50,423 | INFO  | MCF-1-thread-1   | HttpServiceContext               | 117 - org.ops4j.pax.web.pax-web-jetty - 4.2.7 | registering JasperInitializer
      2016-07-19 03:32:50,534 | INFO  | MCF-1-thread-1   | ContextHandler                   | 96 - org.eclipse.jetty.util - 9.2.17.v20160517 | Started HttpServiceContext{httpContext=DefaultHttpContext [bundle=io.fabric8.fabric-maven-proxy [192], contextID=default]}
      2016-07-19 03:32:50,555 | INFO  | MCF-1-thread-1   | JettyServerWrapper               | 117 - org.ops4j.pax.web.pax-web-jetty - 4.2.7 | will add org.eclipse.jetty.websocket.jsr356.server.deploy.WebSocketServerContainerInitializer to ServletContainerInitializers
      2016-07-19 03:32:50,671 | INFO  | admin-1-thread-1 | FabricConfigAdminBridge          | 174 - io.fabric8.fabric-configadmin - 1.2.0.redhat-630133 | Updating configuration io.fabric8.ports
      2016-07-19 03:32:50,672 | INFO  | admin-1-thread-1 | FabricConfigAdminBridge          | 174 - io.fabric8.fabric-configadmin - 1.2.0.redhat-630133 | Updating configuration io.fabric8.docker.provider
      2016-07-19 03:32:50,673 | INFO  | admin-1-thread-1 | FabricConfigAdminBridge          | 174 - io.fabric8.fabric-configadmin - 1.2.0.redhat-630133 | Updating configuration io.fabric8.environment
      2016-07-19 03:32:50,674 | INFO  | admin-1-thread-1 | FabricConfigAdminBridge          | 174 - io.fabric8.fabric-configadmin - 1.2.0.redhat-630133 | Updating configuration jmx.acl.io.fabric8.Fabric
      2016-07-19 03:32:50,687 | INFO  | admin-1-thread-1 | FabricConfigAdminBridge          | 174 - io.fabric8.fabric-configadmin - 1.2.0.redhat-630133 | Updating configuration org.ops4j.pax.logging
      2016-07-19 03:32:50,689 | INFO  | admin-1-thread-1 | FabricConfigAdminBridge          | 174 - io.fabric8.fabric-configadmin - 1.2.0.redhat-630133 | Updating configuration org.apache.cxf.osgi
      2016-07-19 03:32:50,690 | INFO  | admin-1-thread-1 | FabricConfigAdminBridge          | 174 - io.fabric8.fabric-configadmin - 1.2.0.redhat-630133 | Updating configuration io.fabric8.jolokia
      2016-07-19 03:32:50,691 | INFO  | admin-1-thread-1 | FabricConfigAdminBridge          | 174 - io.fabric8.fabric-configadmin - 1.2.0.redhat-630133 | Updating configuration io.fabric8.maven.proxy
      2016-07-19 03:32:50,692 | INFO  | admin-1-thread-1 | FabricConfigAdminBridge          | 174 - io.fabric8.fabric-configadmin - 1.2.0.redhat-630133 | Updating configuration org.ops4j.pax.web
      2016-07-19 03:32:50,693 | INFO  | admin-1-thread-1 | FabricConfigAdminBridge          | 174 - io.fabric8.fabric-configadmin - 1.2.0.redhat-630133 | Updating configuration io.fabric8.zookeeper
      2016-07-19 03:32:50,702 | INFO  | admin-1-thread-1 | FabricConfigAdminBridge          | 174 - io.fabric8.fabric-configadmin - 1.2.0.redhat-630133 | Updating configuration io.fabric8.version
      2016-07-19 03:32:50,705 | INFO  | admin-1-thread-1 | FabricConfigAdminBridge          | 174 - io.fabric8.fabric-configadmin - 1.2.0.redhat-630133 | Updating configuration io.fabric8.jaas
      2016-07-19 03:32:50,706 | INFO  | admin-1-thread-1 | FabricConfigAdminBridge          | 174 - io.fabric8.fabric-configadmin - 1.2.0.redhat-630133 | Updating configuration io.fabric8.maven
      2016-07-19 03:32:50,706 | INFO  | admin-1-thread-1 | FabricConfigAdminBridge          | 174 - io.fabric8.fabric-configadmin - 1.2.0.redhat-630133 | Updating configuration org.apache.karaf.command.acl.fabric
      2016-07-19 03:32:50,708 | INFO  | admin-1-thread-1 | FabricConfigAdminBridge          | 174 - io.fabric8.fabric-configadmin - 1.2.0.redhat-630133 | Updating configuration io.fabric8.agent
      2016-07-19 03:32:50,708 | INFO  | admin-1-thread-1 | FabricConfigAdminBridge          | 174 - io.fabric8.fabric-configadmin - 1.2.0.redhat-630133 | Deleting configuration io.fabric8.mq.fabric.server.8c63f318-2355-4bac-bd91-a1ccda0dd1bd
      2016-07-19 03:32:50,719 | INFO  | MCF-1-thread-1   | HttpServiceContext               | 117 - org.ops4j.pax.web.pax-web-jetty - 4.2.7 | registering context MavenSecureHttpContext{io.fabric8.fabric-maven-proxy - 192}, with context-name: 
      2016-07-19 03:32:50,719 | INFO  | MCF-1-thread-1   | HttpServiceContext               | 117 - org.ops4j.pax.web.pax-web-jetty - 4.2.7 | registering JasperInitializer
      2016-07-19 03:32:50,763 | INFO  | MCF-1-thread-1   | ContextHandler                   | 96 - org.eclipse.jetty.util - 9.2.17.v20160517 | Started HttpServiceContext{httpContext=MavenSecureHttpContext{io.fabric8.fabric-maven-proxy - 192}}
      2016-07-19 03:32:50,782 | INFO  | pool-10-thread-2 | HttpServiceFactoryImpl           | 116 - org.ops4j.pax.web.pax-web-runtime - 4.2.7 | Unbinding bundle: [io.fabric8.fabric-git-server [185]]
      2016-07-19 03:32:50,786 | INFO  | pool-10-thread-2 | SupportAuthenticationFilter      | 287 - io.fabric8.support.support-webapp - 1.2.0.redhat-630133 | Starting hawtio authentication filter, JAAS realm: "karaf" authorized role(s): "admin,manager,viewer,Monitor, Operator, Maintainer, Deployer, Auditor, Administrator, SuperUser" role principal classes: "org.apache.karaf.jaas.boot.principal.RolePrincipal,org.apache.karaf.jaas.modules.RolePrincipal,org.apache.karaf.jaas.boot.principal.GroupPrincipal"
      2016-07-19 03:32:50,788 | INFO  | pool-10-thread-2 | SupportAuthenticationFilter      | 287 - io.fabric8.support.support-webapp - 1.2.0.redhat-630133 | Starting hawtio authentication filter, JAAS realm: "karaf" authorized role(s): "admin,manager,viewer,Monitor, Operator, Maintainer, Deployer, Auditor, Administrator, SuperUser" role principal classes: "org.apache.karaf.jaas.boot.principal.RolePrincipal,org.apache.karaf.jaas.modules.RolePrincipal,org.apache.karaf.jaas.boot.principal.GroupPrincipal"
      2016-07-19 03:32:50,790 | INFO  | pool-10-thread-2 | SupportAuthenticationFilter      | 287 - io.fabric8.support.support-webapp - 1.2.0.redhat-630133 | Starting hawtio authentication filter, JAAS realm: "karaf" authorized role(s): "admin,manager,viewer,Monitor, Operator, Maintainer, Deployer, Auditor, Administrator, SuperUser" role principal classes: "org.apache.karaf.jaas.boot.principal.RolePrincipal,org.apache.karaf.jaas.modules.RolePrincipal,org.apache.karaf.jaas.boot.principal.GroupPrincipal"
      2016-07-19 03:32:50,790 | INFO  | pool-10-thread-2 | SupportAuthenticationFilter      | 287 - io.fabric8.support.support-webapp - 1.2.0.redhat-630133 | Destroying hawtio authentication filter
      2016-07-19 03:32:50,796 | INFO  | pool-10-thread-2 | HttpServiceFactoryImpl           | 116 - org.ops4j.pax.web.pax-web-runtime - 4.2.7 | Unbinding bundle: [io.fabric8.support.support-webapp [287]]
      2016-07-19 03:32:50,800 | INFO  | pool-10-thread-2 | PluginContextListener            | 289 - io.hawt.hawtio-redhat-fuse-branding - 1.4.0.redhat-630133 | Destroyed hawtio-redhat-fuse-branding plugin
      2016-07-19 03:32:50,801 | INFO  | pool-10-thread-2 | HttpServiceFactoryImpl           | 116 - org.ops4j.pax.web.pax-web-runtime - 4.2.7 | Unbinding bundle: [io.hawt.hawtio-redhat-fuse-branding [289]]
      2016-07-19 03:32:50,803 | INFO  | pool-10-thread-2 | AuthenticationFilter             | 283 - io.hawt.hawtio-karaf-terminal - 1.4.0.redhat-630133 | Starting hawtio authentication filter, JAAS realm: "karaf" authorized role(s): "admin,manager,viewer,Monitor, Operator, Maintainer, Deployer, Auditor, Administrator, SuperUser" role principal classes: "org.apache.karaf.jaas.boot.principal.RolePrincipal,org.apache.karaf.jaas.modules.RolePrincipal,org.apache.karaf.jaas.boot.principal.GroupPrincipal"
      2016-07-19 03:32:50,803 | INFO  | pool-10-thread-2 | AuthenticationFilter             | 283 - io.hawt.hawtio-karaf-terminal - 1.4.0.redhat-630133 | Starting hawtio authentication filter, JAAS realm: "karaf" authorized role(s): "admin,manager,viewer,Monitor, Operator, Maintainer, Deployer, Auditor, Administrator, SuperUser" role principal classes: "org.apache.karaf.jaas.boot.principal.RolePrincipal,org.apache.karaf.jaas.modules.RolePrincipal,org.apache.karaf.jaas.boot.principal.GroupPrincipal"
      2016-07-19 03:32:50,804 | INFO  | pool-10-thread-2 | AuthenticationFilter             | 283 - io.hawt.hawtio-karaf-terminal - 1.4.0.redhat-630133 | Starting hawtio authentication filter, JAAS realm: "karaf" authorized role(s): "admin,manager,viewer,Monitor, Operator, Maintainer, Deployer, Auditor, Administrator, SuperUser" role principal classes: "org.apache.karaf.jaas.boot.principal.RolePrincipal,org.apache.karaf.jaas.modules.RolePrincipal,org.apache.karaf.jaas.boot.principal.GroupPrincipal"
      2016-07-19 03:32:50,804 | INFO  | pool-10-thread-2 | AuthenticationFilter             | 283 - io.hawt.hawtio-karaf-terminal - 1.4.0.redhat-630133 | Destroying hawtio authentication filter
      2016-07-19 03:32:50,807 | INFO  | pool-10-thread-2 | HttpServiceFactoryImpl           | 116 - org.ops4j.pax.web.pax-web-runtime - 4.2.7 | Unbinding bundle: [io.hawt.hawtio-karaf-terminal [283]]
      2016-07-19 03:32:50,810 | INFO  | ic8.maven.proxy) | HttpServiceFactoryImpl           | 116 - org.ops4j.pax.web.pax-web-runtime - 4.2.7 | Unbinding bundle: [io.fabric8.fabric-maven-proxy [192]]
      2016-07-19 03:32:50,811 | INFO  | o.fabric8.agent) | DeploymentAgent                  | 175 - io.fabric8.fabric-agent - 1.2.0.redhat-630133 | DeploymentAgent fabric-agent-175.0.1 updated with {attribute.parents=fabric, bundle.pax-url-war=mvn:org.ops4j.pax.url/pax-url-war/2.4.7/jar/uber, config.org.apache.aries.blueprint.synchronous=true, fabric.zookeeper.pid=io.fabric8.agent, feature.esb-commands-startup=esb-commands-startup, feature.fabric-agent=fabric-agent, feature.fabric-agent-commands=fabric-agent-commands, feature.fabric-archetype-commands=fabric-archetype-commands, feature.fabric-commands=fabric-commands, feature.fabric-core=fabric-core, feature.fabric-git=fabric-git, feature.fabric-git-server=fabric-git-server, feature.fabric-groovy=fabric-groovy, feature.fabric-hawtio=fabric-hawtio, feature.fabric-hawtio-swagger=fabric-hawtio-swagger, feature.fabric-jaas=fabric-jaas, feature.fabric-maven-proxy=fabric-maven-proxy, feature.fabric-redirect=fabric-redirect, feature.fabric-ssh=fabric-ssh, feature.fabric-startup=fabric-startup, feature.fabric-web=fabric-web, feature.hawtio-redhat-amq-branding=hawtio-redhat-amq-branding, feature.hawtio-redhat-fuse-branding=hawtio-redhat-fuse-branding, feature.insight-log=insight-log, feature.jolokia=jolokia, feature.karaf=karaf, feature.patch=patch, feature.shell=shell, feature.swagger=swagger, feature.war=war, hash=, io.fabric8.agent.download.threads=4, metadata#org.apache.aries.blueprint.core#[1.0,2.0)#Provide-Capability=osgi.service;effective:=active;objectClass="org.apache.aries.blueprint.services.ParserService", metadata#org.apache.felix.metatype#[1.0,2.0)#Provide-Capability=osgi.service;effective:=active;objectClass="org.osgi.service.metatype.MetaTypeService", metadata#org.apache.felix.scr#[1.0,2.0)#Provide-Capability=osgi.extender;osgi.extender=osgi.service.component,osgi.service;effective:=active;objectClass="org.apache.felix.scr.ScrService", optional.ops4j-base-lang=mvn:org.ops4j.base/ops4j-base-lang/1.5.0, org.ops4j.pax.url.mvn.defaultRepositories=file:/home/fuse/jboss-fuse-6.3.0.redhat-133/system@snapshots@id=karaf-default, file:/home/fuse/jboss-fuse-6.3.0.redhat-133/data/maven/upload@snapshots@id=fabric-upload, file:/home/fuse/.m2/repository@snapshots@id=local, org.ops4j.pax.url.mvn.globalChecksumPolicy=warn, org.ops4j.pax.url.mvn.globalUpdatePolicy=daily, org.ops4j.pax.url.mvn.localRepository=/home/fuse/jboss-fuse-6.3.0.redhat-133/data/repository-agent, org.ops4j.pax.url.mvn.repositories=http://repo1.maven.org/maven2@id=maven.central.repo, https://maven.repository.redhat.com/ga@id=redhat.ga.repo, https://maven.repository.redhat.com/earlyaccess/all@id=redhat.ea.repo, https://repository.jboss.org/nexus/content/groups/ea@id=fuseearlyaccess, org.ops4j.pax.url.mvn.settings=/home/fuse/jboss-fuse-6.3.0.redhat-133/etc/maven-settings.xml, org.ops4j.pax.url.mvn.useFallbackRepositories=false, override.jsch=mvn:org.apache.servicemix.bundles/org.apache.servicemix.bundles.jsch/0.1.53_1, override.pax-logging-api=mvn:org.ops4j.pax.logging/pax-logging-api/1.8.5, override.pax-logging-service=mvn:org.ops4j.pax.logging/pax-logging-service/1.8.5, repository.activemq=mvn:org.apache.activemq/activemq-karaf/5.11.0.redhat-630133/xml/features, repository.apache-camel=mvn:org.apache.camel.karaf/apache-camel/2.17.0.redhat-630133/xml/features, repository.apache-cxf=mvn:org.apache.cxf.karaf/apache-cxf/3.1.5.redhat-630133/xml/features, repository.fabric8=mvn:io.fabric8/fabric8-karaf/1.2.0.redhat-630133/xml/features, repository.fabric8-patch=mvn:io.fabric8.patch/patch-features/1.2.0.redhat-630133/xml/features, repository.fuse=mvn:org.jboss.fuse/jboss-fuse/6.3.0.redhat-133/xml/features, repository.hawtio=mvn:io.hawt/hawtio-karaf/1.4.0.redhat-630133/xml/features, repository.karaf-enterprise=mvn:org.apache.karaf.assemblies.features/enterprise/2.4.0.redhat-630133/xml/features, repository.karaf-spring=mvn:org.apache.karaf.assemblies.features/spring/2.4.0.redhat-630133/xml/features, repository.karaf-standard=mvn:org.apache.karaf.assemblies.features/standard/2.4.0.redhat-630133/xml/features, resolve.optional.imports=true, service.pid=io.fabric8.agent}
      2016-07-19 03:32:50,812 | INFO  | pool-10-thread-2 | HttpServiceFactoryImpl           | 116 - org.ops4j.pax.web.pax-web-runtime - 4.2.7 | Unbinding bundle: [io.fabric8.fabric-redirect [187]]
      2016-07-19 03:32:50,815 | INFO  | pool-10-thread-2 | AuthenticationFilter             | 281 - io.hawt.hawtio-web - 1.4.0.redhat-630133 | Starting hawtio authentication filter, JAAS realm: "karaf" authorized role(s): "admin,manager,viewer,Monitor, Operator, Maintainer, Deployer, Auditor, Administrator, SuperUser" role principal classes: "org.apache.karaf.jaas.boot.principal.RolePrincipal,org.apache.karaf.jaas.modules.RolePrincipal,org.apache.karaf.jaas.boot.principal.GroupPrincipal"
      2016-07-19 03:32:50,816 | INFO  | pool-10-thread-2 | AuthenticationFilter             | 281 - io.hawt.hawtio-web - 1.4.0.redhat-630133 | Starting hawtio authentication filter, JAAS realm: "karaf" authorized role(s): "admin,manager,viewer,Monitor, Operator, Maintainer, Deployer, Auditor, Administrator, SuperUser" role principal classes: "org.apache.karaf.jaas.boot.principal.RolePrincipal,org.apache.karaf.jaas.modules.RolePrincipal,org.apache.karaf.jaas.boot.principal.GroupPrincipal"
      2016-07-19 03:32:50,817 | INFO  | pool-10-thread-2 | AuthenticationFilter             | 281 - io.hawt.hawtio-web - 1.4.0.redhat-630133 | Starting hawtio authentication filter, JAAS realm: "karaf" authorized role(s): "admin,manager,viewer,Monitor, Operator, Maintainer, Deployer, Auditor, Administrator, SuperUser" role principal classes: "org.apache.karaf.jaas.boot.principal.RolePrincipal,org.apache.karaf.jaas.modules.RolePrincipal,org.apache.karaf.jaas.boot.principal.GroupPrincipal"
      2016-07-19 03:32:50,817 | INFO  | pool-10-thread-2 | AuthenticationFilter             | 281 - io.hawt.hawtio-web - 1.4.0.redhat-630133 | Starting hawtio authentication filter, JAAS realm: "karaf" authorized role(s): "admin,manager,viewer,Monitor, Operator, Maintainer, Deployer, Auditor, Administrator, SuperUser" role principal classes: "org.apache.karaf.jaas.boot.principal.RolePrincipal,org.apache.karaf.jaas.modules.RolePrincipal,org.apache.karaf.jaas.boot.principal.GroupPrincipal"
      2016-07-19 03:32:50,817 | INFO  | pool-10-thread-2 | AuthenticationFilter             | 281 - io.hawt.hawtio-web - 1.4.0.redhat-630133 | Starting hawtio authentication filter, JAAS realm: "karaf" authorized role(s): "admin,manager,viewer,Monitor, Operator, Maintainer, Deployer, Auditor, Administrator, SuperUser" role principal classes: "org.apache.karaf.jaas.boot.principal.RolePrincipal,org.apache.karaf.jaas.modules.RolePrincipal,org.apache.karaf.jaas.boot.principal.GroupPrincipal"
      2016-07-19 03:32:50,818 | INFO  | pool-10-thread-2 | AuthenticationFilter             | 281 - io.hawt.hawtio-web - 1.4.0.redhat-630133 | Starting hawtio authentication filter, JAAS realm: "karaf" authorized role(s): "admin,manager,viewer,Monitor, Operator, Maintainer, Deployer, Auditor, Administrator, SuperUser" role principal classes: "org.apache.karaf.jaas.boot.principal.RolePrincipal,org.apache.karaf.jaas.modules.RolePrincipal,org.apache.karaf.jaas.boot.principal.GroupPrincipal"
      2016-07-19 03:32:50,818 | INFO  | pool-10-thread-2 | AuthenticationFilter             | 281 - io.hawt.hawtio-web - 1.4.0.redhat-630133 | Starting hawtio authentication filter, JAAS realm: "karaf" authorized role(s): "admin,manager,viewer,Monitor, Operator, Maintainer, Deployer, Auditor, Administrator, SuperUser" role principal classes: "org.apache.karaf.jaas.boot.principal.RolePrincipal,org.apache.karaf.jaas.modules.RolePrincipal,org.apache.karaf.jaas.boot.principal.GroupPrincipal"
      2016-07-19 03:32:50,818 | INFO  | pool-10-thread-2 | AuthenticationFilter             | 281 - io.hawt.hawtio-web - 1.4.0.redhat-630133 | Starting hawtio authentication filter, JAAS realm: "karaf" authorized role(s): "admin,manager,viewer,Monitor, Operator, Maintainer, Deployer, Auditor, Administrator, SuperUser" role principal classes: "org.apache.karaf.jaas.boot.principal.RolePrincipal,org.apache.karaf.jaas.modules.RolePrincipal,org.apache.karaf.jaas.boot.principal.GroupPrincipal"
      2016-07-19 03:32:50,819 | INFO  | pool-10-thread-2 | AuthenticationFilter             | 281 - io.hawt.hawtio-web - 1.4.0.redhat-630133 | Destroying hawtio authentication filter
      2016-07-19 03:32:50,830 | INFO  | MCF-1-thread-1   | CuratorFrameworkImpl             | 178 - io.fabric8.fabric-zookeeper - 1.2.0.redhat-630133 | Starting
      2016-07-19 03:32:50,832 | WARN  | 5.0.1-1-thread-1 | DeploymentAgent                  | 175 - io.fabric8.fabric-agent - 1.2.0.redhat-630133 | Unable to set provisioning result
      2016-07-19 03:32:50,833 | ERROR | 5.0.1-1-thread-1 | DeploymentAgent                  | 175 - io.fabric8.fabric-agent - 1.2.0.redhat-630133 | Unable to update agent
      java.lang.IllegalStateException: Client is not started
      	at com.google.common.base.Preconditions.checkState(Preconditions.java:173)[131:com.google.guava:18.0.0.redhat-1]
      	at org.apache.curator.CuratorZookeeperClient.getZooKeeper(CuratorZookeeperClient.java:113)[178:io.fabric8.fabric-zookeeper:1.2.0.redhat-630133]
      	at org.apache.curator.framework.imps.CuratorFrameworkImpl.getZooKeeper(CuratorFrameworkImpl.java:474)[178:io.fabric8.fabric-zookeeper:1.2.0.redhat-630133]
      	at org.apache.curator.framework.imps.ExistsBuilderImpl$2.call(ExistsBuilderImpl.java:172)[178:io.fabric8.fabric-zookeeper:1.2.0.redhat-630133]
      	at org.apache.curator.framework.imps.ExistsBuilderImpl$2.call(ExistsBuilderImpl.java:161)[178:io.fabric8.fabric-zookeeper:1.2.0.redhat-630133]
      	at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:107)[178:io.fabric8.fabric-zookeeper:1.2.0.redhat-630133]
      	at org.apache.curator.framework.imps.ExistsBuilderImpl.pathInForeground(ExistsBuilderImpl.java:157)[178:io.fabric8.fabric-zookeeper:1.2.0.redhat-630133]
      	at org.apache.curator.framework.imps.ExistsBuilderImpl.forPath(ExistsBuilderImpl.java:148)[178:io.fabric8.fabric-zookeeper:1.2.0.redhat-630133]
      	at org.apache.curator.framework.imps.ExistsBuilderImpl.forPath(ExistsBuilderImpl.java:36)[178:io.fabric8.fabric-zookeeper:1.2.0.redhat-630133]
      	at io.fabric8.zookeeper.utils.ZooKeeperUtils.getChildrenSafe(ZooKeeperUtils.java:133)[178:io.fabric8.fabric-zookeeper:1.2.0.redhat-630133]
      	at io.fabric8.service.ZkDataStoreImpl.getContainers(ZkDataStoreImpl.java:239)[171:io.fabric8.fabric-core:1.2.0.redhat-630133]
      	at io.fabric8.service.ZkDataStoreImpl.hasContainer(ZkDataStoreImpl.java:248)[171:io.fabric8.fabric-core:1.2.0.redhat-630133]
      	at io.fabric8.service.FabricServiceImpl.getContainer(FabricServiceImpl.java:316)[171:io.fabric8.fabric-core:1.2.0.redhat-630133]
      	at io.fabric8.service.FabricServiceImpl.getCurrentContainer(FabricServiceImpl.java:232)[171:io.fabric8.fabric-core:1.2.0.redhat-630133]
      	at io.fabric8.agent.DeploymentAgent.updateMavenRepositoryConfiguration(DeploymentAgent.java:207)[175:io.fabric8.fabric-agent:1.2.0.redhat-630133]
      	at io.fabric8.agent.DeploymentAgent.doUpdate(DeploymentAgent.java:411)[175:io.fabric8.fabric-agent:1.2.0.redhat-630133]
      	at io.fabric8.agent.DeploymentAgent$4.run(DeploymentAgent.java:289)[175:io.fabric8.fabric-agent:1.2.0.redhat-630133]
      	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)[:1.7.0_75]
      	at java.util.concurrent.FutureTask.run(FutureTask.java:262)[:1.7.0_75]
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)[:1.7.0_75]
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)[:1.7.0_75]
      	at java.lang.Thread.run(Thread.java:745)[:1.7.0_75]
      2016-07-19 03:32:50,837 | WARN  | 5.0.1-1-thread-1 | DeploymentAgent                  | 175 - io.fabric8.fabric-agent - 1.2.0.redhat-630133 | Unable to set provisioning result
      2016-07-19 03:32:50,833 | INFO  | MCF-1-thread-1   | ZooKeeper                        | 178 - io.fabric8.fabric-zookeeper - 1.2.0.redhat-630133 | Initiating client connection, connectString=host-172-16-117-125:2181 sessionTimeout=60000 watcher=org.apache.curator.ConnectionState@3e24578
      2016-07-19 03:32:50,849 | INFO  | ad-1-EventThread | ConnectionStateManager           | 178 - io.fabric8.fabric-zookeeper - 1.2.0.redhat-630133 | State change: CONNECTED
      2016-07-19 03:32:50,856 | INFO  | pool-10-thread-2 | HttpServiceFactoryImpl           | 116 - org.ops4j.pax.web.pax-web-runtime - 4.2.7 | Unbinding bundle: [io.hawt.hawtio-web [281]]
      2016-07-19 03:32:50,856 | INFO  | pool-10-thread-2 | HttpServiceFactoryImpl           | 116 - org.ops4j.pax.web.pax-web-runtime - 4.2.7 | Unbinding bundle: [org.apache.cxf.cxf-rt-transports-http [118]]
      2016-07-19 03:32:50,860 | INFO  | pool-10-thread-2 | ServerConnector                  | 96 - org.eclipse.jetty.util - 9.2.17.v20160517 | Stopped default@5692b1aa{HTTP/1.1}{0.0.0.0:8181}
      2016-07-19 03:32:50,868 | INFO  | pool-10-thread-2 | ContextHandler                   | 96 - org.eclipse.jetty.util - 9.2.17.v20160517 | Stopped HttpServiceContext{httpContext=DefaultHttpContext [bundle=org.apache.cxf.cxf-rt-transports-http [118], contextID=default]}
      2016-07-19 03:32:50,870 | INFO  | pool-10-thread-2 | ContextHandler                   | 96 - org.eclipse.jetty.util - 9.2.17.v20160517 | Stopped HttpServiceContext{httpContext=DefaultHttpContext [bundle=io.fabric8.fabric-redirect [187], contextID=default]}
      2016-07-19 03:32:50,872 | INFO  | pool-10-thread-2 | ContextHandler                   | 96 - org.eclipse.jetty.util - 9.2.17.v20160517 | Stopped HttpServiceContext{httpContext=WebAppHttpContext{io.hawt.hawtio-web - 281}}
      2016-07-19 03:32:50,873 | INFO  | pool-10-thread-2 | ContextHandler                   | 96 - org.eclipse.jetty.util - 9.2.17.v20160517 | Stopped HttpServiceContext{httpContext=WebAppHttpContext{io.hawt.hawtio-karaf-terminal - 283}}
      2016-07-19 03:32:50,874 | INFO  | pool-10-thread-2 | ContextHandler                   | 96 - org.eclipse.jetty.util - 9.2.17.v20160517 | Stopped HttpServiceContext{httpContext=WebAppHttpContext{io.fabric8.support.support-webapp - 287}}
      2016-07-19 03:32:50,877 | INFO  | pool-10-thread-2 | ContextHandler                   | 96 - org.eclipse.jetty.util - 9.2.17.v20160517 | Stopped HttpServiceContext{httpContext=WebAppHttpContext{io.hawt.hawtio-redhat-fuse-branding - 289}}
      2016-07-19 03:32:50,881 | INFO  | pool-10-thread-2 | ContextHandler                   | 96 - org.eclipse.jetty.util - 9.2.17.v20160517 | Stopped HttpServiceContext{httpContext=DefaultHttpContext [bundle=io.fabric8.fabric-maven-proxy [192], contextID=default]}
      2016-07-19 03:32:50,884 | INFO  | pool-10-thread-2 | ContextHandler                   | 96 - org.eclipse.jetty.util - 9.2.17.v20160517 | Stopped HttpServiceContext{httpContext=MavenSecureHttpContext{io.fabric8.fabric-maven-proxy - 192}}
      2016-07-19 03:32:50,890 | ERROR | pool-10-thread-2 | JettyServerImpl                  | 117 - org.ops4j.pax.web.pax-web-jetty - 4.2.7 | Exception while starting Jetty
      java.lang.IllegalStateException: Client is not started
      	at com.google.common.base.Preconditions.checkState(Preconditions.java:173)[131:com.google.guava:18.0.0.redhat-1]
      	at org.apache.curator.CuratorZookeeperClient.getZooKeeper(CuratorZookeeperClient.java:113)[178:io.fabric8.fabric-zookeeper:1.2.0.redhat-630133]
      	at org.apache.curator.framework.imps.CuratorFrameworkImpl.getZooKeeper(CuratorFrameworkImpl.java:474)[178:io.fabric8.fabric-zookeeper:1.2.0.redhat-630133]
      	at org.apache.curator.framework.imps.ExistsBuilderImpl$2.call(ExistsBuilderImpl.java:172)[178:io.fabric8.fabric-zookeeper:1.2.0.redhat-630133]
      	at org.apache.curator.framework.imps.ExistsBuilderImpl$2.call(ExistsBuilderImpl.java:161)[178:io.fabric8.fabric-zookeeper:1.2.0.redhat-630133]
      	at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:107)[178:io.fabric8.fabric-zookeeper:1.2.0.redhat-630133]
      	at org.apache.curator.framework.imps.ExistsBuilderImpl.pathInForeground(ExistsBuilderImpl.java:157)[178:io.fabric8.fabric-zookeeper:1.2.0.redhat-630133]
      	at org.apache.curator.framework.imps.ExistsBuilderImpl.forPath(ExistsBuilderImpl.java:148)[178:io.fabric8.fabric-zookeeper:1.2.0.redhat-630133]
      	at org.apache.curator.framework.imps.ExistsBuilderImpl.forPath(ExistsBuilderImpl.java:36)[178:io.fabric8.fabric-zookeeper:1.2.0.redhat-630133]
      	at io.fabric8.zookeeper.utils.ZooKeeperUtils.getChildrenSafe(ZooKeeperUtils.java:133)[178:io.fabric8.fabric-zookeeper:1.2.0.redhat-630133]
      	at io.fabric8.service.ZkDataStoreImpl.getContainers(ZkDataStoreImpl.java:239)[171:io.fabric8.fabric-core:1.2.0.redhat-630133]
      	at io.fabric8.service.ZkDataStoreImpl.hasContainer(ZkDataStoreImpl.java:248)[171:io.fabric8.fabric-core:1.2.0.redhat-630133]
      	at io.fabric8.service.FabricServiceImpl.getContainer(FabricServiceImpl.java:316)[171:io.fabric8.fabric-core:1.2.0.redhat-630133]
      	at io.fabric8.service.FabricServiceImpl.getCurrentContainer(FabricServiceImpl.java:232)[171:io.fabric8.fabric-core:1.2.0.redhat-630133]
      	at io.fabric8.service.ProfileUrlHandler$Connection.getInputStream(ProfileUrlHandler.java:111)[171:io.fabric8.fabric-core:1.2.0.redhat-630133]
      	at org.apache.xerces.impl.XMLEntityManager.setupCurrentEntity(XMLEntityManager.java:967)[:2.11.0.SP4-redhat-1]
      	at org.apache.xerces.impl.XMLVersionDetector.determineDocVersion(XMLVersionDetector.java:144)[:2.11.0.SP4-redhat-1]
      	at org.apache.xerces.parsers.XML11Configuration.parse(XML11Configuration.java:832)[:2.11.0.SP4-redhat-1]
      	at org.apache.xerces.parsers.XML11Configuration.parse(XML11Configuration.java:798)[:2.11.0.SP4-redhat-1]
      	at org.apache.xerces.parsers.XMLParser.parse(XMLParser.java:108)[:2.11.0.SP4-redhat-1]
      	at org.apache.xerces.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1198)[:2.11.0.SP4-redhat-1]
      	at org.apache.xerces.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:564)[:2.11.0.SP4-redhat-1]
      	at org.apache.xerces.jaxp.SAXParserImpl.parse(SAXParserImpl.java:298)[:2.11.0.SP4-redhat-1]
      	at org.eclipse.jetty.xml.XmlParser.parse(XmlParser.java:210)[100:org.eclipse.jetty.xml:9.2.17.v20160517]
      	at org.eclipse.jetty.xml.XmlParser.parse(XmlParser.java:226)[100:org.eclipse.jetty.xml:9.2.17.v20160517]
      	at org.eclipse.jetty.xml.XmlConfiguration.<init>(XmlConfiguration.java:138)[100:org.eclipse.jetty.xml:9.2.17.v20160517]
      	at org.ops4j.pax.web.service.jetty.internal.JettyServerImpl.start(JettyServerImpl.java:146)[117:org.ops4j.pax.web.pax-web-jetty:4.2.7]
      	at org.ops4j.pax.web.service.jetty.internal.ServerControllerImpl$Stopped.start(ServerControllerImpl.java:415)[117:org.ops4j.pax.web.pax-web-jetty:4.2.7]
      	at org.ops4j.pax.web.service.jetty.internal.ServerControllerImpl.start(ServerControllerImpl.java:71)[117:org.ops4j.pax.web.pax-web-jetty:4.2.7]
      	at org.ops4j.pax.web.service.jetty.internal.ServerControllerImpl$Unconfigured.configure(ServerControllerImpl.java:707)[117:org.ops4j.pax.web.pax-web-jetty:4.2.7]
      	at org.ops4j.pax.web.service.jetty.internal.ServerControllerImpl.configure(ServerControllerImpl.java:87)[117:org.ops4j.pax.web.pax-web-jetty:4.2.7]
      	at org.ops4j.pax.web.service.internal.Activator.updateController(Activator.java:352)[116:org.ops4j.pax.web.pax-web-runtime:4.2.7]
      	at org.ops4j.pax.web.service.internal.Activator$2.run(Activator.java:285)[116:org.ops4j.pax.web.pax-web-runtime:4.2.7]
      	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)[:1.7.0_75]
      	at java.util.concurrent.FutureTask.run(FutureTask.java:262)[:1.7.0_75]
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)[:1.7.0_75]
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)[:1.7.0_75]
      	at java.lang.Thread.run(Thread.java:745)[:1.7.0_75]
      2016-07-19 03:32:50,891 | ERROR | pool-10-thread-2 | Activator                        | 116 - org.ops4j.pax.web.pax-web-runtime - 4.2.7 | Unable to start pax web server: Exception while starting Jetty
      java.lang.RuntimeException: Exception while starting Jetty
      	at org.ops4j.pax.web.service.jetty.internal.JettyServerImpl.start(JettyServerImpl.java:188)[117:org.ops4j.pax.web.pax-web-jetty:4.2.7]
      	at org.ops4j.pax.web.service.jetty.internal.ServerControllerImpl$Stopped.start(ServerControllerImpl.java:415)[117:org.ops4j.pax.web.pax-web-jetty:4.2.7]
      	at org.ops4j.pax.web.service.jetty.internal.ServerControllerImpl.start(ServerControllerImpl.java:71)[117:org.ops4j.pax.web.pax-web-jetty:4.2.7]
      	at org.ops4j.pax.web.service.jetty.internal.ServerControllerImpl$Unconfigured.configure(ServerControllerImpl.java:707)[117:org.ops4j.pax.web.pax-web-jetty:4.2.7]
      	at org.ops4j.pax.web.service.jetty.internal.ServerControllerImpl.configure(ServerControllerImpl.java:87)[117:org.ops4j.pax.web.pax-web-jetty:4.2.7]
      	at org.ops4j.pax.web.service.internal.Activator.updateController(Activator.java:352)[116:org.ops4j.pax.web.pax-web-runtime:4.2.7]
      	at org.ops4j.pax.web.service.internal.Activator$2.run(Activator.java:285)[116:org.ops4j.pax.web.pax-web-runtime:4.2.7]
      	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)[:1.7.0_75]
      	at java.util.concurrent.FutureTask.run(FutureTask.java:262)[:1.7.0_75]
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)[:1.7.0_75]
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)[:1.7.0_75]
      	at java.lang.Thread.run(Thread.java:745)[:1.7.0_75]
      Caused by: java.lang.IllegalStateException: Client is not started
      	at com.google.common.base.Preconditions.checkState(Preconditions.java:173)[131:com.google.guava:18.0.0.redhat-1]
      	at org.apache.curator.CuratorZookeeperClient.getZooKeeper(CuratorZookeeperClient.java:113)[178:io.fabric8.fabric-zookeeper:1.2.0.redhat-630133]
      	at org.apache.curator.framework.imps.CuratorFrameworkImpl.getZooKeeper(CuratorFrameworkImpl.java:474)[178:io.fabric8.fabric-zookeeper:1.2.0.redhat-630133]
      	at org.apache.curator.framework.imps.ExistsBuilderImpl$2.call(ExistsBuilderImpl.java:172)[178:io.fabric8.fabric-zookeeper:1.2.0.redhat-630133]
      	at org.apache.curator.framework.imps.ExistsBuilderImpl$2.call(ExistsBuilderImpl.java:161)[178:io.fabric8.fabric-zookeeper:1.2.0.redhat-630133]
      	at org.apache.curator.RetryLoop.callWithRetry(RetryLoop.java:107)[178:io.fabric8.fabric-zookeeper:1.2.0.redhat-630133]
      	at org.apache.curator.framework.imps.ExistsBuilderImpl.pathInForeground(ExistsBuilderImpl.java:157)[178:io.fabric8.fabric-zookeeper:1.2.0.redhat-630133]
      	at org.apache.curator.framework.imps.ExistsBuilderImpl.forPath(ExistsBuilderImpl.java:148)[178:io.fabric8.fabric-zookeeper:1.2.0.redhat-630133]
      	at org.apache.curator.framework.imps.ExistsBuilderImpl.forPath(ExistsBuilderImpl.java:36)[178:io.fabric8.fabric-zookeeper:1.2.0.redhat-630133]
      	at io.fabric8.zookeeper.utils.ZooKeeperUtils.getChildrenSafe(ZooKeeperUtils.java:133)[178:io.fabric8.fabric-zookeeper:1.2.0.redhat-630133]
      	at io.fabric8.service.ZkDataStoreImpl.getContainers(ZkDataStoreImpl.java:239)[171:io.fabric8.fabric-core:1.2.0.redhat-630133]
      	at io.fabric8.service.ZkDataStoreImpl.hasContainer(ZkDataStoreImpl.java:248)[171:io.fabric8.fabric-core:1.2.0.redhat-630133]
      	at io.fabric8.service.FabricServiceImpl.getContainer(FabricServiceImpl.java:316)[171:io.fabric8.fabric-core:1.2.0.redhat-630133]
      	at io.fabric8.service.FabricServiceImpl.getCurrentContainer(FabricServiceImpl.java:232)[171:io.fabric8.fabric-core:1.2.0.redhat-630133]
      	at io.fabric8.service.ProfileUrlHandler$Connection.getInputStream(ProfileUrlHandler.java:111)[171:io.fabric8.fabric-core:1.2.0.redhat-630133]
      	at org.apache.xerces.impl.XMLEntityManager.setupCurrentEntity(XMLEntityManager.java:967)[:2.11.0.SP4-redhat-1]
      	at org.apache.xerces.impl.XMLVersionDetector.determineDocVersion(XMLVersionDetector.java:144)[:2.11.0.SP4-redhat-1]
      	at org.apache.xerces.parsers.XML11Configuration.parse(XML11Configuration.java:832)[:2.11.0.SP4-redhat-1]
      	at org.apache.xerces.parsers.XML11Configuration.parse(XML11Configuration.java:798)[:2.11.0.SP4-redhat-1]
      	at org.apache.xerces.parsers.XMLParser.parse(XMLParser.java:108)[:2.11.0.SP4-redhat-1]
      	at org.apache.xerces.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1198)[:2.11.0.SP4-redhat-1]
      	at org.apache.xerces.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:564)[:2.11.0.SP4-redhat-1]
      	at org.apache.xerces.jaxp.SAXParserImpl.parse(SAXParserImpl.java:298)[:2.11.0.SP4-redhat-1]
      	at org.eclipse.jetty.xml.XmlParser.parse(XmlParser.java:210)[100:org.eclipse.jetty.xml:9.2.17.v20160517]
      	at org.eclipse.jetty.xml.XmlParser.parse(XmlParser.java:226)[100:org.eclipse.jetty.xml:9.2.17.v20160517]
      	at org.eclipse.jetty.xml.XmlConfiguration.<init>(XmlConfiguration.java:138)[100:org.eclipse.jetty.xml:9.2.17.v20160517]
      	at org.ops4j.pax.web.service.jetty.internal.JettyServerImpl.start(JettyServerImpl.java:146)[117:org.ops4j.pax.web.pax-web-jetty:4.2.7]
      	... 11 more
      2016-07-19 03:32:51,017 | INFO  | MCF-1-thread-1   | ZooKeeper                        | 178 - io.fabric8.fabric-zookeeper - 1.2.0.redhat-630133 | Session: 0x15602103ac20003 closed
      2016-07-19 03:32:51,020 | INFO  | mand.acl.fabric) | SecuredCommandConfigTransformer  | 38 - org.apache.karaf.shell.console - 2.4.0.redhat-630133 | Generating command ACL config org.apache.karaf.command.acl.fabric into service ACL configs [org.apache.karaf.service.acl.command.fabric.profile-scale, org.apache.karaf.service.acl.command.fabric.profile-change-parents, org.apache.karaf.service.acl.command.fabric.ensemble-add, org.apache.karaf.service.acl.command.fabric.container-rollback, org.apache.karaf.service.acl.command.fabric.join, org.apache.karaf.service.acl.command.fabric.crypt-algorithm-set, org.apache.karaf.service.acl.command.fabric.container-connect, org.apache.karaf.service.acl.command.fabric.profile-rename, org.apache.karaf.service.acl.command.fabric.version-set-default, org.apache.karaf.service.acl.command.fabric.require-profile-set, org.apache.karaf.service.acl.command.fabric.profile-display, org.apache.karaf.service.acl.command.fabric.require-profile-delete, org.apache.karaf.service.acl.command.fabric.container-stop, org.apache.karaf.service.acl.command.fabric.ensemble-list, org.apache.karaf.service.acl.command.fabric.crypt-algorithm-get, org.apache.karaf.service.acl.command.fabric.ensemble-password, org.apache.karaf.service.acl.command.fabric.wait-for-provisioning, org.apache.karaf.service.acl.command.fabric.profile-refresh, org.apache.karaf.service.acl.command.fabric.container-delete, org.apache.karaf.service.acl.command.fabric.container-create-child, org.apache.karaf.service.acl.command.fabric.fabric, org.apache.karaf.service.acl.command.fabric.container-default-jvm-options, org.apache.karaf.service.acl.command.fabric.version-list, org.apache.karaf.service.acl.command.fabric.container-jmx-domains, org.apache.karaf.service.acl.command.fabric.container-start, org.apache.karaf.service.acl.command.fabric.container-resolver-set, org.apache.karaf.service.acl.command.fabric.crypt-password-set, org.apache.karaf.service.acl.command.fabric.crypt-password-get, org.apache.karaf.service.acl.command.fabric.container-update-ssh-credentials, org.apache.karaf.service.acl.command.fabric.ensemble-remove, org.apache.karaf.service.acl.command.fabric.container-info, org.apache.karaf.service.acl.command.fabric.encrypt-message, org.apache.karaf.service.acl.command.fabric.status, org.apache.karaf.service.acl.command.fabric.create, org.apache.karaf.service.acl.command.fabric.version-create, org.apache.karaf.service.acl.command.fabric.container-change-profile, org.apache.karaf.service.acl.command.fabric.import, org.apache.karaf.service.acl.command.fabric.version-delete, org.apache.karaf.service.acl.command.fabric.container-remove-profile, org.apache.karaf.service.acl.command.fabric.container-list, org.apache.karaf.service.acl.command.fabric.profile-create, org.apache.karaf.service.acl.command.fabric.require-profile-list, org.apache.karaf.service.acl.command.fabric.mq-create, org.apache.karaf.service.acl.command.fabric.profile-copy, org.apache.karaf.service.acl.command.fabric.container-edit-jvm-options, org.apache.karaf.service.acl.command.fabric.cluster-list, org.apache.karaf.service.acl.command.fabric.container-upgrade, org.apache.karaf.service.acl.command.fabric.profile-edit, org.apache.karaf.service.acl.command.fabric.profile-list, org.apache.karaf.service.acl.command.fabric.container-add-profile, org.apache.karaf.service.acl.command.fabric.patch-apply, org.apache.karaf.service.acl.command.fabric.export, org.apache.karaf.service.acl.command.fabric.profile-delete]
      2016-07-19 03:32:51,035 | INFO  | MCF-1-thread-1   | ZooKeeperGroup                   | 172 - io.fabric8.fabric-groups - 1.2.0.redhat-630133 | Creating ZK Group for path "/fabric/registry/clusters/git"
      2016-07-19 03:32:51,036 | INFO  | MCF-1-thread-1   | ZooKeeperGroup                   | 172 - io.fabric8.fabric-groups - 1.2.0.redhat-630133 | Starting ZK Group for path "/fabric/registry/clusters/git"
      2016-07-19 03:32:51,040 | INFO  | 6b-9b6a0560d0b0) | ActiveMQServiceFactory           | 230 - io.fabric8.mq.mq-fabric - 1.2.0.redhat-630133 | Stopping the broker amq.
      2016-07-19 03:32:51,041 | INFO  | AMQ-1-thread-1   | BrokerService                    | 218 - org.apache.activemq.activemq-osgi - 5.11.0.redhat-630133 | Apache ActiveMQ 5.11.0.redhat-630133 (amq, ID:host-172-16-117-126-40632-1468913419990-0:1) is shutting down
      2016-07-19 03:32:51,042 | INFO  | AMQ-1-thread-1   | TransportConnector               | 218 - org.apache.activemq.activemq-osgi - 5.11.0.redhat-630133 | Connector openwire stopped
      2016-07-19 03:32:51,044 | INFO  | AMQ-1-thread-1   | PListStoreImpl                   | 218 - org.apache.activemq.activemq-osgi - 5.11.0.redhat-630133 | PListStore:[/home/fuse/jboss-fuse-6.3.0.redhat-133/data/amq/amq/tmp_storage] stopped
      2016-07-19 03:32:51,044 | INFO  | AMQ-1-thread-1   | KahaDBStore                      | 218 - org.apache.activemq.activemq-osgi - 5.11.0.redhat-630133 | Stopping async queue tasks
      2016-07-19 03:32:51,045 | INFO  | AMQ-1-thread-1   | KahaDBStore                      | 218 - org.apache.activemq.activemq-osgi - 5.11.0.redhat-630133 | Stopping async topic tasks
      2016-07-19 03:32:51,045 | INFO  | AMQ-1-thread-1   | KahaDBStore                      | 218 - org.apache.activemq.activemq-osgi - 5.11.0.redhat-630133 | Stopped KahaDB
      2016-07-19 03:32:51,065 | INFO  | AMQ-1-thread-1   | BrokerService                    | 218 - org.apache.activemq.activemq-osgi - 5.11.0.redhat-630133 | Apache ActiveMQ 5.11.0.redhat-630133 (amq, ID:host-172-16-117-126-40632-1468913419990-0:1) uptime 2 minutes
      2016-07-19 03:32:51,065 | INFO  | AMQ-1-thread-1   | BrokerService                    | 218 - org.apache.activemq.activemq-osgi - 5.11.0.redhat-630133 | Apache ActiveMQ 5.11.0.redhat-630133 (amq, ID:host-172-16-117-126-40632-1468913419990-0:1) is shutdown
      2016-07-19 03:32:51,065 | INFO  | AMQ-1-thread-1   | ActiveMQServiceFactory$1         | 204 - org.apache.servicemix.bundles.spring-context - 3.2.16.RELEASE_1 | Closing io.fabric8.mq.fabric.ActiveMQServiceFactory$1@730a1ce9: startup date [Tue Jul 19 03:30:17 EDT 2016]; root of context hierarchy
      2016-07-19 03:32:51,065 | INFO  | AMQ-1-thread-1   | DefaultListableBeanFactory       | 202 - org.apache.servicemix.bundles.spring-beans - 3.2.16.RELEASE_1 | Destroying singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@76b6a229: defining beans [org.springframework.beans.factory.config.PropertyPlaceholderConfigurer#0,org.apache.activemq.xbean.XBeanBrokerService#0]; root of factory hierarchy
      2016-07-19 03:32:51,161 | INFO  | MCF-1-thread-1   | GitDataStoreImpl                 | 173 - io.fabric8.fabric-git - 1.2.0.redhat-630133 | Starting up GitDataStore io.fabric8.git.internal.GitDataStoreImpl@5c71cdd9
      2016-07-19 03:32:51,162 | INFO  | MCF-1-thread-1   | GitDataStoreImpl                 | 173 - io.fabric8.fabric-git - 1.2.0.redhat-630133 | Using ZooKeeper SharedCount to react when master git repo is changed, so we can do a git pull to the local git repo.
      2016-07-19 03:32:51,168 | INFO  | MCF-1-thread-1   | DefaultPullPushPolicy            | 173 - io.fabric8.fabric-git - 1.2.0.redhat-630133 | Performing a pull on remote URL: http://host-172-16-117-125:8181/git/fabric/
      2016-07-19 03:32:51,187 | INFO  | MCF-1-thread-1   | DefaultPullPushPolicy            | 173 - io.fabric8.fabric-git - 1.2.0.redhat-630133 | Pull result: [localUpdate=[],remoteUpdate=false,versions=[1.0, master, patches-child, patches-root-amq, patches-root-fuse, patches-ssh-fabric8, patches-ssh-fuse],error=null]
      2016-07-19 03:32:51,356 | INFO  | cache-4-thread-1 | ZkDataStoreImpl                  | 171 - io.fabric8.fabric-core - 1.2.0.redhat-630133 | Event CHILD_ADDED detected on /fabric/configs/versions/1.0/containers/xx with data fabric. Sending notification.
      2016-07-19 03:32:51,358 | INFO  | cache-4-thread-1 | ZkDataStoreImpl                  | 171 - io.fabric8.fabric-core - 1.2.0.redhat-630133 | Event CHILD_ADDED detected on /fabric/configs/ensemble with data 0000. Sending notification.
      2016-07-19 03:32:51,358 | INFO  | cache-4-thread-1 | ZkDataStoreImpl                  | 171 - io.fabric8.fabric-core - 1.2.0.redhat-630133 | Event CHILD_ADDED detected on /fabric/configs/containers/xx with data 1.0. Sending notification.
      2016-07-19 03:32:51,358 | INFO  | cache-4-thread-1 | ZkDataStoreImpl                  | 171 - io.fabric8.fabric-core - 1.2.0.redhat-630133 | Event CHILD_ADDED detected on /fabric/configs/ensemble/url with data ${zk:root/ip}:2181. Sending notification.
      

      It seems the connection is re-established after the initial failure, but the provision status is not set again. This can break automation of creating clusters with fabric:join (as we wait for the success provision status).

      JBossFuse:karaf@root> container-list
      [id]   [version]  [type]  [connected]  [profiles]              [provision status]
      root*  1.0        karaf   yes          fabric                  success           
                                             fabric-ensemble-0000-1                    
                                             jboss-fuse-full                           
      xx     1.0        karaf   yes          fabric
      

      This can be worked around by restarting the joined container after joining into the fabric:

      JBossFuse:karaf@root> container-list
      [id]   [version]  [type]  [connected]  [profiles]              [provision status]
      root*  1.0        karaf   yes          fabric                  success           
                                             fabric-ensemble-0000-1                    
                                             jboss-fuse-full                           
      xx     1.0        karaf   yes          fabric                  success 
      

      Attachments

        Issue Links

          Activity

            People

              pantinor@redhat.com Paolo Antinori
              avano@redhat.com Andrej Vano
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: