Uploaded image for project: 'AMQ Broker'
  1. AMQ Broker
  2. ENTMQBR-3068

Console fails to start when deployed via StatefulSet in certain projects

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Obsolete
    • Icon: Minor Minor
    • None
    • AMQ 7.5.0.GA, AMQ 7.6.0.GA
    • container image
    • None
    • Hide

      1. Create a project in Openshift containing the characters "--"

      2. Deploy AMQ using one of the "persistent" templates. I used this (but it should also be reproducible without SSL):
      https://github.com/jboss-container-images/jboss-amq-7-broker-openshift-image/blob/amq-broker-75/templates/amq-broker-75-persistence-ssl.yaml

      3. Observe the following error in the broker pod's logs upon startup:

      [Fatal Error] :26:84: The string "--" is not permitted within comments.
      2019-11-08 20:58:47,820 WARN  [org.eclipse.jetty.ContextHandler.console] unavailable: java.lang.IllegalArgumentException: Cannot create an instance of custom restrictor class class io.hawt.web.RBACRestrictor
      	at org.jolokia.restrictor.RestrictorFactory.lookupRestrictor(RestrictorFactory.java:97) [jolokia-core-1.6.1.redhat-1.jar:]
      	at org.jolokia.restrictor.RestrictorFactory.createCustomRestrictor(RestrictorFactory.java:74) [jolokia-core-1.6.1.redhat-1.jar:]
      	at org.jolokia.restrictor.RestrictorFactory.createRestrictor(RestrictorFactory.java:42) [jolokia-core-1.6.1.redhat-1.jar:]
      	at org.jolokia.http.AgentServlet.createRestrictor(AgentServlet.java:195) [jolokia-core-1.6.1.redhat-1.jar:]
      	at org.jolokia.http.AgentServlet.init(AgentServlet.java:135) [jolokia-core-1.6.1.redhat-1.jar:]
      	at io.hawt.web.JolokiaConfiguredAgentServlet.init(JolokiaConfiguredAgentServlet.java:60) [hawtio-system-1.4.0.redhat-630416.jar:1.4.0.redhat-630416]
      	at org.eclipse.jetty.servlet.ServletHolder.initServlet(ServletHolder.java:637)
      	at org.eclipse.jetty.servlet.ServletHolder.initialize(ServletHolder.java:421)
      	at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:744)
      	at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:348)
      	at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1480)
      	at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1442)
      	at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:799)
      	at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:261)
      	at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:540)
      	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
      	at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:131)
      	at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:113)
      	at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
      	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
      	at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:131)
      	at org.eclipse.jetty.server.Server.start(Server.java:452)
      	at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:105)
      	at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
      	at org.eclipse.jetty.server.Server.doStart(Server.java:419)
      	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
      	at org.apache.activemq.artemis.component.WebServerComponent.start(WebServerComponent.java:229) [artemis-web-2.10.0.redhat-00004.jar:2.10.0.redhat-00004]
      	at org.apache.activemq.artemis.cli.commands.Run.execute(Run.java:105) [artemis-cli-2.10.0.redhat-00004.jar:2.10.0.redhat-00004]
      	at org.apache.activemq.artemis.cli.Artemis.internalExecute(Artemis.java:150) [artemis-cli-2.10.0.redhat-00004.jar:2.10.0.redhat-00004]
      	at org.apache.activemq.artemis.cli.Artemis.execute(Artemis.java:98) [artemis-cli-2.10.0.redhat-00004.jar:2.10.0.redhat-00004]
      	at org.apache.activemq.artemis.cli.Artemis.execute(Artemis.java:125) [artemis-cli-2.10.0.redhat-00004.jar:2.10.0.redhat-00004]
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.8.0_232]
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) [rt.jar:1.8.0_232]
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_232]
      	at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_232]
      	at org.apache.activemq.artemis.boot.Artemis.execute(Artemis.java:129) [artemis-boot.jar:2.10.0.redhat-00004]
      	at org.apache.activemq.artemis.boot.Artemis.main(Artemis.java:49) [artemis-boot.jar:2.10.0.redhat-00004]
      Caused by: java.lang.reflect.InvocationTargetException
      	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) [rt.jar:1.8.0_232]
      	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) [rt.jar:1.8.0_232]
      	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) [rt.jar:1.8.0_232]
      	at java.lang.reflect.Constructor.newInstance(Constructor.java:423) [rt.jar:1.8.0_232]
      	at org.jolokia.restrictor.RestrictorFactory.lookupRestrictor(RestrictorFactory.java:82) [jolokia-core-1.6.1.redhat-1.jar:]
      	... 36 more
      Caused by: java.lang.SecurityException: Cannot parse policy file: org.xml.sax.SAXParseException; lineNumber: 26; columnNumber: 84; The string "--" is not permitted within comments.
      	at org.jolokia.restrictor.PolicyRestrictor.<init>(PolicyRestrictor.java:85) [jolokia-core-1.6.1.redhat-1.jar:]
      	at org.jolokia.restrictor.RestrictorFactory.lookupPolicyRestrictor(RestrictorFactory.java:121) [jolokia-core-1.6.1.redhat-1.jar:]
      	at io.hawt.web.RBACRestrictor.initDelegate(RBACRestrictor.java:55) [hawtio-system-1.4.0.redhat-630416.jar:1.4.0.redhat-630416]
      	at io.hawt.web.RBACRestrictor.<init>(RBACRestrictor.java:49) [hawtio-system-1.4.0.redhat-630416.jar:1.4.0.redhat-630416]
      	at io.hawt.web.RBACRestrictor.<init>(RBACRestrictor.java:45) [hawtio-system-1.4.0.redhat-630416.jar:1.4.0.redhat-630416]
      	... 41 more
      Caused by: org.xml.sax.SAXParseException; lineNumber: 26; columnNumber: 84; The string "--" is not permitted within comments.
      	at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:257) [rt.jar:1.8.0_232]
      	at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:339) [rt.jar:1.8.0_232]
      	at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:121) [rt.jar:1.8.0_232]
      	at org.jolokia.restrictor.PolicyRestrictor.createDocument(PolicyRestrictor.java:104) [jolokia-core-1.6.1.redhat-1.jar:]
      	at org.jolokia.restrictor.PolicyRestrictor.<init>(PolicyRestrictor.java:72) [jolokia-core-1.6.1.redhat-1.jar:]
      	... 45 more
      
      Show
      1. Create a project in Openshift containing the characters "--" 2. Deploy AMQ using one of the "persistent" templates. I used this (but it should also be reproducible without SSL): https://github.com/jboss-container-images/jboss-amq-7-broker-openshift-image/blob/amq-broker-75/templates/amq-broker-75-persistence-ssl.yaml 3. Observe the following error in the broker pod's logs upon startup: [Fatal Error] :26:84: The string "--" is not permitted within comments. 2019-11-08 20:58:47,820 WARN [org.eclipse.jetty.ContextHandler.console] unavailable: java.lang.IllegalArgumentException: Cannot create an instance of custom restrictor class class io.hawt.web.RBACRestrictor at org.jolokia.restrictor.RestrictorFactory.lookupRestrictor(RestrictorFactory.java:97) [jolokia-core-1.6.1.redhat-1.jar:] at org.jolokia.restrictor.RestrictorFactory.createCustomRestrictor(RestrictorFactory.java:74) [jolokia-core-1.6.1.redhat-1.jar:] at org.jolokia.restrictor.RestrictorFactory.createRestrictor(RestrictorFactory.java:42) [jolokia-core-1.6.1.redhat-1.jar:] at org.jolokia.http.AgentServlet.createRestrictor(AgentServlet.java:195) [jolokia-core-1.6.1.redhat-1.jar:] at org.jolokia.http.AgentServlet.init(AgentServlet.java:135) [jolokia-core-1.6.1.redhat-1.jar:] at io.hawt.web.JolokiaConfiguredAgentServlet.init(JolokiaConfiguredAgentServlet.java:60) [hawtio-system-1.4.0.redhat-630416.jar:1.4.0.redhat-630416] at org.eclipse.jetty.servlet.ServletHolder.initServlet(ServletHolder.java:637) at org.eclipse.jetty.servlet.ServletHolder.initialize(ServletHolder.java:421) at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:744) at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:348) at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1480) at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1442) at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:799) at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:261) at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:540) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:131) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:113) at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:131) at org.eclipse.jetty.server.Server.start(Server.java:452) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:105) at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113) at org.eclipse.jetty.server.Server.doStart(Server.java:419) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) at org.apache.activemq.artemis.component.WebServerComponent.start(WebServerComponent.java:229) [artemis-web-2.10.0.redhat-00004.jar:2.10.0.redhat-00004] at org.apache.activemq.artemis.cli.commands.Run.execute(Run.java:105) [artemis-cli-2.10.0.redhat-00004.jar:2.10.0.redhat-00004] at org.apache.activemq.artemis.cli.Artemis.internalExecute(Artemis.java:150) [artemis-cli-2.10.0.redhat-00004.jar:2.10.0.redhat-00004] at org.apache.activemq.artemis.cli.Artemis.execute(Artemis.java:98) [artemis-cli-2.10.0.redhat-00004.jar:2.10.0.redhat-00004] at org.apache.activemq.artemis.cli.Artemis.execute(Artemis.java:125) [artemis-cli-2.10.0.redhat-00004.jar:2.10.0.redhat-00004] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.8.0_232] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) [rt.jar:1.8.0_232] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_232] at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_232] at org.apache.activemq.artemis.boot.Artemis.execute(Artemis.java:129) [artemis-boot.jar:2.10.0.redhat-00004] at org.apache.activemq.artemis.boot.Artemis.main(Artemis.java:49) [artemis-boot.jar:2.10.0.redhat-00004] Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) [rt.jar:1.8.0_232] at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) [rt.jar:1.8.0_232] at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) [rt.jar:1.8.0_232] at java.lang.reflect.Constructor.newInstance(Constructor.java:423) [rt.jar:1.8.0_232] at org.jolokia.restrictor.RestrictorFactory.lookupRestrictor(RestrictorFactory.java:82) [jolokia-core-1.6.1.redhat-1.jar:] ... 36 more Caused by: java.lang.SecurityException: Cannot parse policy file: org.xml.sax.SAXParseException; lineNumber: 26; columnNumber: 84; The string "--" is not permitted within comments. at org.jolokia.restrictor.PolicyRestrictor.<init>(PolicyRestrictor.java:85) [jolokia-core-1.6.1.redhat-1.jar:] at org.jolokia.restrictor.RestrictorFactory.lookupPolicyRestrictor(RestrictorFactory.java:121) [jolokia-core-1.6.1.redhat-1.jar:] at io.hawt.web.RBACRestrictor.initDelegate(RBACRestrictor.java:55) [hawtio-system-1.4.0.redhat-630416.jar:1.4.0.redhat-630416] at io.hawt.web.RBACRestrictor.<init>(RBACRestrictor.java:49) [hawtio-system-1.4.0.redhat-630416.jar:1.4.0.redhat-630416] at io.hawt.web.RBACRestrictor.<init>(RBACRestrictor.java:45) [hawtio-system-1.4.0.redhat-630416.jar:1.4.0.redhat-630416] ... 41 more Caused by: org.xml.sax.SAXParseException; lineNumber: 26; columnNumber: 84; The string "--" is not permitted within comments. at com.sun.org.apache.xerces.internal.parsers.DOMParser.parse(DOMParser.java:257) [rt.jar:1.8.0_232] at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderImpl.parse(DocumentBuilderImpl.java:339) [rt.jar:1.8.0_232] at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:121) [rt.jar:1.8.0_232] at org.jolokia.restrictor.PolicyRestrictor.createDocument(PolicyRestrictor.java:104) [jolokia-core-1.6.1.redhat-1.jar:] at org.jolokia.restrictor.PolicyRestrictor.<init>(PolicyRestrictor.java:72) [jolokia-core-1.6.1.redhat-1.jar:] ... 45 more

      When one of the "persistent" OpenShift templates for AMQ 7.5 is deployed into a project/namespace containing two hyphens --, the AMQ Console fails to start with an error:

      [Fatal Error] :26:84: The string "--" is not permitted within comments.
      

      This is because the pod's full hostname is inserted into a comment in jolokia-access.xml. In the case of a statefulset, this includes the pod's namespace which contains the invalid string.

      sh-4.2$ hostname -f
      broker-amq-0.broker-amq-headless.rhsm--prototype.svc.cluster.local
      
      sh-4.2$ cat ~/broker/etc/jolokia-access.xml 
      <?xml version="1.0" encoding="utf-8"?>
      <!--
      Licensed to the Apache Software Foundation (ASF) under one
      or more contributor license agreements.  See the NOTICE file
      distributed with this work for additional information
      regarding copyright ownership.  The ASF licenses this file
      to you under the Apache License, Version 2.0 (the
      "License"); you may not use this file except in compliance
      with the License.  You may obtain a copy of the License at
      
        http://www.apache.org/licenses/LICENSE-2.0
      
      Unless required by applicable law or agreed to in writing,
      software distributed under the License is distributed on an
      "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
      KIND, either express or implied.  See the License for the
      specific language governing permissions and limitations
      under the License.
      -->
      
      <!-- This policy file controls the Jolokia JMX-HTTP bridge security options for the web console.
         see: https://jolokia.org/reference/html/security.html -->
      <restrict>
      
          <cors>
              <!-- Allow cross origin access from broker-amq-0.broker-amq-headless.rhsm--prototype.svc.cluster.local ... -->
              <allow-origin>*://broker-amq-0.broker-amq-headless.rhsm--prototype.svc.cluster.local*</allow-origin>
      
      
              <!-- Options from this point on are auto-generated by Create.java from the Artemis CLI -->
              <!-- Check for the proper origin on the server side, too -->
              
          </cors>
      
      </restrict>
      

      Can the template for jolokia-access.xml be updated so that the pod's hostname is not substituted in that comment?

              rhn-support-rkieley Roderick Kieley
              peasters@redhat.com Patrick Easters (Inactive)
              Mikhail Krutov Mikhail Krutov
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Created:
                Updated:
                Resolved: