It is not possible to set jboss.modules.system.pkgs property per server / server group in domain configuration. Even when the following configuration is added to host.xml:

<server name="server-one" group="main-server-group">
  <jvm name="default">
    <jvm-options>
      <option value="-Djboss.modules.system.pkgs=my.package"/>
...

It is overridden by configuration in domain.conf:

JAVA_OPTS="$JAVA_OPTS -Djboss.modules.system.pkgs=$JBOSS_MODULES_SYSTEM_PKGS -Djava.awt.headless=true"

This is an issue for configuring third-party java agents per-server. A problem described in WFLY-895 adds to this. One should either add a whole bunch of things to all jvms in domain in order to make agent work, or remove config from domain.conf to avoid overrides.