See the attachments on original JBEAP-12366 jira

Description of problem:
CLI is unable to start embed-server multiple times in a loop

CLI freeze or throws java.io.FileNotFoundException: /home/mkopecky/playground/eap/7.1.0.ER2.2/jboss-eap-7.1/standalone/log/server.log (Too many open files)

Priority of this jira is not a blocker, because this is not regression against 7.0.0.

How reproducible:
Always

Steps to Reproduce:

addDate() {
    while IFS= read -r line; do
        echo "`date '+%d.%m.%Y %T'`  $line"
    done
}
rm -f script.cli
for (( i = 1; i <= 20000 ; i++ )) ; do
   echo "echo iteration ${i}" >> script.cli
   echo "embed-server" >> script.cli
   echo "stop-embedded-server" >> script.cli
done
export JAVA_OPTS="-Xmx100m"
./jboss-cli.sh < script.cli 2>&1 | addDate | tee output.txt

Actual results:
I perform more runs

• -Xmx90m: CLI freeze after 1100 iterations
• -Xmx100m: CLI freeze after 1339 iterations
  • thread dump: [^xmx100m.thread.dump.txt]
  • heap dump: [^xmx100m.heapdump.hprof.zip]
  • cli output with time stamps: [^xmx100m.output.txt]
• -Xmx300m: CLI is not able to start embed-server 3836 iterations
  • my hard limit of open files is 4096, my soft limit of open files is 1024
  • cli output with time stamps: [^xmx300m.output.txt]
  • java.io.FileNotFoundException: /home/mkopecky/playground/eap/7.1.0.ER2.2/jboss-eap-7.1/standalone/log/server.log (Too many open files)

Additional info:
In EAP 7.0.0, output of CLI:

26.07.2017 14:27:35  [disconnected /] embed-server
26.07.2017 14:27:39  Cannot start embedded server: WFLYEMB0021: Cannot start embedded process: Operation failed: WFLYSRV0056: Server boot has failed in an unrecoverable manner; exiting. See previous messages for details.
26.07.2017 14:27:39  [disconnected /]