I wrote a rule to detect logging to file which evolved from an existing Windup rule. I also created test data including XML and properties files for Log4J/LOG4J2/Logback.

However, in testing I encountered this issue:

• The number of detected incidents varies from run to run, sometimes detecting 2 incidents, sometimes 3, even though the rule and test data never changed.
• Files that were not detected in one run, e.g. log4j.properties, were detected when running Windup on just that single file instead of the whole test data.

This issue has been really bothering me. I spent hours troubleshooting my test to try and find the error in my rule, but now it seems the issue is not due to a rule error.  

Hope to get a reply on this issue soon. Thanks.