I have attached a reproducer named NotRemovingRules.java that creates a KieContainer with a KieModule that contains a single rule (rule name 1081) and then runs a loop that adds 2 rules and removes 2 rules and calls KieContainer.updateToVersion(). Along the way it creates .dmp files.
If you examine the last .dmp file created names ckpoint3.drl you will see that rules named like
org.drools.compiler.integraionTests.Rule_120#0DefaultConsequenceInvoker are still present. However, if you examine the console or System.out you will notice the code is executing that is removing the rules from the KieModule/KieContainer.