Manage potential race to inject same rule from concurrent threads

The JVM may initiate resolution of the same class from different threads. This can cause a problem for bytecode transformers because the bytecode retrieved from the relevant classloader (normally by loading it from a file) may be scheduled for transformation in more than one resolving thread. A single transformed byte array returned by one of the threads will be accepted as the definitive version from which the class is subsequently defined and resolved. Bytecode returned by other concurrently initiated transformations may be discarded by the JVM.

The JVM does not notify success or failure of specific transformations. Furthermore, it makes no guarantees regarding timing. So, rejected transforms may terminate after the class has been defined and resolved. It is even possible for a rejected transformation to terminate after the successfully transformed bytcode has been executed.

The problem presents a true dilemma with two perils that need dodging. The dilemma arises when the transformation process needs to record details of the transformation it performs, i.e. state that allows injected code to operate. How is the agent to allocate index and subsequently free that state.

For Byteman there is the need to associate a parsed Rule instance with the injected code, allowing the rule to be type-checked and (initially) interpreted when the injected code is first executed. Assume that a rule (script) r is to be injected into method m of a class C that is being resolved. Let R1, R2 ... be the instances of class Rule created from r in threads t1, t2 ... etc. Finally, assume that the JVM accepts the bytecode array returned by thread tk associated with rule Rk to define class C and rejects all other

If Byteman retains references to all the rules R1, R2, ... then it needs to be able to index all such rules even though only one associated transform is active and to garbage collect the redundant rules as soon as possible.

If Byteman retains reference to a single live rule and drops all other references then it must either i) somehow manage to identify which resolving thread has won the transformation race (well, that's not an option) or ensure that it makes no difference to the injected code which of the rules R1, R2, ... is retained.

The former tack is problematic. The obvious point to cull rules is at the point where the successfully transformed code is executed. At that point all other rules associated with r, C and m can be discarded. Except, new rules may still be in the process of being created in other resolving threads. Also, to make things more complicated retaining and then trashing multiple versions of rules runs the risk of running extra, redundant Helper install and uninstall lifecycle callbacks.

The latter tack has been tried but implemented incorrectly. Byteman currently retains the latest Rule created form a transform up to a successful execute and then rejects subsequent rules. However, it fails to ensure that the retained will operate with whatever injected code is accepted and also fails successfully to distinguish rules created via rule redefinitions from rules created by racing transform threads.