The algorithm which propagates outstanding monitor enters through the control flow graph is failing to correctly propagate opens along exception control flow paths.

Firstly the algorithm has an error in it which means it does not correctly detect when an exception flow can propagate from a normal block to a handler block. This needs fixing by performing the propagation block by block in method carryForward. Each exit open at the end of the block and each exit closed in the block needs to be checked against each active try catch region (i.e. one which overlaps the block). The comparison needs to restrict the enter/exit and try/catch regions to the block itself and then see if they overlap. This constitutes a potential exception flow. However, there is a futher wrinkle.

The second problem with the current algorithm is that it fails to identify cases where a nested try catch region shadows an outer region and encloses the enter/exit i.e. if the nested region catches type A and the outer region catches type B and either A isSuper , A == B or A == null (catch-all). We cannot detect A< B without loading code but we can detect the other two cases. So, a potential exception flow must be checked by tracking shadowing regions and, for each block, ensuring that no shadowing occurs.