A RULE local var binding may sometimes attempt to downcast an initializater value during initial assignment of the variable. This is particularly likely when the initializer is a ternary ?: expression with one of the limbs having type null. The interpret method for Binding attempts to ensure that the value is assignable to the downcast type by calling Class.isInstance. This works if the value is non-null but results in a ClassCastException if the value is null.

Note that this problem does not occur when the rule is compiled to bytecode. The checkcast instruction planted to ensure assignability allows null vales to pass through.

Original discussion follows:

I have the following rule definition:

RULE OpenTracing SpecialAgent Intercept for ClassLoader#findClass

CLASS ^java.lang.ClassLoader

METHOD Class findClass(String)
AT EXIT
BIND
  bytecode:byte[] = $! == null ? io.opentracing.contrib.specialagent.Agent.findClass($0, $1) : null;
IF bytecode != null
DO
  traceln(">>>>>>>> defineClass(" + $1 + ")");
  RETURN $0.defineClass($1, bytecode, 0, bytecode.length, null);
ENDRULE

The return type of `io.opentracing.contrib.specialagent.Agent.findClass($0, $1)` is `byte[]`.

Upon the type check phase, I receive the following exception:

org.jboss.byteman.rule.exception.TypeException: Variable.typeCheck() : invalid result type : byte[]

If I remove the explicit mention of `byte[]` from the rule, it seems to pass the type check, but then I receive the following exception during the interpret phase:

java.lang.ClassCastException: Cannot cast null to class byte[]
	at org.jboss.byteman.rule.binding.Binding.interpret(Binding.java:316)
	at org.jboss.byteman.rule.Event.interpret(Event.java:292)
	at org.jboss.byteman.rule.helper.InterpretedHelper.bind(InterpretedHelper.java:156)
	at org.jboss.byteman.rule.helper.InterpretedHelper.execute0(InterpretedHelper.java:136)
	at org.jboss.byteman.rule.helper.InterpretedHelper.execute(InterpretedHelper.java:100)
	at org.jboss.byteman.rule.Rule.execute(Rule.java:808)
	at org.jboss.byteman.rule.Rule.execute(Rule.java:777)
	at java.net.URLClassLoader.findClass(URLClassLoader.java:383)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:338)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
        ...

I'm not sure whether this is a bug, or my oversight regarding a detail of the Byteman rule semantics that I'm missing here.

Why is the type check failing when `byte[]` is explicitly specified in the rule?

When the explicit `byte[]` type is removed from the rule, why would `null` fail to be cast to `byte[]`? It seems that the type assignment in this case is not correct.