I encountered a rare ConcurrentModificationException while starting the all profile of JBoss AS 5.0.0.GA:

10:34:45,077 ERROR [AbstractKernelController] Error installing to Start: name=HAPartition state=Create
org.jgroups.ChannelException: connect() failed
at org.jgroups.JChannel.connect(JChannel.java:374)
at org.jboss.ha.framework.server.ClusterPartition$ChannelConnectTask.run(ClusterPartition.java:167)
at org.jboss.util.threadpool.RunnableTaskWrapper.run(RunnableTaskWrapper.java:147)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
at java.lang.Thread.run(Thread.java:595)
Caused by: java.util.ConcurrentModificationException
at java.util.AbstractList$Itr.checkForComodification(AbstractList.java:449)
at java.util.AbstractList$Itr.next(AbstractList.java:420)
at java.util.Collections$UnmodifiableCollection$1.next(Collections.java:1010)
at java.util.AbstractCollection.addAll(AbstractCollection.java:318)
at org.jgroups.protocols.TP.handleDownEvent(TP.java:1575)
at org.jgroups.protocols.TP.down(TP.java:1148)
at org.jgroups.protocols.TP$ProtocolAdapter.down(TP.java:2308)
at org.jgroups.protocols.Discovery.down(Discovery.java:354)
at org.jgroups.protocols.MERGE2.down(MERGE2.java:153)
at org.jgroups.protocols.FD_SOCK.down(FD_SOCK.java:370)
at org.jgroups.protocols.FD.down(FD.java:294)
at org.jgroups.protocols.VERIFY_SUSPECT.down(VERIFY_SUSPECT.java:95)
at org.jgroups.protocols.pbcast.NAKACK.down(NAKACK.java:672)
at org.jgroups.protocols.UNICAST.down(UNICAST.java:462)
at org.jgroups.protocols.pbcast.STABLE.down(STABLE.java:299)
at org.jgroups.protocols.pbcast.GMS.installView(GMS.java:545)
at org.jgroups.protocols.pbcast.GMS.installView(GMS.java:453)
at org.jgroups.protocols.pbcast.ClientGmsImpl.becomeSingletonMember(ClientGmsImpl.java:356)
at org.jgroups.protocols.pbcast.ClientGmsImpl.join(ClientGmsImpl.java:86)
at org.jgroups.protocols.pbcast.ClientGmsImpl.join(ClientGmsImpl.java:39)
at org.jgroups.protocols.pbcast.GMS.down(GMS.java:832)
at org.jgroups.protocols.FC.down(FC.java:430)
at org.jgroups.protocols.FRAG2.down(FRAG2.java:158)
at org.jgroups.protocols.pbcast.STATE_TRANSFER.down(STATE_TRANSFER.java:200)
at org.jgroups.protocols.pbcast.FLUSH.down(FLUSH.java:272)
at org.jgroups.stack.ProtocolStack.down(ProtocolStack.java:475)
at org.jgroups.JChannel.downcall(JChannel.java:1478)
at org.jgroups.JChannel.connect(JChannel.java:370)

Looking at the source, here are the problematic lines:

1573: ProtocolAdapter ad=(ProtocolAdapter)prot;
1574: List<Address> tmp=ad.getMembers();
1575: members.addAll(tmp);

Looking now at ProtocolAdapter.getMembers(), I see that it does this:
public List<Address> getMembers() {
return Collections.unmodifiableList(members);
}

While Collections.unmodifiableList(...) ensures that the caller of getMembers() cannot modify the list, it can still iterate over it (e.g. line 1575). The offending mutation code is probably in ProtocolAdapter.down(Event):

case Event.VIEW_CHANGE:
View view=(View)evt.getArg();
Vector<Address> tmp=view.getMembers();
members.clear();
members.addAll(tmp);
break;

There are several ways fix this, including, but not limited to:

• Make ProtocolAdapter.members volatile, and create a new member list on view change
• Make ProcotolAdapter.members a CopyOnWriteArrayList