In addition to the maximum message count and window size limits which are already negotiated on channel open, we should also provide the option to negotiate message limits.
Both outbound and inbound message limits should be advertised. Each peer should send messages no larger than the minimum of that side's sent outbound and received inbound maximums. Bounds checking for message limits should occur on both the sending and receiving side. If the receiving side gets a message overflow, it should be treated equivalently to
exceeding the window size or message count (i.e. it's treated as hostile and the connection message should be closed asynchronously; edited: since previous protocol versions do not have this concept, they will have to rely on the remote side truncating the channel asynchronously, therefore the connection should remain open).
On the sending side, sending more data than the maximum allowed should result in an exception, and no data should be transferred from the offending operation. It is still the caller's responsibility to close the message in this case (the connection will not be closed).