Hudson should comment on the pull request when:

Test starts
A set of tests pass (e.g. XTS crash rec
When the job passes/fails

The following job is a PoC that I created: http://172.17.131.2/job/paul-test/configure

Here you will see that we have a jbosstm-bot user that creates the comment.

I think the code will need to go in narayana.sh as only it knows the progress of the tests.