I just checked in a change to bitcoinj git master that makes it much easier to create a pull tester jar. Here are instructions for how to do it.

You will need:

A Java Development Kit (JDK), version 6 or up should work. As Java 6 was released eight years ago, this should not be a challenging requirement. If you have a Mac just running "java" from the command line should give you a GUI prompt to install it automatically. Otherwise apt-get or fetch the latest from the interwebs.
Apache Maven. This is a rough equivalent of autotools, except it does dependency resolution for you. Grab it from http://maven.apache.org/download.cgi then unzip it and make sure the bin directory is in your PATH. You may need to set the JAVA_HOME environment variable if you installed Java to an odd place.
git

Make sure you can run "javac" from the command line, then make sure you can run "mvn", it should complain it can't find a POM (this is a build config file) and not, say, that it can't find Java.

Now grab bitcoinj from git master:

git clone https://github.com/bitcoinj/bitcoinj.git

... and build ....

cd bitcoinj

mvn -DskipTests package

It will go off and download the libraries needed, compile, and create a bundled executable JAR called core/target/pull-tests.jar. This is sort of analogous to static linking in the Java world. It should be fast - expect a full build plus downloads to take less than a minute. You can use it either with the QA scripts in the bitcoin core qa/pull-tester directory or just run things directly:

./bitcoind -regtest -connect=0.0.0.0 -listen -whitelist=127.0.0.1 -datadir=/tmp/pulltester

java -jar core/target/pull-tests.jar

It should go ahead and print lots of debug spew, then at the end say it's happy.

Let me know if you encounter any problems with this.

Java JARs (which are just zip files renamed) are easily reproduced if you use the same version of javac and the same bitcoinj version. The ZIP container has timestamps, but unzipping them and simply diffing the files between two builds should reveal no differences. I am happy to provide a pull-tests.jar from my local machine if anyone would like to do this.