Hey Ben,
It happens that I don't think it is currently possible to do that programmatically [at least, I don't know any way of doing this].
However, in our day-to-day work, we use intellij and maven.
With maven you can run them all with "mvn test" and in intellij you can create your own run configuration per file/per class/per package and many other filters.
You could open an issue on GitHub about this, can you however explain how "this doesn't work"? Is there a specific error message? Does it just *not* run the tests or crashes somehow?
Thanks for reporting this by the way.
Have a good day :)