To answer the first question, the "res" facility in partest fails correctly.
$ ls test/files/res/t5167/
t5167_1.scala t5167_2.scala
apm@halyard /cygdrive/m/apm/projects/scala-snytt
$ cat test/files/res/t5167.res
t5167/t5167_1.scala
t5167/t5167_2.scala
apm@halyard /cygdrive/m/apm/projects/scala-snytt
$ cat test/files/res/t5167.check
nsc>
nsc>
nsc>
apm@halyard /cygdrive/m/apm/projects/scala-snytt
$ test/partest test/files/res/t5167.res
test/partest: line 37: cd: /cygdrive/m/apm/projects/scala-snytt/test
/cygdrive/m/apm/projects/scala-snytt/test/..: No such file or directory
Testing individual files
testing: [...]\files\res\t5167.res [FAILED]
3,5c3,4
< nsc> t5167/t5167_2.scala:8: error: not enough arguments for method buggyMethod: (argWithDefault$1: String)Unit
< sut.buggyMethod()
< ^
---
> nsc>
> nsc>
1 of 1 tests failed (elapsed time: 00:00:03)