Looks like the check I implemented is also inadvertently the place where the tests will fail if there is any syntax error. So I'm thinking python -m compileall isn't the greatest way to test for this.
I'm almost to the point of just removing the invalid escape checks. They will be syntax error probably in Python 3.7 or 3.8, so maybe we should just wait until those are released and we support them, and it will be much easier to test for them.
Unless someone can think of a better way to test them. The problem is that they are only checked when the Python bytecode is compiled, and only when Python is run with the -We:invalid flag. So if the bytecode has already been compiled and -We:invalid wasn't included, it won't error. Furthermore, when setup.py install is run, Python byte compiles all the files and just happily ignores syntax errors, meaning we can't rely on this particular bytecode compilation to do the check for us.
I already implemented two incorrect checks that weren't really erroring (almost three, but I was more careful by then). Now we have a check that works, but the messaging is awful.
Aaron Meurer