So I started adding more logging to karma, and I think I have ruled out karma as being the problem. It looks like awesome-typescript-loader spends several minutes calling ensureInstance (hundreds of calls). The number of calls increases with the number of tests. When I have zero tests it makes 414 calls. 15 tests = 453 calls. 76 tests = 468 calls.
This is version 3.1.3. Has anyone seen this or know what is going on?