Actually, even Core uses permutations, for exception stacktraces collection:
https://github.com/gwtproject/gwt/blob/master/user/src/com/google/gwt/core/CoreWithUserAgent.gwt.xml
Fwiw, if everything compiled down to the same code, there'd be a single *.cache.js output, so it wouldn't matter much whether you compile for various browsers or not (besides compilation time). Seeing more than one *.cache.js means there are differences, and if those differences are minimal you may want to <collapse-all-properties /> or at a minimum <collapse-property name="user.agent" values="*" />