Hi JC, John,
the default --outFilterMismatchNmax 10 mismatches in STAR is quite arbitrary, and needs to be adjusted according to your particular situation.
A good way to set is with --outFilterMismatchNoverLmax, which is scaled to the read length (you would also need to specify a large number for --outFilterMismatchNmax since the smaller of the two number will be used).
Both these parameters relate to the total number of mismatches in the paired alignment, which I think makes more sense that setting this threshold for each mate separately.
Of course the mismatches include sequencing errors, SNPs and RNA-editing, that's why I think we need to allow quite a few.
In the current ENCODE production with are using --outFilterMismatchNoverLmax 0.04, which means, for instance, 8 MM for 2x100 pair.
On the other hand, the this parameter is not as important for STAR than it is for TopHat/Bowtie1. Unless you enforce end-to-end alignment, STAR will trim reads whenever the number of mismatches exceeds --outFilterMismatchNmax, and may still be able to map the read. Note that mismatches are not counted in the trimmed ("soft-clipped") portion of the reads.
Cheers
Alex