Use Bowtie 2 instead of Bowtie to align reads. Since currently RSEM does not handle indel, local and discordant alignments, the Bowtie2 parameters are set in a way to avoid those alignments. In particular, we use options '--very-sensitive --dpad 0 --gbar 99999999 --mp 1,1 --np 1 --score-min L,0,-rate'. "-rate", the last parameter of '--score-min' is the negative value of the mismatch rate provided by option '--bowtie2-mismatch-rate'. If reads are paired-end, we additionally use options '--no-mixed' and '--no-discordant'.
However, I recently tried using using the bowtie2 option in RSEM 1.2.11 and noticed that the preset being called is "sensitive" and not "very-sensitive" as stated above. And looking at line 68 of the rsem-calculate-expression perl script:
my $bowtie2_sensitivity_level = "sensitive"; # must be one of "very_fast", "fast", "sensitive", "very_sensitive"
Just thought I would point that out. I'm trying to figure out why my (bowtie/bowtie2) overall alignment rate dropped from 81.12% with bowtie to 49.03% with bowtie2. I'm aligning 3 million 454 cDNA reads (single end) to a de novo transcriptome. I used the RSEM defaults for both bowtie and bowtie2, and specified -fragment-length mean and sd.
(Bowtie 2 parameter) Set Bowtie 2's preset options in --end-to-end mode. This option controls how hard Bowtie 2 tries to find alignments. <string> must be one of "very_fast", "fast", "sensitive" and "very_sensitive". The four candidates correspond to Bowtie 2's "--very-fast", "--fast", "--sensitive" and "--very-sensitive" options. (Default: "sensitive" - use Bowtie 2's default)