Blast failure with 18S data using Silva119

[S Wollney]

Greetings,

I have been trying to run 18S eukaryote data using the Silva 119 release files using pick_open_reference.py with the command:
split_libraries.py -f /scratch/seth.wollney/fall15_euk/102815TMeuk7F_full.fasta -q /scratch/seth.wollney/fall15_euk/102815TMeuk7F_full.qual -m /scratch/seth.wollney/fall15_euk/102815TMeuk7F_mapping2.txt -o /scratch/seth.wollney/fall15_euk/split_lib -b 8

The script picks otus just fine but then returns no results when it reaches the assign_taxonomy step.  Any ideas what's happening here and how I can correct it?

Am I using the appropriate Silva files for this process?   I've also attached the parameter file.

Thanks,
Seth

Here's the log file from the blast output directory:

BlastTaxonAssigner parameters:
Application:blastn/megablast
Max E value:0.001
Min percent identity:90.0
blast_db:/scratch/seth.wollney/silva/core_Silva119_alignment.fna
id_to_taxonomy_filepath:/scratch/seth.wollney/silva/taxonomy_97_all_levels_18S.txt
Number of sequences inspected: 8093
Number with no blast hits: 8093
Result path: "/scratch/seth.wollney/tmp/assign-taxP91LvEpq0MRuNewPN1nT.txt"

[S Wollney]

FYI:  I am using QIIME 1.8

[TonyWalters]

Hello,

This looks like the issue:

blast_db:/scratch/seth.wollney/silva/core_Silva119_alignment.fna

I would change the path to point to this file (I'm not sure of the exact path for your system): Silva_119/Silva119_release/rep_set_eukaryotes/97/Silva_119_rep_set97_18S.fna

[S Wollney]

Thank you Tony! 

[S Wollney]

Hi Tony,

It seems that's not the issue, or the only issue.  I reran the script with the Silva_119_rep_set97_18S.fna and still no blast hits were made.  It seems that the script to choking on make_phylogeny step as well....


*** ERROR RAISED DURING STEP: Build phylogenetic tree
Command run was:
 make_phylogeny.py -i /scratch/seth.wollney/fall15_euk/otus/pynast_aligned_seqs/rep_set_aligned_pfiltered.fasta -o /scratch/seth.wollney/fall15_euk/otus/rep_set.tre
Command returned exit status: 1
Stdout:

#error from the output directory for blast:

BlastTaxonAssigner parameters:
Application:blastn/megablast
Max E value:0.001
Min percent identity:90.0

blast_db:/scratch/seth.wollney/silva/Silva_119_rep_set97_18S.fna
id_to_taxonomy_filepath:/scratch/seth.wollney/silva/taxonomy_97_all_levels_18S.txt
Number of sequences inspected: 7460
Number with no blast hits: 7460
Result path: "/scratch/seth.wollney/tmp/assign-taxFAfKqT431GbdKX1zKxeI.txt"

[TonyWalters]

Hello,

Do these reads cluster against the SILVA database in the OTU picking step? Or did they all get clustered de novo? Could the reads be 16S (the reference database and taxonomy mapping files you are using are eukaryote-only, maybe use the other rep_set/taxonomy files (includes all 16S/18S data) and see how that turns out. Can you manually blast a few of the reads on NCBI and check those results, and see if the reads hit SSU sequences and don't have large segments that don't match-this can indicate that sequencing/PCR artifacts are present that might interfere with taxonomic assignment.

For the tree building, did you follow the notes.txt file that came with the SILVA release that describe the differences from the default settings with regards to aligning and filtering said alignment before tree building?

[S Wollney]

Hi Tony,

Yes, I use the Silva files for OTU picking with the pick_open_reference_otus.py script.

I just blasted a few reads against NCBI and the e-value for each was many orders of magnitude lower than threshold I have set in the parameter file.  Could this be the reason no hits are being returned?

I'll read over the notes.txt files for tree building. 

Thanks,
Seth

[TonyWalters]

The e-value is a distinct possibility. Did you have identities far away from 100%?

[S Wollney]

No, the 10 seqs I tried were all above 90% but the e-values ranged from 4e-24 to 1e-10

[S Wollney]

HI Tony,

I have tried to run another data set with the exact same set-up as in my original post and still resulted in no blast results being returned.

So, I think this is pointing to an issue with how I have the scripts set up.  I also tried using the complete data sets, not just the euk only files.  The max e-value is set as the default 0.001, so I assume that value such as 4e-124 would still be captured?

The sequencing facility was able to generate otu tables, etc with this data, so I'm ruling out any major issues fastas I'm using.... thoughts?

Thanks,
Seth

Here is the command I'm running for my lab mate's data:
pick_open_reference_otus.py -i /scratch/jennifer.servis/edna/concat_seqs.fna -o /scratch/jennifer.servis/edna/otus -p /scratch/jennifer.servis/edna/para.txt -r /scratch/jennifer.servis/edna/silva/Silva119_release/rep_set/97/Silva_119_rep_set97.fna --prefilter_percent_id 0.0 --suppress_align_and_tree

Here is the parameter file:

filter_alignment:suppress_lane_mask_filter True
filter_alignment:entropy_threshold 0.10
align_seqs:template_fp /scratch/jennifer.servis/edna/silva/Silva119_release/core_alignment/core_Silva119_alignment.fna

pick_otus:enable_rev_strand_match True
assign_taxonomy:assignment_method blast
assign_taxonomy:reference_seqs_fp /scratch/jennifer.servis/edna/silva/Silva119_release/rep_set/97/Silva_119_rep_set97.fna
assign_taxonomy:id_to_taxonomy_fp /scratch/jennifer.servis/edna/silva/Silva119_release/taxonomy/97/taxonomy_97_7_levels.txt
assign_taxonomy:blast_db /scratch/jennifer.servis/edna/silva/Silva119_release/rep_set/97/Silva_119_rep_set97.fna
align_seqs:min_percent_id 60.0

[TonyWalters]

Hi, 

Can you remove this line from your parameters file:

assign_taxonomy:blast_db /scratch/jennifer.servis/edna/silva/Silva119_release/rep_set/97/Silva_119_rep_set97.fna

That should make it build the database from the rep set file that is provided.

[S Wollney]

IT WORKED!!!!   Thank you SOOOO much Tony!  You ended more than a week's worth of stressing out! 

Have a great weekend!!!!