Logging started at 17:29:03 on 10 Feb 2017 QIIME version: 1.9.1 qiime_config values: blastmat_dir /opt/blast/2.2.22/data pick_otus_reference_seqs_fp /opt/qiime/1.9.1/lib/python2.7/site-packages/qiime_default_reference-0.1.3-py2.7.egg/qiime_default_reference/gg_13_8_otus/rep_set/97_otus.fasta sc_queue all.q topiaryexplorer_project_dir None pynast_template_alignment_fp /opt/qiime/1.9.1/lib/python2.7/site-packages/qiime_default_reference-0.1.3-py2.7.egg/qiime_default_reference/gg_13_8_otus/rep_set_aligned/85_otus.pynast.fasta cluster_jobs_fp start_parallel_jobs.py pynast_template_alignment_blastdb None assign_taxonomy_reference_seqs_fp /opt/qiime/1.9.1/lib/python2.7/site-packages/qiime_default_reference-0.1.3-py2.7.egg/qiime_default_reference/gg_13_8_otus/rep_set/97_otus.fasta torque_queue friendlyq jobs_to_start 1 slurm_time None denoiser_min_per_core 50 assign_taxonomy_id_to_taxonomy_fp /opt/qiime/1.9.1/lib/python2.7/site-packages/qiime_default_reference-0.1.3-py2.7.egg/qiime_default_reference/gg_13_8_otus/taxonomy/97_otu_taxonomy.txt temp_dir /tmp/ slurm_memory None slurm_queue None blastall_fp /opt/blast/2.2.22/bin/blastall seconds_to_sleep 1 parameter file values: parallel:jobs_to_start 40 Input file md5 sums: /gpfs0/home/resgoodman/vav002/170106_amplicons_qiime/seqs_chimeras_pos_filtered.fna: 29ddacbb42cf024cd014556e32bba537 /reference/greengenes/gg_13_8_otus/rep_set/97_otus.fasta: 50b2269712b3738afb41892bed936c29 Forcing --suppress_new_clusters as this is reference-based OTU picking. Executing commands. # Pick Reference OTUs command pick_otus.py -i /gpfs0/home/resgoodman/vav002/170106_amplicons_qiime/seqs_chimeras_pos_filtered.fna -o /gpfs0/home/resgoodman/vav002/170106_amplicons_qiime/pos_OTUs_gg/step1_otus -r /reference/greengenes/gg_13_8_otus/rep_set/97_otus.fasta -m usearch61_ref --suppress_new_clusters Stdout: Stderr: # Generate full failures fasta file command filter_fasta.py -f /gpfs0/home/resgoodman/vav002/170106_amplicons_qiime/seqs_chimeras_pos_filtered.fna -s /gpfs0/home/resgoodman/vav002/170106_amplicons_qiime/pos_OTUs_gg/step1_otus/seqs_chimeras_pos_filtered_failures.txt -o /gpfs0/home/resgoodman/vav002/170106_amplicons_qiime/pos_OTUs_gg/step1_otus/failures.fasta Stdout: Stderr: Executing commands. # Pick rep set command pick_rep_set.py -i /gpfs0/home/resgoodman/vav002/170106_amplicons_qiime/pos_OTUs_gg/step1_otus/seqs_chimeras_pos_filtered_otus.txt -o /gpfs0/home/resgoodman/vav002/170106_amplicons_qiime/pos_OTUs_gg/step1_otus/step1_rep_set.fna -f /gpfs0/home/resgoodman/vav002/170106_amplicons_qiime/seqs_chimeras_pos_filtered.fna Stdout: Stderr: Executing commands. # Pick de novo OTUs on step1 failures command pick_otus.py -i /gpfs0/home/resgoodman/vav002/170106_amplicons_qiime/pos_OTUs_gg/step1_otus/failures.fasta -o /gpfs0/home/resgoodman/vav002/170106_amplicons_qiime/pos_OTUs_gg/step4_otus/ -m usearch61 --denovo_otu_id_prefix New.CleanUp.ReferenceOTU Stdout: Stderr: # Merge OTU maps command cat /gpfs0/home/resgoodman/vav002/170106_amplicons_qiime/pos_OTUs_gg/step1_otus/seqs_chimeras_pos_filtered_otus.txt /gpfs0/home/resgoodman/vav002/170106_amplicons_qiime/pos_OTUs_gg/step4_otus//failures_otus.txt > /gpfs0/home/resgoodman/vav002/170106_amplicons_qiime/pos_OTUs_gg/final_otu_map.txt Stdout: Stderr: # Pick representative set for subsampled failures command pick_rep_set.py -i /gpfs0/home/resgoodman/vav002/170106_amplicons_qiime/pos_OTUs_gg/step4_otus//failures_otus.txt -o /gpfs0/home/resgoodman/vav002/170106_amplicons_qiime/pos_OTUs_gg/step4_otus//step4_rep_set.fna -f /gpfs0/home/resgoodman/vav002/170106_amplicons_qiime/pos_OTUs_gg/step1_otus/failures.fasta Stdout: Stderr: # Filter singletons from the otu map using API python -c "import qiime; qiime.filter.filter_otus_from_otu_map('/gpfs0/home/resgoodman/vav002/170106_amplicons_qiime/pos_OTUs_gg/final_otu_map.txt', '/gpfs0/home/resgoodman/vav002/170106_amplicons_qiime/pos_OTUs_gg/final_otu_map_mc2.txt', '2')" # Write non-singleton otus representative sequences from step1 to the final rep set file: /gpfs0/home/resgoodman/vav002/170106_amplicons_qiime/pos_OTUs_gg/rep_set.fna # Copy the full input refseqs file to the new refseq file cp /reference/greengenes/gg_13_8_otus/rep_set/97_otus.fasta /gpfs0/home/resgoodman/vav002/170106_amplicons_qiime/pos_OTUs_gg/new_refseqs.fna # Write non-singleton otus representative sequences from step 4 to the final representative set and the new reference set (/gpfs0/home/resgoodman/vav002/170106_amplicons_qiime/pos_OTUs_gg/rep_set.fna and /gpfs0/home/resgoodman/vav002/170106_amplicons_qiime/pos_OTUs_gg/new_refseqs.fna respectively) Executing commands. # Make the otu table command make_otu_table.py -i /gpfs0/home/resgoodman/vav002/170106_amplicons_qiime/pos_OTUs_gg/final_otu_map_mc2.txt -o /gpfs0/home/resgoodman/vav002/170106_amplicons_qiime/pos_OTUs_gg/otu_table_mc2.biom Stdout: Stderr: Executing commands. # Assign taxonomy command parallel_assign_taxonomy_uclust.py -i /gpfs0/home/resgoodman/vav002/170106_amplicons_qiime/pos_OTUs_gg/rep_set.fna -o /gpfs0/home/resgoodman/vav002/170106_amplicons_qiime/pos_OTUs_gg/uclust_assigned_taxonomy -T --jobs_to_start 40 Stdout: Stderr: Executing commands. # Add taxa to OTU table command biom add-metadata -i /gpfs0/home/resgoodman/vav002/170106_amplicons_qiime/pos_OTUs_gg/otu_table_mc2.biom --observation-metadata-fp /gpfs0/home/resgoodman/vav002/170106_amplicons_qiime/pos_OTUs_gg/uclust_assigned_taxonomy/rep_set_tax_assignments.txt -o /gpfs0/home/resgoodman/vav002/170106_amplicons_qiime/pos_OTUs_gg/otu_table_mc2_w_tax.biom --sc-separated taxonomy --observation-header OTUID,taxonomy Stdout: Stderr: Executing commands. # Align sequences command parallel_align_seqs_pynast.py -i /gpfs0/home/resgoodman/vav002/170106_amplicons_qiime/pos_OTUs_gg/rep_set.fna -o /gpfs0/home/resgoodman/vav002/170106_amplicons_qiime/pos_OTUs_gg/pynast_aligned_seqs -T --jobs_to_start 40 Stdout: Stderr: # Filter alignment command filter_alignment.py -o /gpfs0/home/resgoodman/vav002/170106_amplicons_qiime/pos_OTUs_gg/pynast_aligned_seqs -i /gpfs0/home/resgoodman/vav002/170106_amplicons_qiime/pos_OTUs_gg/pynast_aligned_seqs/rep_set_aligned.fasta Stdout: Stderr: # Build phylogenetic tree command make_phylogeny.py -i /gpfs0/home/resgoodman/vav002/170106_amplicons_qiime/pos_OTUs_gg/pynast_aligned_seqs/rep_set_aligned_pfiltered.fasta -o /gpfs0/home/resgoodman/vav002/170106_amplicons_qiime/pos_OTUs_gg/rep_set.tre Stdout: Stderr: Executing commands. Logging stopped at 17:31:11 on 10 Feb 2017