Skip to first unread message

ciara keating

unread,
Feb 9, 2016, 2:54:48 PM2/9/16
to qiime...@googlegroups.com
Hi everyone,

Complete QIIME beginner here. I have been using Mothur and now want to use MacQIIME for my core diversity etc analysis. However, I am encountering error messages and the only output I get is the log file, rarefied number biom and biom table summary. I've the log below and have attached the mapping file (it appears that the other files are too large to attach). I'd appreciate any help or advice you could give. I've been researching myself to figure it out but I've been unsuccessful! I'm wondering is it maybe to do with my negative control being in the sample which would be one sample on its own different to all the others?

The command I ran was: (9986 being my lowest sample count/selected non phylogenetic diversity as I had issues making the tree [I had the same tree issue in Mothur as my distance file was 249gb])

core_diversity_analyses.py -i /Users/ciarakeating2/codindex.01.0.01.biom -m /Users/ciarakeating2/mapping_fileCK.txt -e 9986 -o /Users/ciarakeating2/core_diversity01/ -c feed,tank,week,health --nonphylogenetic_diversity --suppress_group_significance


The log file is as follows:

Logging started at 16:16:18 on 09 Feb 2016
QIIME version: 1.9.1

qiime_config values:
pick_otus_reference_seqs_fp /macqiime/anaconda/lib/python2.7/site-packages/qiime_default_reference/gg_13_8_otus/rep_set/97_otus.fasta
sc_queue all.q
pynast_template_alignment_fp /macqiime/anaconda/lib/python2.7/site-packages/qiime_default_reference/gg_13_8_otus/rep_set_aligned/85_otus.pynast.fasta
cluster_jobs_fp start_parallel_jobs.py
assign_taxonomy_reference_seqs_fp /macqiime/anaconda/lib/python2.7/site-packages/qiime_default_reference/gg_13_8_otus/rep_set/97_otus.fasta
torque_queue friendlyq
jobs_to_start 1
denoiser_min_per_core 50
assign_taxonomy_id_to_taxonomy_fp /macqiime/anaconda/lib/python2.7/site-packages/qiime_default_reference/gg_13_8_otus/taxonomy/97_otu_taxonomy.txt
temp_dir /tmp/
blastall_fp blastall
seconds_to_sleep 60

parameter file values:
beta_diversity:metrics bray_curtis
alpha_diversity:metrics observed_otus,chao1
parallel:jobs_to_start 1

Input file md5 sums:
/Users/ciarakeating2/codindex.01.0.01.biom: b03c55151235c24de4e2611394762ce9
/Users/ciarakeating2/mapping_fileCK.txt: b9330b9c0d2f2c31b249770ffd866aec

Executing commands.

# Generate BIOM table summary command 
biom summarize-table -i /Users/ciarakeating2/codindex.01.0.01.biom -o /Users/ciarakeating2/core_diversity01//biom_table_summary.txt 

Stdout:

Stderr:

# Filter low sequence count samples from table (minimum sequence count: 9986) command 
filter_samples_from_otu_table.py -i /Users/ciarakeating2/codindex.01.0.01.biom -o /Users/ciarakeating2/core_diversity01//table_mc9986.biom -n 9986



*** ERROR RAISED DURING STEP: Filter low sequence count samples from table (minimum sequence count: 9986)
Command run was:
 filter_samples_from_otu_table.py -i /Users/ciarakeating2/codindex.01.0.01.biom -o /Users/ciarakeating2/core_diversity01//table_mc9986.biom -n 9986
Command returned exit status: 1
Stdout:

Stderr
Traceback (most recent call last):
  File "/macqiime/anaconda/bin/filter_samples_from_otu_table.py", line 162, in <module>
    main()
  File "/macqiime/anaconda/bin/filter_samples_from_otu_table.py", line 138, in main
    write_biom_table(filtered_otu_table, output_fp)
  File "/macqiime/anaconda/lib/python2.7/site-packages/qiime/util.py", line 577, in write_biom_table
    biom_table.to_hdf5(biom_file, generated_by, compress)
  File "/macqiime/anaconda/lib/python2.7/site-packages/biom/table.py", line 3535, in to_hdf5
    self.group_metadata(axis='observation'), 'csr', compression)
  File "/macqiime/anaconda/lib/python2.7/site-packages/biom/table.py", line 3507, in axis_dump
    formatter[category](grp, category, md, compression)
  File "/macqiime/anaconda/lib/python2.7/site-packages/biom/table.py", line 243, in general_formatter
    compression=compression)
  File "/macqiime/anaconda/lib/python2.7/site-packages/h5py/_hl/group.py", line 99, in create_dataset
    dsid = dataset.make_new_dset(self, shape, dtype, data, **kwds)
  File "/macqiime/anaconda/lib/python2.7/site-packages/h5py/_hl/dataset.py", line 60, in make_new_dset
    raise ValueError("Shape tuple is incompatible with data")
ValueError: Shape tuple is incompatible with data

log_20160209161618.txt
mapping_fileCK.txt

Husen Zhang

unread,
Feb 9, 2016, 11:18:22 PM2/9/16
to qiime...@googlegroups.com
Ciara -
Did you generate your biom file in mothur?  If so, this thread may help you out.

Husen Zhang

ciara keating

unread,
Feb 10, 2016, 6:34:05 AM2/10/16
to Qiime 1 Forum
Hi Dr. Zhang,

That's perfect that conversion worked for the command. Could you advise me on whether I should do the core diversity analysis on my other biom files. Ie at 02, 03 and 04 levels from Mothur? Also if there is an issue using the Mothur generated fasta file to use the make.phylogeny command? I tried to do this command in Mothur (Dist.seqs and Clearcut) but it failed at clearcut as the distance file was 249 gb. I'm just wondering is there a way around this in QIIME? 

Thanks for your help,

Ciara

Colin Brislawn

unread,
Feb 10, 2016, 3:32:55 PM2/10/16
to Qiime 1 Forum
Hello Ciara,

I'm not sure how mothur attempts tree building, but I can describe how qiime does it. Qiime aligns OTU centroids to greengenes database using a nast alignment, filters elements of that alignment, then constructs a tree using fasttree. While only using OTU centroids, you will have a much smaller alignment and tree file, so this may scale better for your project. It sounds like your 'Mothur generated fasta file' contains all your reads, so you could try the qiime method on the fasta file only containing OTU centroids. 

Colin Brislawn

Reply all
Reply to author
Forward
0 new messages