single_rarefaction. py error

Wale Adebayo

unread,

Sep 18, 2016, 8:56:17 PM9/18/16

to Qiime 1 Forum

Hi, all pls help. A similar question(on alpha rarefaction) was asked here last year bit it is now closed and the answers therein did not solve this one. I have tried re-installing pandas and upgrading, also I tried converting to json or tsv, no solution. The otu table is from QIIME originally. Thanks

ubuntu@ubuntu:~$ single_rarefaction.py -i Downloads/iuon.biom -o Downloads/otuEven1000.biom -d 1000
Traceback (most recent call last):
File "/usr/local/bin/single_rarefaction.py", line 87, in <module>
main()
File "/usr/local/bin/single_rarefaction.py", line 84, in main
empty_otus_removed=(not opts.keep_empty_otus), subsample_f=subsample_f)
File "/usr/local/lib/python2.7/dist-packages/qiime/rarefaction.py", line 61, in rarefy_to_file
subsample_f=subsample_f)
File "/usr/local/lib/python2.7/dist-packages/qiime/rarefaction.py", line 187, in get_rare_data
subsampled_otu_table = otu_table.transform(func, axis='sample')
File "/usr/local/lib/python2.7/dist-packages/biom/table.py", line 2718, in transform
_transform(arr, ids, metadata, f, axis)
File "biom/_transform.pyx", line 51, in biom._transform._transform (biom/_transform.c:1550)
File "/usr/local/lib/python2.7/dist-packages/qiime/rarefaction.py", line 185, in func
return subsample_f(x.astype(int), seqs_per_sample)
File "/usr/local/lib/python2.7/dist-packages/skbio/stats/_subsample.py", line 212, in subsample
return subsample_counts(counts, n, replace=replace)
File "/usr/local/lib/python2.7/dist-packages/skbio/stats/_subsample.py", line 311, in subsample_counts
counts_sum)
File "__subsample.pyx", line 15, in skbio.stats.__subsample._subsample_counts_without_replacement (skbio/stats/__subsample.c:1418)
ValueError: Buffer dtype mismatch, expected 'int64_t' but got 'long'

stem information
==================
Platform: linux2
Python version: 2.7.3 (default, Jun 22 2015, 19:43:34) [GCC 4.6.3]
Python executable: /usr/bin/python

QIIME default reference information
===================================
For details on what files are used as QIIME's default references, see here:
https://github.com/biocore/qiime-default-reference/releases/tag/0.1.3

Dependency versions
===================
QIIME library version: 1.9.1
QIIME script version: 1.9.1
qiime-default-reference version: 0.1.3
NumPy version: 1.11.1
SciPy version: 0.18.0
pandas version: 0.18.1
matplotlib version: 1.5.3
biom-format version: 2.1.5
h5py version: 2.6.0 (HDF5 version: 1.8.17)
qcli version: 0.1.1
pyqi version: 0.3.2
scikit-bio version: 0.2.3
PyNAST version: 1.2.2
Emperor version: 0.9.51
burrito version: 0.9.1
burrito-fillings version: 0.1.1
sortmerna version: SortMeRNA version 2.0, 29/11/2014
sumaclust version: SUMACLUST Version 1.0.00
swarm version: Not installed.
gdata: Installed.

QIIME config values
===================
For definitions of these settings and to learn how to configure QIIME, see here:
http://qiime.org/install/qiime_config.html
http://qiime.org/tutorials/parallel_qiime.html

blastmat_dir: None
pick_otus_reference_seqs_fp: /usr/local/lib/python2.7/dist-packages/qiime_default_reference/gg_13_8_otus/rep_set/97_otus.fasta
sc_queue: all.q
topiaryexplorer_project_dir: None
pynast_template_alignment_fp: /usr/local/lib/python2.7/dist-packages/qiime_default_reference/gg_13_8_otus/rep_set_aligned/85_otus.pynast.fasta
cluster_jobs_fp: start_parallel_jobs.py
pynast_template_alignment_blastdb: None
assign_taxonomy_reference_seqs_fp: /usr/local/lib/python2.7/dist-packages/qiime_default_reference/gg_13_8_otus/rep_set/97_otus.fasta
torque_queue: friendlyq
jobs_to_start: 1
slurm_time: None
denoiser_min_per_core: 50
assign_taxonomy_id_to_taxonomy_fp: /usr/local/lib/python2.7/dist-packages/qiime_default_reference/gg_13_8_otus/taxonomy/97_otu_taxonomy.txt
temp_dir: /tmp/
slurm_memory: None
slurm_queue: None
blastall_fp: blastall
seconds_to_sleep: 1

QIIME base install test results
===============================
........F
======================================================================
FAIL: test_uclust_supported_version (__main__.QIIMEDependencyBase)
uclust is in path and version is supported
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/local/bin/print_qiime_config.py", line 336, in test_uclust_supported_version
% ('.'.join(map(str, acceptable_version)), version_string))
AssertionError: Unsupported uclust version. 1.2.22 is required, but running /usr/local/bin/uclust: 1: /usr/local/bin/uclust: Syntax error: word unexpected (expecting ")")
.
I have tried reinstalling uclust, though I doubt it is the problem.

Greg Caporaso

unread,

Sep 19, 2016, 11:42:38 AM9/19/16

to Qiime 1 Forum

Hi Wale, Would you be able to send me the OTU table? You can email that to me directly at gregca...@gmail.com if you don't want to post it to the forum. I'll follow up with you on the forum.

Wale Adebayo

unread,

Sep 19, 2016, 11:25:09 PM9/19/16

to Qiime 1 Forum

Many Thanks, I have sent the file.

Greg Caporaso

unread,

Sep 21, 2016, 3:02:06 PM9/21/16

to Qiime 1 Forum

Hi Wale, I'm not able to reproduce this - the single_rarefaction.py command completes for me with the file that you sent (I've emailed the result to you). I suspect that this has something to do with your dependencies being slightly different than mine. I have conda installed QIIME (according to the new instructions here) and my print_qiime_config.py output is below. Would it be possible for you to install QIIME with conda?

System information

==================

Platform: darwin

Python version: 2.7.12 |Continuum Analytics, Inc.| (default, Jul 2 2016, 17:43:17) [GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2336.11.00)]

Python executable: /Users/caporaso/miniconda3/envs/qiime191/bin/python

QIIME default reference information

===================================

For details on what files are used as QIIME's default references, see here:

https://github.com/biocore/qiime-default-reference/releases/tag/0.1.3

Dependency versions

===================

QIIME library version: 1.9.1

QIIME script version: 1.9.1

qiime-default-reference version: 0.1.3

NumPy version: 1.10.4

SciPy version: 0.17.1

pandas version: 0.18.1

matplotlib version: 1.4.3

biom-format version: 2.1.5

h5py version: 2.6.0 (HDF5 version: 1.8.16)

qcli version: 0.1.1

pyqi version: 0.3.2

scikit-bio version: 0.2.3

PyNAST version: 1.2.2

Emperor version: 0.9.51

burrito version: 0.9.1

burrito-fillings version: 0.1.1

sortmerna version: SortMeRNA version 2.0, 29/11/2014

sumaclust version: Not installed.

swarm version: Swarm 1.2.19 [Mar 5 2016 17:12:19]

gdata: Installed.

QIIME config values

===================

For definitions of these settings and to learn how to configure QIIME, see here:

http://qiime.org/install/qiime_config.html

http://qiime.org/tutorials/parallel_qiime.html

blastmat_dir: None

pick_otus_reference_seqs_fp: /Users/caporaso/miniconda3/envs/qiime191/lib/python2.7/site-packages/qiime_default_reference/gg_13_8_otus/rep_set/97_otus.fasta

sc_queue: all.q

topiaryexplorer_project_dir: None

pynast_template_alignment_fp: /Users/caporaso/miniconda3/envs/qiime191/lib/python2.7/site-packages/qiime_default_reference/gg_13_8_otus/rep_set_aligned/85_otus.pynast.fasta

cluster_jobs_fp: start_parallel_jobs.py

pynast_template_alignment_blastdb: None

assign_taxonomy_reference_seqs_fp: /Users/caporaso/miniconda3/envs/qiime191/lib/python2.7/site-packages/qiime_default_reference/gg_13_8_otus/rep_set/97_otus.fasta

torque_queue: friendlyq

jobs_to_start: 1

slurm_time: None

denoiser_min_per_core: 50

assign_taxonomy_id_to_taxonomy_fp: /Users/caporaso/miniconda3/envs/qiime191/lib/python2.7/site-packages/qiime_default_reference/gg_13_8_otus/taxonomy/97_otu_taxonomy.txt

temp_dir: /Users/caporaso/temp/

Wale Adebayo

unread,

Sep 21, 2016, 3:40:48 PM9/21/16

to Qiime 1 Forum

Many Thanks, Dr Caporaso. Before using pip and numpy, I did attempt using Miniconda according to the instructions in the link you sent. After conda installation, creating the qiime1 environment brings up error: package not found in linux-32 channels. I do use a 32 bit system. In all, it seems my rarefaction error is also probably due to pandas having issues with Linux 32 bit. At least from readings about the ValueError.

Thanks so much

Reply all

Reply to author

Forward