Help with Qiime install for a Linux newbie, step by step

GeorgeWatts

unread,

Sep 11, 2012, 4:16:02 PM9/11/12

to qiime...@googlegroups.com

I am trying to install Qiime and on Linux 12.04. My home directory (the one I get to when I type "cd" and then "pwd" is:

gwatts@futscherlab32gb:~$ pwd

/mnt/datas/home/gwatts

I am following the instructions here: http://www.qiime.org/install/install.html

Step 1) First, change to the directory where you would like to download QIIME:

cd /home/qiime_user

In my case, this command is unnecessary, I am already in the equivalent: /mnt/datas/home/gwatts

Step 2) Get the tar from the link and ftp it into /mnt/datas/home/gwatts:

gwatts@futscherlab32gb:~$ ls

BugSeq16s Qiime-1.5.0.tar.gz QIIME_files

Note that I have two directories of data I am hoping to analyze in my home dierectory: BugSeq16s and QIIME_files, these are irrelevant to the install.

Step 3) Unpack the tar file using: tar -xvzf Qiime-1.5.0.tar.gz This produces a lot of output that scrolls by and creates a new directory Qiime-1.5.0:

gwatts@futscherlab32gb:~$ ls

BugSeq16s Qiime-1.5.0 Qiime-1.5.0.tar.gz QIIME_files

Step 4) Create a symbolic link between the Qiime-1.5.0 directory and a new directory called Qiime using: ln -s /home/qiime/Qiime-1.5.0 /home/qiime/Qiime

in my case, this command becomes:

gwatts@futscherlab32gb:~$ ln -s /mnt/datas/home/gwatts/Qiime-1.5.0 /mnt/datas/home/gwatts/Qiime

gwatts@futscherlab32gb:~$ ls

BugSeq16s Qiime Qiime-1.5.0 Qiime-1.5.0.tar.gz QIIME_files

Ok, good, I have a new directory Qiime and it's colored differently in my terminal - presumably to indicate it's a symbolically linked directory. Checking the contents of Qiime reveals they are the same as Qiime-1.5.0 as you'd expect:

gwatts@futscherlab32gb:~$ cd Qiime

gwatts@futscherlab32gb:~/Qiime$ ls

ChangeLog doc INSTALL qiime qiime_parameters.txt qiime_tutorial README requirements.txt scripts setup.py tests web

gwatts@futscherlab32gb:~/Qiime$ cd ..

gwatts@futscherlab32gb:~$ cd Qiime-1.5.0/

gwatts@futscherlab32gb:~/Qiime-1.5.0$ ls

ChangeLog doc INSTALL qiime qiime_parameters.txt qiime_tutorial README requirements.txt scripts setup.py tests web

Step 5) Install using setup.py. First change to the top level Qiime directory: cd /home/qiime/Qiime. This is the directory I just created with the ln command and in my case it is here: /mnt/datas/home/gwatts/Qiime

so:

gwatts@futscherlab32gb:~$ pwd

/mnt/datas/home/gwatts

gwatts@futscherlab32gb:~$ cd Qiime

Ok, so I am in the symbolically linked directory called Qiime, now execute the install with: "python setup.py install --install-scripts=/home/qiime/bin/ --install-purelib=/home/qiime/lib/ --install-data=/home/qiime/lib/". In my case, this command becomes:

gwatts@futscherlab32gb:~/Qiime$ python setup.py install --install-scripts=/mnt/datas/home/gwatts/Qiime/bin/ --install-purelib= /mnt/datas/home/gwatts/Qiime/lib/ --install-data= /mnt/datas/home/gwatts/Qiime/lib/

PyCogent not installed but required. (Is it installed? Is it in the current user's $PYTHONPATH or site-packages?) See http://pycogent.sourceforge.net.

Ok, so I am missing PyCogent...or did I get the paths wrong? Searching the forums yielded suggestions to add the following to my .bashrc file:

export PATH=FULL_PATH_TO_THE_SCRIPTS_FOLDER_IN_THE_QIIME_FOLDER:$PATH

export PYTHONPATH=FULL_PATH_TO_THE_QIIME_FOLDER:$PYTHONPATH

In my case this becomes:

export PATH=/mnt/datas/home/gwatts/Qiime/scripts:$PATH

export PYTHONPATH=/mnt/datas/home/gwatts/Qiime/:$PYTHONPATH

After adding these lines to .bashrc using vi, I source it by:

gwatts@futscherlab32gb:~$ source .bashrc

Now, I move back to the Qiime directory and retry the command:

gwatts@futscherlab32gb:~$ cd Qiime

gwatts@futscherlab32gb:~/Qiime$ python setup.py install --install-scripts=/mnt/datas/home/gwatts/Qiime/bin/ --install-purelib= /mnt/datas/home/gwatts/Qiime/lib/ --install-data= /mnt/datas/home/gwatts/Qiime/lib/

PyCogent not installed but required. (Is it installed? Is it in the current user's $PYTHONPATH or site-packages?) See http://pycogent.sourceforge.net.

gwatts@futscherlab32gb:~/Qiime$

What have I done wrong?

Will Van Treuren

unread,

Sep 11, 2012, 4:42:33 PM9/11/12

to qiime...@googlegroups.com

Hi George,

I am working on this one and will get back to you when I have a solution.

Will

--

GeorgeWatts

unread,

Sep 11, 2012, 4:51:04 PM9/11/12

to qiime...@googlegroups.com

Thanks for the prompt reply. I tried to be complete and clear in my post as I am new to Linux, I also want to leave a trail for those that may come after me. BTW, I have tried the Ubuntu 12.04-specific instructions here: http://www.qiime.org/install/ubuntu_install.html but this led to a non-functioning install, so I am trying it the long way this time...

Will Van Treuren

unread,

Sep 11, 2012, 5:38:15 PM9/11/12

to qiime...@googlegroups.com

Hi again,

I think we can get this resolved pretty rapidly. There are a couple things you need to do. Below are my instructions with bolded things being commands you have to issue and # indicating comments to explain what those things are doing.

Two starting notes -- you will need the package managers 'git' and 'svn' on your system. They should be already installed. To check type:

which svn #identifies the path (if one exists) to the executable code that runs the version control system svn

which git #identifies the path (if one exists) to the executable code that runs the version control system git

If either of these commands does not return a path -- should be something like /usr/bin/svn -- then you need to issue the following command.

sudo apt-get install svn #substitute git if thats the one that is missing, and follow any prompts it gives you.

Additionally, just to make sure, issue the following commands.

sudo apt-get install build-essential #this has some packages that I think should be installed by default, but cant hurt to make sure.

sudo apt-get install python-dev #this has some packages that I think should be installed by default, but cant hurt to make sure. it has solved some crazy errors in the past.

sudo apt-get install python-matplotlib #this installs matplotlib which contains libraries necessary for you to view graphics and for QIIME to work.

Now we are ready to start the QIIME installation.

First, lets erase what you have now.

rm -rf /mnt/datas/home/gwatts/Qiime-1.5.0 #erases the qiime directory you have download

rm /mnt/datas/home/gwatts/Qiime #erases the symbolic link you have created

Then, lets make a new directory where we can put all this source code.

mkdir ~/src #this makes a new directory in home called 'src'. we will put all the source code we need here

cd ~/src/ #move our current location to this new directory

Then, lets pull down the development versions of the tools we will need.

svn co https://qiime.svn.sourceforge.net/svnroot/qiime/trunk Qiime #this 'checks out' or grabs the latest version of qiime from sourceforge. Note the space between the address and the Qiime. This means that svn will put all the files into a new directory in source called 'Qiime'

svn co https://pycogent.svn.sourceforge.net/svnroot/pycogent/trunk pycogent #this checks out the latest version of PyCogent which is a backend that qiime depends on.

git clone git://github.com/biom-format/biom-format.git #we need to use a different version control system 'git' for the biom format, which is the final component we need.

To check that everything has gone smoothly up to this point, here is what it looks on my system following the above instructions:

will ~/test_for_george $ls

Qiime biom-format pycogent

Now that you have all the source code you will need for basic functionality, we need to tell your computer where to find the source code.

cd ~ #go to your home directory

vim .bashrc #opens your bash profile in vim

If you have any lines that currently define PATH or PYTHONPATH (the ones you added at the end in your original attempt) you need to delete them. Then you need to add the following lines.

PATH=/mnt/datas/home/gwatts/src/Qiime/scripts/:/mnt/datas/home/gwatts/src/biom-format/scripts/:${PATH}

PYTHONPATH=/mnt/datas/home/gwatts/src/Qiime/:/mnt/datas/home/gwatts/src/biom-format/python-code/:/mnt/datas/home/gwatts/src/pycogent/:${PYTHONPATH}

export PATH

export PYTHONPATH

Save your .bashrc profile and open a new terminal. Issue the following command to check if the installation worked.

print_qiime_config.py

If it works you should get some output like the following (it will be different because you are on Ubuntu and I am on a Mac).

System information

==================

Platform: darwin

Python version: 2.7.1 (r271:86882M, Nov 30 2010, 10:35:34) [GCC 4.2.1 (Apple Inc. build 5664)]

Python executable: /Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python

Dependency versions

===================

PyCogent version: 1.5.2-dev

NumPy version: 1.5.1

matplotlib version: 1.1.0

biom-format version: 1.0.0-dev

QIIME library version: 1.5.0-dev, svn revision 3170

Hopefully it works and you have an installation that is ready to go. If it does not then contact me again and we can do some troubleshooting.

Will

--

GeorgeWatts

unread,

Sep 11, 2012, 7:23:27 PM9/11/12

to qiime...@googlegroups.com

Will, Thanks for the reply, I tried your instructions and nearly got there:

Step 1) gwatts@futscherlab32gb:~/Qiime$ which svn

/usr/bin/svn

gwatts@futscherlab32gb:~/Qiime$ which git

gwatts@futscherlab32gb:~/Qiime$

No git, so:

sudo apt-get install git

then:

sudo apt-get install build-essential

sudo apt-get install python-dev

sudo apt-get install python-matplotlib

All went well without errors.

Step 2) In my home:

rm -rf /mnt/datas/home/gwatts/Qiime-1.5.0 Qiime #removed both directories with one command (getting fancy with Linux...)

mkdir src #to make the src directory.

svn co https://qiime.svn.sourceforge.net/svnroot/qiime/trunk Qiime

svn co https://pycogent.svn.sourceforge.net/svnroot/pycogent/trunk pycogent

git clone git://github.com/biom-format/biom-format.git

All went well, and ls now yields:

gwatts@futscherlab32gb:~/src$ ls

biom-format pycogent Qiime

so I have matched your example.

Step 3) Now edit .bashrc to show python where things are. I commented out the prior entries and added the ones you indicated, below is the last few lines of my .bashrc:

#source /mnt/datas/home/gwatts/qiime/activate.sh

#export PYTHONPATH=/mnt/datas/home/gwatts/qiime/qiime-1.5.0-release/:$PYTHONPATH

#export PYTHONPATH=/mnt/datas/home/gwatts/Qiime/:$PYTHONPATH

#export PATH=/mnt/datas/home/gwatts/Qiime/scripts/:$PATH

PATH=/mnt/datas/home/gwatts/src/Qiime/scripts/:/mnt/datas/home/gwatts/src/biom-format/scripts/:${PATH}

PYTHONPATH=/mnt/datas/home/gwatts/src/Qiime/:/mnt/datas/home/gwatts/src/biom-format/python-code/:/mnt/datas/home/gwatts/src/pycogent/:${PYTHONPATH}

export PATH

export PYTHONPATH

Closed my terminal, re-connected, and typed:

gwatts@futscherlab32gb:~$ ls

BugSeq16s QIIME_files src

gwatts@futscherlab32gb:~$ print_qiime_config.py

print_qiime_config.py: command not found

Ok, maybe you meant for me to run this from the src directory:

gwatts@futscherlab32gb:~$ cd src

gwatts@futscherlab32gb:~/src$ print_qiime_config.py

print_qiime_config.py: command not found

Ok, maybe from inside the Qiime directory:

gwatts@futscherlab32gb:~/src$ ls

biom-format pycogent Qiime

gwatts@futscherlab32gb:~/src$ cd Qiime/

gwatts@futscherlab32gb:~/src/Qiime$ print_qiime_config.py

print_qiime_config.py: command not found

Ok, maybe from inside the scripts directory:

gwatts@futscherlab32gb:~/src/Qiime$ ls

ChangeLog examples qiime README scripts tests

doc INSTALL qiime_tutorial requirements.txt setup.py web

gwatts@futscherlab32gb:~/src/Qiime$ cd scripts/

gwatts@futscherlab32gb:~/src/Qiime/scripts$ print_qiime_config.py

print_qiime_config.py: command not found

Is the print_qiime_config.py script there in the scripts directory?

gwatts@futscherlab32gb:~/src/Qiime/scripts$ ls

...lots of files.....

print_qiime_config.py

....more files....the script IS here so why is "command not found"?

Maybe calling python will help:

gwatts@futscherlab32gb:~/src/Qiime/scripts$ python print_qiime_config.py

Traceback (most recent call last):

File "print_qiime_config.py", line 15, in <module>

from qiime.util import make_option

ImportError: No module named qiime.util

gwatts@futscherlab32gb:~/src/Qiime/scripts$

I'm stuck. What next?

Will Van Treuren

unread,

Sep 11, 2012, 7:30:26 PM9/11/12

to qiime...@googlegroups.com

Hey George,

It shouldn't matter where print_qiime_config.py is run from. I may have typed your path wrong.

can you type the following and give me the output:

echo $PATH

echo $PYTHONPATH

Thanks,

Will

--

GeorgeWatts

unread,

Sep 11, 2012, 7:32:17 PM9/11/12

to qiime...@googlegroups.com

Looks like there is something wrong with the PYTHONPATH, as the echo returns nothing....

gwatts@futscherlab32gb:~/src$ echo $PATH

/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games

gwatts@futscherlab32gb:~/src$ echo $PYTHONPATH

gwatts@futscherlab32gb:~/src$

Will Van Treuren

unread,

Sep 11, 2012, 7:33:43 PM9/11/12

to qiime...@googlegroups.com

What kind of access do you have to this system? It appears that whatever is in your bash profile is not getting added to the path or the python path.

can you attach your .bashrc file?

--

GeorgeWatts

unread,

Sep 11, 2012, 7:56:19 PM9/11/12

to qiime...@googlegroups.com

That makes sense, I checked the paths that I put into my .bashrc file and they all exist and are correct. Earlier today I was googling "source .bashrc" and I found a forum in which a user was having to source their .bashrc every time they logged in and the response was that the changes should go into .bashrc_profile. The forum is here: http://apple.stackexchange.com/questions/42537/why-must-i-source-bashrc-every-time-i-open-terminal-for-aliases-to-work

So I tried sourcing my .bashrc and then ran the print_qiime_config.py:

gwatts@futscherlab32gb:~$ source .bashrc

gwatts@futscherlab32gb:~$ ls

bashrc BugSeq16s QIIME_files src

gwatts@futscherlab32gb:~$ print_qiime_config.py

System information

==================

Platform: linux2

Python version: 2.7.3 (default, Aug 1 2012, 05:14:39) [GCC 4.6.3]

Python executable: /usr/bin/python

Dependency versions

===================

PyCogent version: 1.5.2-dev

NumPy version: 1.6.1

matplotlib version: 1.1.1rc

biom-format version: 1.0.0-dev

QIIME library version: 1.5.0-dev, svn revision 3265

QIIME script version: 1.5.0-dev

PyNAST version (if installed): Not installed.

RDP Classifier version (if installed): Not installed.

QIIME config values

===================

blastmat_dir: None

sc_queue: all.q

topiaryexplorer_project_dir: None

pynast_template_alignment_fp: None

cluster_jobs_fp: None

pynast_template_alignment_blastdb: None

assign_taxonomy_reference_seqs_fp: None

torque_queue: friendlyq

qiime_test_data_dir: None

template_alignment_lanemask_fp: None

jobs_to_start: 1

cloud_environment: False

qiime_scripts_dir: None

denoiser_min_per_core: 50

working_dir: None

python_exe_fp: python

temp_dir: /tmp/

blastall_fp: blastall

seconds_to_sleep: 2

assign_taxonomy_id_to_taxonomy_fp: None

gwatts@futscherlab32gb:~$

Looks like my .bashrc is not getting executed when I log in...how does my print_qiime_config.py output look?

Also, the bashrc (minus the ".") file shown in my home directory is the copy that I've attached as a .txt file (google forums rejects it as an octet stream....)

bashrc.txt

Will Van Treuren

unread,

Sep 11, 2012, 8:09:49 PM9/11/12

to qiime...@googlegroups.com

That output from print_qiime_config.py looks good. I think your solution is spot on, we need to add the sourcing statement to the profile that gets executed for logon rather than interactive shells. so I am thinking we need to add the statement:

source ~/.bashrc

to the top of the ~/.bash_profile

Try adding that, logging in and out, and see if print_qiime_config.py works without sourcing.

--

Will Van Treuren

unread,

Sep 11, 2012, 8:10:29 PM9/11/12

to qiime...@googlegroups.com

here is the reference as well:

http://ubuntuforums.org/showthread.php?t=1868728

Will

GeorgeWatts

unread,

Sep 11, 2012, 8:18:55 PM9/11/12

to qiime...@googlegroups.com

Success!

I added:

"source ~/.bashrc" to my .profile, exited my terminal, logged back in, and print_qiime_config.py now works without having to source the .bashrc manually:

gwatts@futscherlab32gb:~$ ls

Now to actually get on with the data analysis....

Many thanks for the help, I hope this thread helps a future newbie.

Will Van Treuren

unread,

Sep 11, 2012, 8:21:43 PM9/11/12

to qiime...@googlegroups.com

Great. I agree that this thread will be very helpful to future 12.04 installers.

Hope the analysis goes well,

Will

--

GeorgeWatts

unread,

Sep 12, 2012, 9:16:55 PM9/12/12

to qiime...@googlegroups.com

Will, I'm back. Spent all day trying to get pick_otus_through_otu_table.py to work on my data file. Since I am faking the output of split_libraries.py with PGM data, I figured I had formatted something wrong in the file. I finally decided to try with the test data that had worked for me before.

1) get test files from: ftp://thebeast.colorado.edu/pub/QIIME-v1.5.0-dependencies/qiime_tutorial-v1.5.0.zip, put them into my QIIME_files directory and follow the instructions here: http://www.qiime.org/tutorials/tutorial.html. I have put my comments in the following with #.

#where am I?

gwatts@futscherlab32gb:~$ pwd

/mnt/datas/home/gwatts

#what's here?

gwatts@futscherlab32gb:~$ ls

BugSeq16s QIIME_files src

#let's get into the QIIME_files directory where I am keeping all my data...

gwatts@futscherlab32gb:~$ cd QIIME_files/

#what's here?

gwatts@futscherlab32gb:~/QIIME_files$ ls

BC.works JB.Mapping.txt not.working.fna QIIMEfastaFormatter.pl qiime_tutorial-v1.5.0 TEST working.fna

#get into the qiime_tutorial-v1.5.0 directory

gwatts@futscherlab32gb:~/QIIME_files$ cd qiime_tutorial-v1.5.0/

#what's here?

gwatts@futscherlab32gb:~/QIIME_files/qiime_tutorial-v1.5.0$ ls

18S_tutorial_files Fasting_Example.fna Fasting_Example.qual Fasting_Example.sff Fasting_Example.sff.txt Fasting_Map.txt qiime_tutorial_commands_parallel.sh qiime_tutorial_commands_serial.sh README

#ok, all the test files are here so let's get on with the tutorial:

gwatts@futscherlab32gb:~/QIIME_files/qiime_tutorial-v1.5.0$ check_id_map.py -m Fasting_Map.txt -o mapping_output

No errors or warnings were found in mapping file.

#great! the mapping file is good, so let's produce the seqs.fna file we need for all following steps using split_libraries.py:

gwatts@futscherlab32gb:~/QIIME_files/qiime_tutorial-v1.5.0$ split_libraries.py -m Fasting_Map.txt -f Fasting_Example.fna -q Fasting_Example.qual -o split_library_output

Compiled module "_likelihood_tree" not found. Will use pure Python/NumPy likelihoodihood tree.

Compiled module "_pairwise_pogs" not found. Will use slow Python alignment implementation.

Compiled module "_pairwise_seqs" not found. Will use slow Python alignment implementation.

Compiled module "_compare" not found. Will use slow Python dotplot.

#looks ok...let's check what was added by the script:

gwatts@futscherlab32gb:~/QIIME_files/qiime_tutorial-v1.5.0$ ls

18S_tutorial_files Fasting_Example.qual Fasting_Example.sff.txt mapping_output qiime_tutorial_commands_serial.sh split_library_output

Fasting_Example.fna Fasting_Example.sff Fasting_Map.txt qiime_tutorial_commands_parallel.sh README

gwatts@futscherlab32gb:~/QIIME_files/qiime_tutorial-v1.5.0$ cd split_library_output/

gwatts@futscherlab32gb:~/QIIME_files/qiime_tutorial-v1.5.0/split_library_output$ ls

histograms.txt seqs.fna split_library_log.txt

#let's check the log file:

gwatts@futscherlab32gb:~/QIIME_files/qiime_tutorial-v1.5.0/split_library_output$ more split_library_log.txt

Number raw input seqs 1339

Length outside bounds of 200 and 1000 0

Num ambiguous bases exceeds limit of 6 0

Missing Qual Score 0

Mean qual score below minimum of 25 1

Max homopolymer run exceeds limit of 6 0

Num mismatches in primer exceeds limit of 0: 1

Sequence length details for all sequences passing quality filters:

Raw len min/max/avg 233.0/293.0/265.8

Wrote len min/max/avg 200.0/260.0/232.8

Barcodes corrected/not 0/0

Uncorrected barcodes will not be written to the output fasta file.

Corrected barcodes will be written with the appropriate barcode category.

Corrected but unassigned sequences will not be written unless --retain_unassigned_reads is enabled.

Total valid barcodes that are not in mapping file 0

Sequences associated with valid barcodes that are not in the mapping file will not be written.

Barcodes in mapping file

Num Samples 9

Sample ct min/max/mean: 146 / 150 / 148.56

Sample Sequence Count Barcode

PC.634 150 ACAGAGTCGGCT

PC.356 150 ACAGACCACTCA

PC.593 149 AGCAGCACTTGT

PC.354 149 AGCACGAGCCTA

PC.635 149 ACCGCAGAGTCA

PC.607 149 AACTGTGCGTAC

PC.636 148 ACGGTGAGTGTC

PC.355 147 AACTCGTCGATG

PC.481 146 ACCAGCGACTAG

Total number seqs written 1337

#Looks good, so let's move up a directory and run the next step of the tutorial, pick_otus_through_table.py:

gwatts@futscherlab32gb:~/QIIME_files/qiime_tutorial-v1.5.0/split_library_output$ cd ..

gwatts@futscherlab32gb:~/QIIME_files/qiime_tutorial-v1.5.0$ ls

18S_tutorial_files Fasting_Example.qual Fasting_Example.sff.txt mapping_output qiime_tutorial_commands_serial.sh split_library_output

Fasting_Example.fna Fasting_Example.sff Fasting_Map.txt qiime_tutorial_commands_parallel.sh README

gwatts@futscherlab32gb:~/QIIME_files/qiime_tutorial-v1.5.0$ pick_otus_through_otu_table.py -i split_library_output/seqs.fna -o otus

Traceback (most recent call last):

File "/mnt/datas/home/gwatts/src/Qiime/scripts/pick_otus_through_otu_table.py", line 132, in <module>

main()

File "/mnt/datas/home/gwatts/src/Qiime/scripts/pick_otus_through_otu_table.py", line 129, in main

status_update_callback=status_update_callback)

File "/mnt/datas/home/gwatts/src/Qiime/qiime/workflow.py", line 471, in run_pick_otus_through_otu_table

close_logger_on_success=close_logger_on_success)

File "/mnt/datas/home/gwatts/src/Qiime/qiime/workflow.py", line 135, in call_commands_serially

raise WorkflowError, msg

qiime.workflow.WorkflowError:

*** ERROR RAISED DURING STEP: Pick OTUs

Command run was:

python /mnt/datas/home/gwatts/src/Qiime/scripts/pick_otus.py -i split_library_output/seqs.fna -o otus/uclust_picked_otus

Command returned exit status: 1

Stdout:

Stderr

Traceback (most recent call last):

File "/mnt/datas/home/gwatts/src/Qiime/scripts/pick_otus.py", line 593, in <module>

main()

File "/mnt/datas/home/gwatts/src/Qiime/scripts/pick_otus.py", line 476, in main

result_path=result_path,log_path=log_path,HALT_EXEC=False)

File "/mnt/datas/home/gwatts/src/Qiime/qiime/pick_otus.py", line 881, in __call__

HALT_EXEC=HALT_EXEC)

File "/mnt/datas/home/gwatts/src/pycogent/cogent/app/uclust.py", line 550, in get_clusters_from_fasta_filepath

raise ApplicationError, ('Error running uclust. Possible causes are '

cogent.app.util.ApplicationError: Error running uclust. Possible causes are unsupported version (current supported version is v1.2.22) is installed or improperly formatted input file was provided

gwatts@futscherlab32gb:~/QIIME_files/qiime_tutorial-v1.5.0$

#Uh, oh, the very same error I have had all day with my own .fna files! So it's not my data file, there's something not right with my install? I looked at another thread here on the forums, one suggestion was to make sure uclust was named correctly. So searching for uclust anywhere in my directories:

gwatts@futscherlab32gb:~/QIIME_files/qiime_tutorial-v1.5.0$ cd

gwatts@futscherlab32gb:~$ pwd

/mnt/datas/home/gwatts

gwatts@futscherlab32gb:~$ find -iname '*uclust*'

./src/Qiime/doc/scripts/parallel_pick_otus_uclust_ref.rst

./src/Qiime/doc/scripts/.svn/text-base/parallel_pick_otus_uclust_ref.rst.svn-base

./src/Qiime/scripts/.svn/prop-base/parallel_pick_otus_uclust_ref.py.svn-base

./src/Qiime/scripts/.svn/text-base/parallel_pick_otus_uclust_ref.py.svn-base

./src/Qiime/scripts/parallel_pick_otus_uclust_ref.py

./src/pycogent/cogent/app/uclust.py

./src/pycogent/cogent/app/.svn/prop-base/uclust.py.svn-base

./src/pycogent/cogent/app/.svn/text-base/uclust.py.svn-base

./src/pycogent/cogent/app/uclust.pyc

./src/pycogent/tests/test_app/test_uclust.py

./src/pycogent/tests/test_app/.svn/prop-base/test_uclust.py.svn-base

./src/pycogent/tests/test_app/.svn/text-base/test_uclust.py.svn-base

./QIIME_files/qiime_tutorial-v1.5.0/otus/uclust_picked_otus

./QIIME_files/BC.works/BC_otus/uclust_picked_otus

gwatts@futscherlab32gb:~$

#doesn't look like there is a uclust anywhere, just other things that contain the word...Another suggestion was to determine what version of uclust was installed:

gwatts@futscherlab32gb:~$ pwd

/mnt/datas/home/gwatts

gwatts@futscherlab32gb:~$ ls

BugSeq16s QIIME_files src

gwatts@futscherlab32gb:~$ uclust --version

uclust: command not found

gwatts@futscherlab32gb:~$ cd src

gwatts@futscherlab32gb:~/src$ uclust --version

uclust: command not found

gwatts@futscherlab32gb:~/src$ ls

biom-format pycogent Qiime

gwatts@futscherlab32gb:~/src$ cd Qiime/

gwatts@futscherlab32gb:~/src/Qiime$ uclust --version

uclust: command not found

gwatts@futscherlab32gb:~/src/Qiime$

# no luck getting this to run in any of the three likely directories...another suggestion was to run print_qiime_config.py -t:

cd-hit is in path and version is supported ... FAIL

test_chimeraSlayer_install (__main__.Qiime_config)

no obvious problems with ChimeraSlayer install ... FAIL

test_clearcut_supported_version (__main__.Qiime_config)

clearcut is in path and version is supported ... FAIL

test_cluster_jobs_fp (__main__.Qiime_config)

cluster_jobs_fp is set to a valid path and is executable ... FAIL

test_denoiser_supported_version (__main__.Qiime_config)

denoiser aligner is ready to use ... FAIL

test_for_obsolete_values (__main__.Qiime_config)

local qiime_config has no extra params ... ok

test_matplotlib_suported_version (__main__.Qiime_config)

maptplotlib version is supported ... ERROR

test_mothur_supported_version (__main__.Qiime_config)

mothur is in path and version is supported ... FAIL

test_muscle_supported_version (__main__.Qiime_config)

muscle is in path and version is supported ... FAIL

test_numpy_suported_version (__main__.Qiime_config)

numpy version is supported ... FAIL

test_pplacer_supported_version (__main__.Qiime_config)

pplacer is in path and version is supported ... FAIL

test_pynast_suported_version (__main__.Qiime_config)

pynast version is supported ... FAIL

test_pynast_template_alignment_blastdb_fp (__main__.Qiime_config)

pynast_template_alignment_blastdb, if set, is set to a valid path ... ok

test_pynast_template_alignment_fp (__main__.Qiime_config)

pynast_template_alignment, if set, is set to a valid path ... ok

test_python_exe_fp (__main__.Qiime_config)

python_exe_fp is set to a working python env ... ok

test_python_supported_version (__main__.Qiime_config)

python is in path and version is supported ... FAIL

test_qiime_scripts_dir (__main__.Qiime_config)

qiime_scripts_dir, if set, is set to a valid path ... ok

test_qiime_test_data_dir (__main__.Qiime_config)

qiime_test_data_dir, if set, is set to a valid path ... ok

test_raxmlHPC_supported_version (__main__.Qiime_config)

raxmlHPC is in path and version is supported ... FAIL

test_rtax_supported_version (__main__.Qiime_config)

rtax is in path and version is supported ... FAIL

test_temp_dir (__main__.Qiime_config)

temp_dir, if set, is set to a valid path ... ok

test_template_alignment_lanemask_fp (__main__.Qiime_config)

template_alignment_lanemask, if set, is set to a valid path ... ok

test_uclust_supported_version (__main__.Qiime_config)

uclust is in path and version is supported ... FAIL

test_usearch_supported_version (__main__.Qiime_config)

usearch is in path and version is supported ... FAIL

test_working_dir (__main__.Qiime_config)

working_dir, if set, is set to a valid path ... ok

======================================================================

ERROR: test_matplotlib_suported_version (__main__.Qiime_config)

maptplotlib version is supported

----------------------------------------------------------------------

Traceback (most recent call last):

File "/mnt/datas/home/gwatts/src/Qiime/scripts/print_qiime_config.py", line 365, in test_matplotlib_suported_version

version = tuple(map(int,matplotlib_lib_version.split('.')))

ValueError: invalid literal for int() with base 10: '1rc'

======================================================================

FAIL: test_FastTree_supported_version (__main__.Qiime_config)

FastTree is in path and version is supported

----------------------------------------------------------------------

Traceback (most recent call last):

File "/mnt/datas/home/gwatts/src/Qiime/scripts/print_qiime_config.py", line 424, in test_FastTree_supported_version

"which components of QIIME you plan to use.")

AssertionError: FastTree not found. This may or may not be a problem depending on which components of QIIME you plan to use.

======================================================================

FAIL: test_INFERNAL_supported_version (__main__.Qiime_config)

INFERNAL is in path and version is supported

----------------------------------------------------------------------

Traceback (most recent call last):

File "/mnt/datas/home/gwatts/src/Qiime/scripts/print_qiime_config.py", line 466, in test_INFERNAL_supported_version

"which components of QIIME you plan to use.")

AssertionError: Infernal not found. This may or may not be a problem depending on which components of QIIME you plan to use.

======================================================================

FAIL: test_ParsInsert_supported_version (__main__.Qiime_config)

ParsInsert is in path and version is supported

----------------------------------------------------------------------

Traceback (most recent call last):

File "/mnt/datas/home/gwatts/src/Qiime/scripts/print_qiime_config.py", line 652, in test_ParsInsert_supported_version

"which components of QIIME you plan to use.")

AssertionError: ParsInsert not found. This may or may not be a problem depending on which components of QIIME you plan to use.

======================================================================

FAIL: test_R_supported_version (__main__.Qiime_config)

R is in path and version is supported

----------------------------------------------------------------------

Traceback (most recent call last):

File "/mnt/datas/home/gwatts/src/Qiime/scripts/print_qiime_config.py", line 711, in test_R_supported_version

% ('.'.join(map(str,acceptable_version)), version_string))

AssertionError: Unsupported R version. (2, 12, 0).(2, 12, 0) is required, but running 2.15.1.

======================================================================

FAIL: test_ampliconnoise_install (__main__.Qiime_config)

AmpliconNoise install looks sane.

----------------------------------------------------------------------

Traceback (most recent call last):

File "/mnt/datas/home/gwatts/src/Qiime/scripts/print_qiime_config.py", line 143, in test_ampliconnoise_install

"$PYRO_LOOKUP_FILE variable is not set. See %s for help." % url)

AssertionError: $PYRO_LOOKUP_FILE variable is not set. See http://www.qiime.org/install/install.html#ampliconnoise-install for help.

======================================================================

FAIL: test_blast_supported_version (__main__.Qiime_config)

blast is in path and version is supported

----------------------------------------------------------------------

Traceback (most recent call last):

File "/mnt/datas/home/gwatts/src/Qiime/scripts/print_qiime_config.py", line 417, in test_blast_supported_version

% ('.'.join(map(str,acceptable_version)), version_string))

AssertionError: Unsupported blast version. 2.2.22 is required, but running 2.2.25.

======================================================================

FAIL: test_cdbtools_supported_version (__main__.Qiime_config)

cdbtools is in path and version is supported

----------------------------------------------------------------------

Traceback (most recent call last):

File "/mnt/datas/home/gwatts/src/Qiime/scripts/print_qiime_config.py", line 445, in test_cdbtools_supported_version

"which components of QIIME you plan to use.")

AssertionError: cdbtools not found. This may or may not be a problem depending on which components of QIIME you plan to use.

======================================================================

FAIL: test_cdhit_supported_version (__main__.Qiime_config)

cd-hit is in path and version is supported

----------------------------------------------------------------------

Traceback (most recent call last):

File "/mnt/datas/home/gwatts/src/Qiime/scripts/print_qiime_config.py", line 588, in test_cdhit_supported_version

"which components of QIIME you plan to use.")

AssertionError: cd-hit not found. This may or may not be a problem depending on which components of QIIME you plan to use.

======================================================================

FAIL: test_chimeraSlayer_install (__main__.Qiime_config)

no obvious problems with ChimeraSlayer install

----------------------------------------------------------------------

Traceback (most recent call last):

File "/mnt/datas/home/gwatts/src/Qiime/scripts/print_qiime_config.py", line 288, in test_chimeraSlayer_install

self.assertTrue(chim_slay,"ChimeraSlayer was not found in your $PATH")

AssertionError: ChimeraSlayer was not found in your $PATH

======================================================================

FAIL: test_clearcut_supported_version (__main__.Qiime_config)

clearcut is in path and version is supported

----------------------------------------------------------------------

Traceback (most recent call last):

File "/mnt/datas/home/gwatts/src/Qiime/scripts/print_qiime_config.py", line 568, in test_clearcut_supported_version

"which components of QIIME you plan to use.")

AssertionError: clearcut not found. This may or may not be a problem depending on which components of QIIME you plan to use.

======================================================================

FAIL: test_cluster_jobs_fp (__main__.Qiime_config)

cluster_jobs_fp is set to a valid path and is executable

----------------------------------------------------------------------

Traceback (most recent call last):

File "/mnt/datas/home/gwatts/src/Qiime/scripts/print_qiime_config.py", line 102, in test_cluster_jobs_fp

self.fail("Your qiime_config file doesn't have cluster_jobs_fp\n.")

AssertionError: Your qiime_config file doesn't have cluster_jobs_fp

.

======================================================================

FAIL: test_denoiser_supported_version (__main__.Qiime_config)

denoiser aligner is ready to use

----------------------------------------------------------------------

Traceback (most recent call last):

File "/mnt/datas/home/gwatts/src/Qiime/scripts/print_qiime_config.py", line 540, in test_denoiser_supported_version

"which components of QIIME you plan to use.")

AssertionError: Denoiser flowgram aligner not found or not executable.This may or may not be a problem depending on which components of QIIME you plan to use.

======================================================================

FAIL: test_mothur_supported_version (__main__.Qiime_config)

mothur is in path and version is supported

----------------------------------------------------------------------

Traceback (most recent call last):

File "/mnt/datas/home/gwatts/src/Qiime/scripts/print_qiime_config.py", line 508, in test_mothur_supported_version

"which components of QIIME you plan to use.")

AssertionError: mothur not found. This may or may not be a problem depending on which components of QIIME you plan to use.

======================================================================

FAIL: test_muscle_supported_version (__main__.Qiime_config)

muscle is in path and version is supported

----------------------------------------------------------------------

Traceback (most recent call last):

File "/mnt/datas/home/gwatts/src/Qiime/scripts/print_qiime_config.py", line 487, in test_muscle_supported_version

"which components of QIIME you plan to use.")

AssertionError: muscle not found. This may or may not be a problem depending on which components of QIIME you plan to use.

======================================================================

FAIL: test_numpy_suported_version (__main__.Qiime_config)

numpy version is supported

----------------------------------------------------------------------

Traceback (most recent call last):

File "/mnt/datas/home/gwatts/src/Qiime/scripts/print_qiime_config.py", line 356, in test_numpy_suported_version

version_string))

AssertionError: Unsupported numpy version. Must be >= 1.5.1 and <= 1.5.1 , but running 1.6.1.

======================================================================

FAIL: test_pplacer_supported_version (__main__.Qiime_config)

pplacer is in path and version is supported

----------------------------------------------------------------------

Traceback (most recent call last):

File "/mnt/datas/home/gwatts/src/Qiime/scripts/print_qiime_config.py", line 631, in test_pplacer_supported_version

"which components of QIIME you plan to use.")

AssertionError: pplacer not found. This may or may not be a problem depending on which components of QIIME you plan to use.

======================================================================

FAIL: test_pynast_suported_version (__main__.Qiime_config)

pynast version is supported

----------------------------------------------------------------------

Traceback (most recent call last):

File "/mnt/datas/home/gwatts/src/Qiime/scripts/print_qiime_config.py", line 396, in test_pynast_suported_version

version_string))

AssertionError: Unsupported pynast version. Must be >= 1.1 and < 1.2 , but running Not installed.

======================================================================

FAIL: test_python_supported_version (__main__.Qiime_config)

python is in path and version is supported

----------------------------------------------------------------------

Traceback (most recent call last):

File "/mnt/datas/home/gwatts/src/Qiime/scripts/print_qiime_config.py", line 338, in test_python_supported_version

version_string))

AssertionError: Unsupported python version. Must be >= 2.7.1 and <= 2.7.2 , but running 2.7.3.

======================================================================

FAIL: test_raxmlHPC_supported_version (__main__.Qiime_config)

raxmlHPC is in path and version is supported

----------------------------------------------------------------------

Traceback (most recent call last):

File "/mnt/datas/home/gwatts/src/Qiime/scripts/print_qiime_config.py", line 547, in test_raxmlHPC_supported_version

"which components of QIIME you plan to use.")

AssertionError: raxmlHPC not found. This may or may not be a problem depending on which components of QIIME you plan to use.

======================================================================

FAIL: test_rtax_supported_version (__main__.Qiime_config)

rtax is in path and version is supported

----------------------------------------------------------------------

Traceback (most recent call last):

File "/mnt/datas/home/gwatts/src/Qiime/scripts/print_qiime_config.py", line 610, in test_rtax_supported_version

"which components of QIIME you plan to use.")

AssertionError: rtax not found. This may or may not be a problem depending on which components of QIIME you plan to use.

======================================================================

FAIL: test_uclust_supported_version (__main__.Qiime_config)

uclust is in path and version is supported

----------------------------------------------------------------------

Traceback (most recent call last):

File "/mnt/datas/home/gwatts/src/Qiime/scripts/print_qiime_config.py", line 299, in test_uclust_supported_version

"which components of QIIME you plan to use.")

AssertionError: uclust not found. This may or may not be a problem depending on which components of QIIME you plan to use.

======================================================================

FAIL: test_usearch_supported_version (__main__.Qiime_config)

usearch is in path and version is supported

----------------------------------------------------------------------

Traceback (most recent call last):

File "/mnt/datas/home/gwatts/src/Qiime/scripts/print_qiime_config.py", line 676, in test_usearch_supported_version

"which components of QIIME you plan to use.")

AssertionError: usearch not found. This may or may not be a problem depending on which components of QIIME you plan to use.

----------------------------------------------------------------------

Ran 34 tests in 0.085s

FAILED (failures=22, errors=1)

gwatts@futscherlab32gb:~$

#I don't think this is all of the output, but it's as much as I could grab from the terminal....does the outpur go to a log file so I can post all of it? seems there's a lot of errors...finslly I tried going into the tests directory and running the test_pick_otus.py:

gwatts@futscherlab32gb:~$ ls

BugSeq16s QIIME_files src

gwatts@futscherlab32gb:~$ cd src/

gwatts@futscherlab32gb:~/src$ ls

biom-format pycogent Qiime

gwatts@futscherlab32gb:~/src$ cd Qiime/

gwatts@futscherlab32gb:~/src/Qiime$ ls

ChangeLog doc examples INSTALL qiime qiime_tutorial README requirements.txt scripts setup.py tests web

gwatts@futscherlab32gb:~/src/Qiime$ cd tests/

gwatts@futscherlab32gb:~/src/Qiime/tests$ ls

all_tests.py test_compare_distance_matrices.py test_identify_chimeric_seqs.py test_otu_category_significance.py test_sort.py

__init__.py test_compare_taxa_summaries.py test_insert_seqs_into_tree.py test_parallel test_split_libraries_fastq.py

test_add_qiime_labels.py test_conditional_uncovered_probability.py test_longitudinal_otu_category_significance.py test_parse.py test_split_libraries.py

test_adjust_seq_orientation.py test_convert_fastaqual_fastq.py test_make_2d_plots.py test_pick_otus.py test_split.py

test_align_seqs.py test_core_microbiome.py test_make_3d_plots.py test_pick_rep_set.py test_stats.py

test_alpha_diversity.py test_demultiplex_fasta.py test_make_bootstrapped_tree.py test_pick_subsampled_reference_otus_through_otu_table.py test_submit_to_mgrast.py

test_assign_taxonomy.py test_denoiser test_make_distance_histograms.py test_plot_rank_abundance_graph.py test_summarize_taxa.py

test_barcode.py test_denoise_wrapper.py test_make_fastq.py test_plot_semivariogram.py test_supervised_learning.py

test_beta_diversity.py test_distance_matrix_from_mapping.py test_make_library_id_lists.py test_plot_taxa_summary.py test_support_files

test_beta_metrics.py test_exclude_seqs_by_blast.py test_make_otu_heatmap_html.py test_principal_coordinates.py test_transform_coordinate_matrices.py

test_biplots.py test_filter_alignment.py test_make_otu_heatmap.py test_process_sff.py test_tree_compare.py

test_categorized_dist_scatterplot.py test_filter_otus_by_sample.py test_make_otu_network.py test_pycogent_backports test_trim_sff_primers.py

test_check_id_map.py test_filter.py test_make_otu_table.py test_quality.py test_truncate_fasta_qual_files.py

test_clean_raxml_parsimony_tree.py test_format.py test_make_per_library_sff.py test_quality_scores_plot.py test_truncate_reverse_primer.py

test_cluster_quality.py test_golay.py test_make_phylogeny.py test_rarefaction.py test_util.py

test_collate_alpha.py test_group.py test_make_rarefaction_plots.py test_relatedness.py test_validate_demultiplexed_fasta.py

test_colors.py test_hamming.py test_map_reads_to_reference.py test_shared_phylotypes.py test_workflow.py

test_compare_alpha_diversities.py test_hierarchical_cluster.py test_merge_mapping_files.py test_simsam.py

gwatts@futscherlab32gb:~/src/Qiime/tests$ test_pick_otus.py

test_pick_otus.py: command not found

# no luck, it was suggested to me to add ./ so linux knows to look in the same directory I am in:

gwatts@futscherlab32gb:~/src/Qiime/tests$ ./test_pick_otus.py

-bash: ./test_pick_otus.py: Permission denied

gwatts@futscherlab32gb:~/src/Qiime/tests$

#I am double stuck it seems

What should I do?

Jose Navas

unread,

Sep 12, 2012, 10:07:44 PM9/12/12

to qiime...@googlegroups.com

Hi George,

You don't have uclust installed. In order to install it, you can follow the instructions on the last post of this thread:
https://groups.google.com/group/qiime-forum/browse_thread/thread/b77db7bd5997b5f4

I noticed that you're using Ubuntu 12.04 and in your second post you said that the Ubuntu 12.04 specific instruction doesn't work for you. Can I ask what problems do you find with it? There is a lot of dependencies not installed in your machine so maybe we can give it a chance...

Thanks,

Jose N

2012/9/12 GeorgeWatts <gwa...@azcc.arizona.edu>

--

GeorgeWatts

unread,

Sep 13, 2012, 12:48:31 PM9/13/12

to qiime...@googlegroups.com

Jose, I have tried 3 install methods:

First, I tried the Ubuntu 12.04 install instructions here: http://qiime.org/install/ubuntu_install.html:

I asked my administrator (since I don't have sudo privileges) to run:

sudo pico /etc/apt/sources.list and uncomment the universe and multiverse repositories, then:

sudo apt-get --force-yes -y install libssl-dev libzmq-dev libgsl0-dev openjdk-6-jdk libxml2 libxslt1.1 libxslt1-dev ant subversion build-essential zlib1g-dev libpng12-dev libfreetype6-dev mpich2 libreadline-dev gfortran unzip libmysqlclient18 libmysqlclient-dev ghc

I then ran:

mkdir $HOME/qiime_software/

wget ftp://thebeast.colorado.edu/pub/QIIME-v1.5.0-dependencies/app-deploy-qiime-1.5.0.tgz

tar zxvf app-deploy-qiime-1.5.0.tgz

cd app-deploy-qiime-1.5.0

python app-deploy.py $HOME/qiime_software/ -f etc/qiime_1.5.0.conf --force-remove-failed-dirs

The install actually worked sort of - I could only run scripts in their directories and I had to have my data in the directory as well. I got the tutorial to work with my own test data all the way through to the step where you run make_otu_network.py which is deep into step 7 of the tutorial here: http://qiime.org/tutorials/tutorial.html#step-7-make-otu-table

The problem was that the install didn't seem to know where anything was, so I had problems detailed in an email with Tony below:

Background:
I’ve made a mapping.txt file: JB.Mapping.txt
I have an .FNA file: JB.fna

Here’s what it looks like:
gwatts@futscherlab32gb:~/qiime_software/qiime-1.5.0-release/lib/qiime$ head JB.fna
>112_BC1_1 RXB0T:48:298 orig_bc=ACTCA new_bc=ACTCA bc_diffs=0
ACGAGCTGACGACAGCCATGCACCACCTGTCTACCGGTTCCCGAAGGCACAGTCATACTTCTATGACCTCCCGGAGATGTCAAGGTCTGGTA
>112_BC1_2 RXB0T:49:264 orig_bc=ACTCA new_bc=ACTCA bc_diffs=0
ACGAGCTGACGACAGCCATGCAGCACCTGTGTTACGGTTCCCGAAGGCACTCCTCCGTCTCTGGAGGATTCCGTACATGTCAAGACC
>112_BC1_3 RXB0T:32:335 orig_bc=ACTCA new_bc=ACTCA bc_diffs=0
ACGAGCTGCGACAACCATGCACCACCTGCACACGACCAACTAAATGCCACCACATCTCTGCAGTGTCGCCGTGCATGTCAAGCCT
>112_BC1_4 RXB0T:40:344 orig_bc=ACTCA new_bc=ACTCA bc_diffs=0
ACGAGCTGACGACAACCGTGCACCACCTGTTTTCTGGCTTCCGAAGAAGAGGAACTATCTCTAGTTCTGTCCATCAATGTCAAGACC
>112_BC1_5 RXB0T:44:307 orig_bc=ACTCA new_bc=ACTCA bc_diffs=0
ACGAGCTGACGACAGCCATGCAGCACCTGTCTCATGGTTCCCGAAGGCACTCCTCCGTCTCTGGAGGATTCCGTACATGTCA

Step 1) Move the Mapping and .fna files to: ~/qiime_software/qiime-1.5.0-release/lib/qiime/ , where ~ is short for the full path: /mnt/datas/home/gwatts/qiime_software/qiime-1.5.0-release/lib/qiime

Step 2) Check the Mapping file:

From within ~/qiime_software/qiime-1.5.0-release/lib/qiime/ run:

check_id_map.py

Now, when I did this with the test data (using different Mapping and .fna files) all went well, and I went much deeper through the tutorial, but now I get:

gwatts@futscherlab32gb:~/qiime_software/qiime-1.5.0-release/lib/qiime$ check_id_map.py -m JB.Mapping.txt -o mapping_outputJB
check_id_map.py: command not found

weird, since that script is sitting right there in the same directory…ok…so try calling python specifically:

gwatts@futscherlab32gb:~/qiime_software/qiime-1.5.0-release/lib/qiime$ python check_id_map.py -m JB.Mapping.txt -o mapping_outputJB

Traceback (most recent call last):

File "check_id_map.py", line 58, in <module>
from qiime.util import get_qiime_project_dir, duplicates_indices

ImportError: No module named qiime.util

I ran into many of these ImportErrors when I was trying to figure out what directory to try running things from. Checking the forums it seems to be issues with various packages knowing where various dependencies and scripts are located. Typing echo $PYTHONPATH yields:

gwatts@futscherlab32gb:~/qiime_software/qiime-1.5.0-release/lib/qiime$ echo $PYTHONPATH

gwatts@futscherlab32gb:~/qiime_software/qiime-1.5.0-release/lib/qiime$

Nothing, just returns the prompt.

I installed QIIME using the Ubuntu 12.04 instructions here: http://qiime.org/install/ubuntu_install.html

Paraphrasing,
1) Uncomment the universe and multiverse repositories - they were already uncommented. Good, moving on.
2) get all the pieces with apt-get, this command ran without issue.
3) Make a QIIME directory, wget the tarball, unpack it, and run the deploy script. All libraries installed successfully. No errors.

Once installed, use cd to get into ~/qiime_software/qiime-1.5.0-release/tests/

from within the tests directory, check the QIIME scripts by running:
all_tests.py from within the tests directory. Output scrolled by ending in:

Result summary
==============

Unit test result summary
------------------------

Failed the following unit tests.
/mnt/datas/home/gwatts/qiime_software/qiime-1.5.0-release/tests/test_pick_otus.py
/mnt/datas/home/gwatts/qiime_software/qiime-1.5.0-release/tests/test_pycogent_backports/test_mothur.py

Failed the following unit tests, in part or whole due to missing external applications.
Depending on the QIIME features you plan to use, this may not be critical.
/mnt/datas/home/gwatts/qiime_software/qiime-1.5.0-release/tests/test_assign_taxonomy.py
/mnt/datas/home/gwatts/qiime_software/qiime-1.5.0-release/tests/test_denoiser/test_make_cluster_jobs.py
/mnt/datas/home/gwatts/qiime_software/qiime-1.5.0-release/tests/test_make_per_library_sff.py
/mnt/datas/home/gwatts/qiime_software/qiime-1.5.0-release/tests/test_pycogent_backports/test_rtax.py
/mnt/datas/home/gwatts/qiime_software/qiime-1.5.0-release/tests/test_pycogent_backports/test_usearch.py
/mnt/datas/home/gwatts/qiime_software/qiime-1.5.0-release/tests/test_trim_sff_primers.py

Basic script test result summary
--------------------------------

All basic script tests passed successfully.

Could not run script usage tests because qiime_test_data is not defined in your qiime_config.

Ok, some stuff failed, most succeeded, including basic scripts, so without a log file, moving on... This was the way things stood when I proceeded with the test runs, but now I have ImportError.

Tony, replied:

You might want to keep your data in a separate folder from the library and scripts files-although it's unlikely to happen, you don't want to accidentally delete or overwrite the files that are there.

I think you need to add these two lines to your .bashrc, and reopen the terminal and see if you can call QIIME scripts correctly.

export PYTHONPATH=/mnt/datas/home/gwatts/qiime_software/qiime-1.5.0-release/:$PYTHONPATH

export PYTHONPATH=/mnt/datas/home/gwatts/qiime_software/qiime-1.5.0-release/lib/:$PYTHONPATH

export PATH=/mnt/datas/home/gwatts/qiime_software/qiime-1.5.0-release/scripts/:$PATH

Once this is done, save the .bashrc file, and then close and reopen the terminal.

Then type:

check_id_map.py -h

and see if this gives you the proper help text.

Which I now know was probably the right fix, but was doomed to failure because my .bashrc file is not sourced when I log in unless I also edit my .proifle with: source .bashrc. At the time I didn't know this and I replied:

Thanks for the reply. I edited my .bashrc (attached) and re-opened my terminal, navigated to the same path as before and tried to get the help for the check_id_map.py as you instructed:

gwatts@futscherlab32gb:~/qiime_software/qiime-1.5.0-release/lib/qiime$ check_id_map.py -h
check_id_map.py: command not found
gwatts@futscherlab32gb:~/qiime_software/qiime-1.5.0-release/lib/qiime$ python check_id_map.py -h

Traceback (most recent call last):

File "check_id_map.py", line 58, in <module>
from qiime.util import get_qiime_project_dir, duplicates_indices

ImportError: No module named qiime.util

gwatts@futscherlab32gb:~/qiime_software/qiime-1.5.0-release/lib/qiime$

Regarding my data in separate folders, I would love to do this, but I couldn’t get anything to run before unless I was in the lib/qiime directory….now I can’t get anything to run anywhere.

To which Tony replied:

The install seems quite strange, and I would recommend at this point to edit the .bashrc and comment out all of the lines dealing with QIIME (put a # in front of the lines), redownload QIIME, and install starting with the instructions here: http://www.qiime.org/install/install.html#unpacking-qiime-release-only

Those instructions should get the PATH and PYTHONPATH variable set up correctly, so you can call QIIME scripts, like check_id_map.py, from any location, and it should also be able to properly import the library files.

So I deleted all the files in my home direcotry and started again. Following those instructions led me to started this thread, in which Will had me abandon the second install and instead start a third install using the dev version (see Will's second post in this thread).

And that has left me where I am now. Should I just go back and re-install using the original instructions (http://qiime.org/install/ubuntu_install.html) which should work now that I am sourcing the ,bashrc file in my .profile? If I do try this, I am stuck wondering if I need to get my admin to re-run the sudo apt-get command:

sudo apt-get --force-yes -y install libssl-dev libzmq-dev libgsl0-dev openjdk-6-jdk libxml2 libxslt1.1 libxslt1-dev ant subversion build-essential zlib1g-dev libpng12-dev libfreetype6-dev mpich2 libreadline-dev gfortran unzip libmysqlclient18 libmysqlclient-dev ghc

After all, I don't think I have permission to delete the files that were installed by this command so all of these dependencies should still be installed, yet, as you noted, my current install is missing many dependencies.

I think part of my problem is that all three install attempts probably could work, but I have had 3 different people trying to help me and all have had different approaches...the question is which install method should I try to get working today?

Jai Ram Rideout

unread,

Sep 13, 2012, 1:15:27 PM9/13/12

to qiime...@googlegroups.com

Hi George,

I'd recommend using the Ubuntu 12.04 install instructions that you
started with: http://qiime.org/install/ubuntu_install.html You don't
need to have your sys admin reinstall everything.

I think most things installed correctly from your first attempt, but
the instructions failed to mention a crucial step that will allow you
to run scripts outside of the scripts/ directory. You need to run the
following command after you run the app-deploy.py script:

source ~/.bashrc

We'll fix the documentation to include this step. If you don't use
.bashrc and instead use .bash_profile, you can source your .bashrc
from within your .bash_profile, or just make sure the following line
is at the bottom of your .bash_profile:

source /mnt/datas/home/gwatts/qiime_software/activate.sh

I'd also check that you've removed other lines from your
.bashrc/.bash_profile that mess with you PATH and/or PYTHONPATH from
your previous attempts to get this installed.

Finally, from your home directory, you should be able to run the
following command:

print_qiime_config.py -t

which will test the sanity of your install.

Please let me know how it goes.

-Jai

> --
>
>
>

Jai Ram Rideout

unread,

Sep 13, 2012, 1:26:53 PM9/13/12

to qiime...@googlegroups.com

Sorry George, you should source your .bashrc from within your .profile
(I didn't notice you were using that instead of .bash_profile).

-Jai

GeorgeWatts

unread,

Sep 13, 2012, 3:28:55 PM9/13/12

to qiime...@googlegroups.com

Jai,

Thanks for the reply, I really do appreciate all the help you guys are providing, and I am learning a lot about Linux as I go. So, I went back to the original install instructions (http://qiime.org/install/ubuntu_install.html) as and proceeded as follows:

I will skip the following two steps since they've already been done:

sudo pico /etc/apt/sources.list

sudo apt-get --force-yes -y install libssl-dev libzmq-dev libgsl0-dev openjdk-6-jdk libxml2 libxslt1.1 libxslt1-dev ant subversion build-essential zlib1g-dev libpng12-dev libfreetype6-dev mpich2 libreadline-dev gfortran unzip libmysqlclient18 libmysqlclient-dev ghc #I will skip this step as well

I delete the src directory containing the current install:

gwatts@futscherlab32gb:~$ pwd
/mnt/datas/home/gwatts
gwatts@futscherlab32gb:~$ ls
BugSeq16s QIIME_files src

gwatts@futscherlab32gb:~$ rm -rf src

I run the following commands:

mkdir qiime_software/

wget ftp://thebeast.colorado.edu/pub/QIIME-v1.5.0-dependencies/app-deploy-qiime-1.5.0.tgz

tar zxvf app-deploy-qiime-1.5.0.tgz

cd app-deploy-qiime-1.5.0

python app-deploy.py $HOME/qiime_software/ -f etc/qiime_1.5.0.conf --force-remove-failed-dirs

gwatts@futscherlab32gb:~$ mkdir qiime_software/
gwatts@futscherlab32gb:~$ wget ftp://thebeast.colorado.edu/pub/QIIME-v1.5.0-dependencies/app-deploy-qiime-1.5.0.tgz
--2012-09-13 10:50:04-- ftp://thebeast.colorado.edu/pub/QIIME-v1.5.0-dependencies/app-deploy-qiime-1.5.0.tgz
=> `app-deploy-qiime-1.5.0.tgz'
Resolving thebeast.colorado.edu (thebeast.colorado.edu)... 128.138.93.17
Connecting to thebeast.colorado.edu (thebeast.colorado.edu)|128.138.93.17|:21... connected.
Logging in as anonymous ... Logged in!
==> SYST ... done. ==> PWD ... done.
==> TYPE I ... done. ==> CWD (1) /pub/QIIME-v1.5.0-dependencies ... done.
==> SIZE app-deploy-qiime-1.5.0.tgz ... 21057
==> PASV ... done. ==> RETR app-deploy-qiime-1.5.0.tgz ... done.
Length: 21057 (21K) (unauthoritative)

100%[===================================================================================================================================================================================================>] 21,057 --.-K/s in 0.09s

2012-09-13 10:50:05 (240 KB/s) - `app-deploy-qiime-1.5.0.tgz' saved [21057]

gwatts@futscherlab32gb:~$ tar zxvf app-deploy-qiime-1.5.0.tgz
app-deploy-qiime-1.5.0/
app-deploy-qiime-1.5.0/app-deploy.py
app-deploy-qiime-1.5.0/etc/
app-deploy-qiime-1.5.0/lib/
app-deploy-qiime-1.5.0/README
app-deploy-qiime-1.5.0/lib/__init__.py
app-deploy-qiime-1.5.0/lib/application.py
app-deploy-qiime-1.5.0/lib/config.py
app-deploy-qiime-1.5.0/lib/custom.py
app-deploy-qiime-1.5.0/lib/data.py
app-deploy-qiime-1.5.0/lib/dependencies.py
app-deploy-qiime-1.5.0/lib/deploy.py
app-deploy-qiime-1.5.0/lib/environment.py
app-deploy-qiime-1.5.0/lib/logconfig.py
app-deploy-qiime-1.5.0/lib/util.py
app-deploy-qiime-1.5.0/etc/qiime_1.5.0.conf
app-deploy-qiime-1.5.0/etc/qiime_1.5.0_repository.conf
app-deploy-qiime-1.5.0/etc/qiime_compy.conf

gwatts@futscherlab32gb:~$ cd app-deploy-qiime-1.5.0
gwatts@futscherlab32gb:~/app-deploy-qiime-1.5.0$ python app-deploy.py $HOME/qiime_software/ -f etc/qiime_1.5.0.conf --force-remove-failed-dirs

....lots of output ending with:

DEPLOYMENT SUMMARY

Packages deployed successfully:
data-lanemask, rtax, clearcut, cdbtools, raxml, drisee, ampliconnoise, data-core, cytoscape, vienna, fasttree, pplacer, gg_otus, infernal, blast, muscle, parsinsert, cdhit, rdpclassifier, uclust, chimeraslayer, mothur, python, tornado, setuptools, ipython, SQLAlchemy, pyzmq, MySQL-python, sphinx, mpi4py, numpy, biom-format, pycogent, pprospector, pynast, r, matplotlib, qiime

Packages skipped (assumed successful):

Packages failed to deploy:

Ok, good I have an install. Now to edit my .bashrc file in vi and add the following:

export PYTHONPATH=/mnt/datas/home/gwatts/qiime_software/qiime-1.5.0-release/:$PYTHONPATH
export PATH=/mnt/datas/home/gwatts/qiime_software/qiime-1.5.0-release/scripts/:$PATH
source /mnt/datas/home/gwatts/qiime_software/activate.sh

The edited .bashrc file is attached to this post as a .txt - added lines are at the bottom.

I then make sure my .bashrc is sourced next time I log in by adding:

source /mnt/datas/home/gwatts/.bashrc

to my .profile. The edited .profile file is attached to this post as a .txt - added line is at the bottom.

I don't need to log out, I can source my .bashrc with:

gwatts@futscherlab32gb:~$ source .bashrc
gwatts@futscherlab32gb:~$

Ok, so I should be ready to go with print_qiime_config.py -t:

gwatts@futscherlab32gb:~$ print_qiime_config.py -t

System information
==================
Platform: linux2

Python version: 2.7.3 (default, Aug 1 2012, 05:25:23) [GCC 4.6.3]
Python executable: /mnt/datas/home/gwatts/qiime_software/python-2.7.1-release/bin/python

Dependency versions
===================
PyCogent version: 1.5.1

NumPy version: 1.5.1
matplotlib version: 1.1.0

biom-format version: 0.9.3
QIIME library version: 1.5.0
QIIME script version: 1.5.0
PyNAST version (if installed): 1.1
RDP Classifier version (if installed): rdp_classifier-2.2.jar

QIIME config values
===================
blastmat_dir: /mnt/datas/home/gwatts/qiime_software/blast-2.2.22-release/data
sc_queue: all.q
topiaryexplorer_project_dir: None
pynast_template_alignment_fp: /mnt/datas/home/gwatts/qiime_software/core_set_aligned.fasta.imputed
cluster_jobs_fp: /mnt/datas/home/gwatts/qiime_software/qiime-1.5.0-release/bin/start_parallel_jobs.py
pynast_template_alignment_blastdb: None
assign_taxonomy_reference_seqs_fp: /mnt/datas/home/gwatts/qiime_software/gg_otus-4feb2011-release/rep_set/gg_97_otus_4feb2011.fasta
torque_queue: friendlyq
qiime_test_data_dir: None
template_alignment_lanemask_fp: /mnt/datas/home/gwatts/qiime_software/lanemask_in_1s_and_0s
jobs_to_start: 1
cloud_environment: False
qiime_scripts_dir: /mnt/datas/home/gwatts/qiime_software/qiime-1.5.0-release/bin
denoiser_min_per_core: 50
working_dir: /tmp/
python_exe_fp: /mnt/datas/home/gwatts/qiime_software/python-2.7.1-release/bin/python
temp_dir: /tmp/
blastall_fp: /mnt/datas/home/gwatts/qiime_software/blast-2.2.22-release/bin/blastall
seconds_to_sleep: 60
assign_taxonomy_id_to_taxonomy_fp: /mnt/datas/home/gwatts/qiime_software/gg_otus-4feb2011-release/taxonomies/greengenes_tax_rdp_train.txt

running checks:

test_FastTree_supported_version (__main__.Qiime_config)
FastTree is in path and version is supported ... ok
test_INFERNAL_supported_version (__main__.Qiime_config)
INFERNAL is in path and version is supported ... ok
test_ParsInsert_supported_version (__main__.Qiime_config)
ParsInsert is in path and version is supported ... ok
test_R_supported_version (__main__.Qiime_config)
R is in path and version is supported ... ok
test_ampliconnoise_install (__main__.Qiime_config)
AmpliconNoise install looks sane. ... ok
test_blast_supported_version (__main__.Qiime_config)
blast is in path and version is supported ... ok
test_blastall_fp (__main__.Qiime_config)
blastall_fp is set to a valid path ... ok
test_blastmat_dir (__main__.Qiime_config)
blastmat_dir is set to a valid path. ... ok
test_cdbtools_supported_version (__main__.Qiime_config)
cdbtools is in path and version is supported ... ok
test_cdhit_supported_version (__main__.Qiime_config)
cd-hit is in path and version is supported ... ok
test_chimeraSlayer_install (__main__.Qiime_config)
no obvious problems with ChimeraSlayer install ... ok
test_clearcut_supported_version (__main__.Qiime_config)
clearcut is in path and version is supported ... ok
test_cluster_jobs_fp (__main__.Qiime_config)
cluster_jobs_fp is set to a valid path and is executable ... ok
test_denoiser_supported_version (__main__.Qiime_config)
denoiser aligner is ready to use ... ok

test_for_obsolete_values (__main__.Qiime_config)
local qiime_config has no extra params ... ok
test_matplotlib_suported_version (__main__.Qiime_config)

maptplotlib version is supported ... ok

test_mothur_supported_version (__main__.Qiime_config)
mothur is in path and version is supported ... FAIL
test_muscle_supported_version (__main__.Qiime_config)

muscle is in path and version is supported ... ok
test_numpy_suported_version (__main__.Qiime_config)
numpy version is supported ... ok
test_pplacer_supported_version (__main__.Qiime_config)
pplacer is in path and version is supported ... ok
test_pynast_suported_version (__main__.Qiime_config)
pynast version is supported ... ok

test_pynast_template_alignment_blastdb_fp (__main__.Qiime_config)
pynast_template_alignment_blastdb, if set, is set to a valid path ... ok
test_pynast_template_alignment_fp (__main__.Qiime_config)
pynast_template_alignment, if set, is set to a valid path ... ok
test_python_exe_fp (__main__.Qiime_config)
python_exe_fp is set to a working python env ... ok
test_python_supported_version (__main__.Qiime_config)
python is in path and version is supported ... FAIL
test_qiime_scripts_dir (__main__.Qiime_config)
qiime_scripts_dir, if set, is set to a valid path ... ok
test_qiime_test_data_dir (__main__.Qiime_config)
qiime_test_data_dir, if set, is set to a valid path ... ok
test_raxmlHPC_supported_version (__main__.Qiime_config)

raxmlHPC is in path and version is supported ... ok

test_rtax_supported_version (__main__.Qiime_config)
rtax is in path and version is supported ... FAIL
test_temp_dir (__main__.Qiime_config)
temp_dir, if set, is set to a valid path ... ok
test_template_alignment_lanemask_fp (__main__.Qiime_config)
template_alignment_lanemask, if set, is set to a valid path ... ok
test_uclust_supported_version (__main__.Qiime_config)

uclust is in path and version is supported ... ok

test_usearch_supported_version (__main__.Qiime_config)
usearch is in path and version is supported ... FAIL
test_working_dir (__main__.Qiime_config)
working_dir, if set, is set to a valid path ... ok

======================================================================

FAIL: test_mothur_supported_version (__main__.Qiime_config)

mothur is in path and version is supported
----------------------------------------------------------------------

Traceback (most recent call last):
File "/mnt/datas/home/gwatts/qiime_software/qiime-1.5.0-release/bin/print_qiime_config.py", line 523, in test_mothur_supported_version

% ('.'.join(map(str,acceptable_version)), version_string))

AssertionError: Unsupported mothur version. 1.25.0 is required, but running mothur: error while loading shared libraries: libreadline.so.5: cannot open shared object file: No such file or directory

.

======================================================================
FAIL: test_python_supported_version (__main__.Qiime_config)
python is in path and version is supported
----------------------------------------------------------------------

Traceback (most recent call last):
File "/mnt/datas/home/gwatts/qiime_software/qiime-1.5.0-release/bin/print_qiime_config.py", line 338, in test_python_supported_version

version_string))
AssertionError: Unsupported python version. Must be >= 2.7.1 and <= 2.7.2 , but running 2.7.3.

======================================================================

FAIL: test_rtax_supported_version (__main__.Qiime_config)

rtax is in path and version is supported
----------------------------------------------------------------------

Traceback (most recent call last):
File "/mnt/datas/home/gwatts/qiime_software/qiime-1.5.0-release/bin/print_qiime_config.py", line 620, in test_rtax_supported_version

% ('.'.join(map(str,acceptable_version)), version_string))

AssertionError: Unsupported rtax version. (0, 981).(0, 981) is required, but running 0.982.

======================================================================
FAIL: test_usearch_supported_version (__main__.Qiime_config)
usearch is in path and version is supported
----------------------------------------------------------------------

Traceback (most recent call last):
File "/mnt/datas/home/gwatts/qiime_software/qiime-1.5.0-release/bin/print_qiime_config.py", line 668, in test_usearch_supported_version

"which components of QIIME you plan to use.")
AssertionError: usearch not found. This may or may not be a problem depending on which components of QIIME you plan to use.

----------------------------------------------------------------------

Ran 34 tests in 0.445s

FAILED (failures=4)
gwatts@futscherlab32gb:~$

4 errors reported:

1) Unsupported mothur version

#don't know what to do about this one.

2) Unsupported python version. Must be >= 2.7.1 and <= 2.7.2 , but running 2.7.3.

#so, I have a more recent version, I think I can ignore this error.

3) Unsupported rtax version. (0, 981).(0, 981) is required, but running 0.982.

#so, I have a more recent version, I think I can ignore this error.

4) usearch not found. This may or may not be a problem depending on which components of QIIME you plan to use.

#I think I can ignore.

Moving on, yesterday I could not run test_pick_otus.py, let's try that one:

gwatts@futscherlab32gb:~$ test_pick_otus.py
test_pick_otus.py: command not found
gwatts@futscherlab32gb:~$ pwd
/mnt/datas/home/gwatts
gwatts@futscherlab32gb:~$ cd qiime_software/qiime-1.5.0-release/tests/
gwatts@futscherlab32gb:~/qiime_software/qiime-1.5.0-release/tests$ test_pick_otus.py
test_pick_otus.py: command not found
gwatts@futscherlab32gb:~/qiime_software/qiime-1.5.0-release/tests$

Hmm, so that's not working....how about I try the tutorial steps on the test data I downloaded yesterday:

gwatts@futscherlab32gb:~/QIIME_files/qiime_tutorial-v1.5.0$ ls
18S_tutorial_files Fasting_Example.fna Fasting_Example.qual Fasting_Example.sff Fasting_Example.sff.txt Fasting_Map.txt qiime_tutorial_commands_parallel.sh qiime_tutorial_commands_serial.sh README

gwatts@futscherlab32gb:~/QIIME_files/qiime_tutorial-v1.5.0$ check_id_map.py -m Fasting_Map.txt -o mapping_output

No errors or warnings were found in mapping file.

Ok, the check_id_map.py works....

gwatts@futscherlab32gb:~/QIIME_files/qiime_tutorial-v1.5.0$ split_libraries.py -m Fasting_Map.txt -f Fasting_Example.fna -q Fasting_Example.qual -o split_library_output

gwatts@futscherlab32gb:~/QIIME_files/qiime_tutorial-v1.5.0$ cd split_library_output/

gwatts@futscherlab32gb:~/QIIME_files/qiime_tutorial-v1.5.0/split_library_output$ ls
histograms.txt seqs.fna split_library_log.txt

The split_libraries.py output is there...now let's try the pick_otus_through_otu_table.py:

gwatts@futscherlab32gb:~/QIIME_files/qiime_tutorial-v1.5.0/split_library_output$ cd ..
gwatts@futscherlab32gb:~/QIIME_files/qiime_tutorial-v1.5.0$ pick_otus_through_otu_table.py -i split_library_output/seqs.fna -o otus

gwatts@futscherlab32gb:~/QIIME_files/qiime_tutorial-v1.5.0$ cd otus
gwatts@futscherlab32gb:~/QIIME_files/qiime_tutorial-v1.5.0/otus$ ls
log_20120913114937.txt otu_table.biom pynast_aligned_seqs rdp_assigned_taxonomy rep_set rep_set.tre uclust_picked_otus
gwatts@futscherlab32gb:~/QIIME_files/qiime_tutorial-v1.5.0/otus$

Excellent! Let's try this with some of my own test .fna files:

gwatts@futscherlab32gb:~/QIIME_files$ pick_otus_through_otu_table.py -i test.fna -o test.otus
gwatts@futscherlab32gb:~/QIIME_files$ cd test.otus/
gwatts@futscherlab32gb:~/QIIME_files/test.otus$ ls
log_20120913115752.txt otu_table.biom pynast_aligned_seqs rdp_assigned_taxonomy rep_set rep_set.tre uclust_picked_otus

And now I have a .tre file for use with FigTree, which worked fine.....let's make the heatmap:

gwatts@futscherlab32gb:~/QIIME_files$ make_otu_heatmap_html.py -i test.otus/otu_table.biom -o test.otus/OTU_Heatmap/
gwatts@futscherlab32gb:~/QIIME_files$ cd test.otus/
gwatts@futscherlab32gb:~/QIIME_files/test.otus$ cd OTU_Heatmap/
gwatts@futscherlab32gb:~/QIIME_files/test.otus/OTU_Heatmap$ ls
js otu_table.html

I moved the js directory and .html file to my local machine and the heatmap worked fine. So other than the issue of not being able to run the test script and the errors above, I have success up to the point.

Questions:

1) what about the errors I got with print_qiime_config.py -t?

2) I think that I need to add another line to my .bashrc file to make python aware of the test script location to resolve the command not found when I tried test_pick_otus.py, is this correct?

profile.txt

bashrc.txt

Jai Ram Rideout

unread,

Sep 13, 2012, 4:24:11 PM9/13/12

to qiime...@googlegroups.com

Hi George,

Thanks for your detailed description of the steps that you took- it
makes it easy on us to track down where the problems are!

First, you do not need these lines in your .bashrc (and they may end
up causing you problems later on, so please delete them):

export PYTHONPATH=/mnt/datas/home/gwatts/qiime_software/qiime-1.5.0-release/:$PYTHONPATH
export PATH=/mnt/datas/home/gwatts/qiime_software/qiime-1.5.0-release/scripts/:$PATH

When you source activate.sh in your .bashrc, it sets all of this up
for you automatically.

> Questions:
> 1) what about the errors I got with print_qiime_config.py -t?

You can ignore the usearch error, it just means you don't have that
executable. See http://www.drive5.com/usearch/ if you need it.

I think the app-deploy script is incorrectly downloading the wrong
versions of mothur and rtax. We'll work on fixing this. The versions
you have may or may not work- I recommend running the test suite to
see if their tests fail.

I'm not sure about the python issue, though I *think* you'll be okay
to ignore it. We don't explicitly test on python 2.7.3, but I suspect
everything should work just fine (again, running the test suite is
highly recommended).

> 2) I think that I need to add another line to my .bashrc file to make python aware of the test script location to resolve the command not found when I tried test_pick_otus.py, is this correct?

The default QIIME install does not add the test scripts location to
your path, so nothing went wrong with this part of the installation.
To run all of QIIME's tests, cd into
qiime_software/qiime-1.5.0-release/ and run the following command:

python tests/all_tests.py

Hope this helps,
Jai

> --
>
>
>

GeorgeWatts

unread,

Sep 13, 2012, 7:04:11 PM9/13/12

to qiime...@googlegroups.com

Jai, I am glad the info in my posts is helpful.

I have removed the two lines from my .bashrc file as you suggested and re-sourced my ,bashrc file.

I then went into the qiime-1.5.0-release directory and ran the all_tests.py script. The script took a long time to run and the results scrolled irretrievably off the screen so I re-ran it with a re-direct to a file:

gwatts@futscherlab32gb:~/qiime_software/qiime-1.5.0-release$ python tests/all_tests.py > test_results.txt
gwatts@futscherlab32gb:~/qiime_software/qiime-1.5.0-release$ ls -l
total 408
drwxrwxr-x 2 gwatts gwatts 12288 Sep 13 10:57 bin
drwxrwxr-x 4 gwatts gwatts 4096 Sep 13 10:57 build
-rw-r--r-- 1 gwatts gwatts 44470 Sep 13 10:57 ChangeLog
drwxr-xr-x 12 gwatts gwatts 4096 Sep 13 10:57 doc
-rw-r--r-- 1 gwatts gwatts 105 Sep 13 10:57 INSTALL
drwxrwxr-x 2 gwatts gwatts 4096 Sep 13 15:00 jobs
drwxrwxr-x 3 gwatts gwatts 4096 Sep 13 10:57 lib
drwxr-xr-x 6 gwatts gwatts 4096 Sep 13 10:57 qiime
-rw-r--r-- 1 gwatts gwatts 3322 Sep 13 10:57 qiime_parameters.txt
drwxr-xr-x 3 gwatts gwatts 4096 Sep 13 10:57 qiime_tutorial
-rw-rw-r-- 1 gwatts gwatts 532 Sep 13 15:18 randomized_coords2.txt
-rw-r--r-- 1 gwatts gwatts 3019 Sep 13 10:57 README
-rw-r--r-- 1 gwatts gwatts 46 Sep 13 10:57 requirements.txt
drwxr-xr-x 2 gwatts gwatts 12288 Sep 13 10:57 scripts
-rw-r--r-- 1 gwatts gwatts 3955 Sep 13 10:57 setup.py
-rw-rw-r-- 1 gwatts gwatts 228386 Sep 13 15:39 test_results.txt
drwxr-xr-x 6 gwatts gwatts 4096 Sep 13 10:57 tests
-rw-rw-r-- 1 gwatts gwatts 123 Sep 13 14:33 tmp3tbaBSN4hioE0AZ39UOy.fasta
-rw-rw-r-- 1 gwatts gwatts 124 Sep 13 15:18 tmp7puSUsbkBwT2aaLxg7L7.fasta
-rw-rw-r-- 1 gwatts gwatts 123 Sep 13 14:33 tmpc4Et9OFlrXezgrxzf0q6.fasta
-rw-rw-r-- 1 gwatts gwatts 123 Sep 13 15:02 tmpfXsTl0NIiIzBSGWP07pU.fasta
-rw-rw-r-- 1 gwatts gwatts 123 Sep 13 14:17 tmpGLf0I2ONOWmGXqlhqGS7.fasta
-rw-rw-r-- 1 gwatts gwatts 123 Sep 13 15:02 tmpk0eILDl73dRPepuSK19C.fasta
-rw-rw-r-- 1 gwatts gwatts 123 Sep 13 14:17 tmpkWB0AhM7k5yLX0Ubvr96.fasta
-rw-rw-r-- 1 gwatts gwatts 123 Sep 13 15:02 tmploC67GogMqq4nDZXo0O6.fasta
-rw-rw-r-- 1 gwatts gwatts 57 Sep 13 15:18 tmpOhcgcUwnokFs0qKzit70.fasta
-rw-rw-r-- 1 gwatts gwatts 124 Sep 13 14:33 tmpqGtEC5gagTgED0O7TFPt.fasta
-rw-rw-r-- 1 gwatts gwatts 123 Sep 13 15:18 tmptAlcy4uKAGvpPfi7Oaz6.fasta
-rw-rw-r-- 1 gwatts gwatts 57 Sep 13 14:33 tmpUHGq9GKOZvgYUDdrA0HT.fasta
-rw-rw-r-- 1 gwatts gwatts 123 Sep 13 15:18 tmpWH6afTedrM3rkex7Zoka.fasta
-rw-rw-r-- 1 gwatts gwatts 123 Sep 13 14:17 tmpYW6gKIp4htkuUTx3Xe0Y.fasta
drwxr-xr-x 3 gwatts gwatts 4096 Sep 13 10:57 web

It appears that the two runs of the test script generated two sets of temporary .fasta files along with my results file: test.results.txt which I have attached. The summary at the end of the results file is:

Result summary
==============

Unit test result summary
------------------------

Failed the following unit tests.
/mnt/datas/home/gwatts/qiime_software/qiime-1.5.0-release/tests/test_pick_otus.py
/mnt/datas/home/gwatts/qiime_software/qiime-1.5.0-release/tests/test_pycogent_backports/test_mothur.py

Failed the following unit tests, in part or whole due to missing external applications.
Depending on the QIIME features you plan to use, this may not be critical.
/mnt/datas/home/gwatts/qiime_software/qiime-1.5.0-release/tests/test_assign_taxonomy.py
/mnt/datas/home/gwatts/qiime_software/qiime-1.5.0-release/tests/test_denoiser/test_make_cluster_jobs.py
/mnt/datas/home/gwatts/qiime_software/qiime-1.5.0-release/tests/test_make_per_library_sff.py
/mnt/datas/home/gwatts/qiime_software/qiime-1.5.0-release/tests/test_pycogent_backports/test_rtax.py
/mnt/datas/home/gwatts/qiime_software/qiime-1.5.0-release/tests/test_pycogent_backports/test_usearch.py
/mnt/datas/home/gwatts/qiime_software/qiime-1.5.0-release/tests/test_trim_sff_primers.py

Basic script test result summary
--------------------------------

All basic script tests passed successfully.

Could not run script usage tests because qiime_test_data is not defined in your qiime_config.

The errors reported by the all_test.py result from:

1) Cannot find usearch

2) sfffile is not in $PATH. Is it installed? Have you added it to $PATH?

3) Unacceptable application exit status: 127, command: ['mothur'....

4) sffinfo executable not found. Is it installed and in your $PATH?

While I attempt to download and install usearch using the link you provided, could you provide information about correcting the mothur errors (by deleting the mothur I currently have installed and installing a older version?), it seems the rtax is ok, or is that the source of the sfffile and sffinfo errors?

Lastly, what do I need to change in the qiime_config file so that the qiime_test_data is defined? Is this referring to the test data available from: ftp://thebeast.colorado.edu/pub/QIIME-v1.5.0-dependencies/qiime_tutorial-v1.5.0.zip that is installed here:

gwatts@futscherlab32gb:~/qiime_software/qiime-1.5.0-release/qiime_tutorial$ pwd
/mnt/datas/home/gwatts/qiime_software/qiime-1.5.0-release/qiime_tutorial
gwatts@futscherlab32gb:~/qiime_software/qiime-1.5.0-release/qiime_tutorial$ ls

test_results.txt

GeorgeWatts

unread,

Sep 13, 2012, 8:23:36 PM9/13/12

to qiime...@googlegroups.com

So I downloaded usearch5.2.32_i86linux32, the manual's instructions are:

USEARCH is a stand-alone binary
USEARCH is distributed as one file, known as the binary file or executable file. It is completely self-contained: it does not require configuration files, environment variables, third-party libraries or other external dependencies. There is no setup script or installer because they're not needed. To install it, all you do is download or copy the binary to a directory that is accessible from the computer where you want to run the code. For convenience, you may want to rename the binary file to usearch to avoid typing long names like usearch6.0.98_i86linux32.

Missing DLL problem in Windows
See here if you get missing a vcomp100.dll error.

Execute permissions
Under Linux and OSX you must ensure that you have read and execute permissions for the binary file. If needed, use the chmod command to set the execute bit, e.g.:

chmod +x /usr/bin/usearch6.0.98_i86linux32

PATH or pathname
The binary file must be in one of the directories specified in your PATH variable, or you must type the full path to the file, e.g. for Linux:

/usr/bin/usearch6.0.98_i86linux32 -id 0.9 -cluster_fast reads.fasta -uc results.uc

Under the Windows CMD shell, path names are separated by backslashes, e.g.:

C:\MyPrograms\usearch6.2.98_win32.exe -id 0.9 -cluster_fast reads.fasta -uc results.uc

Ok, so I need to put it in one of the directories specified in the PATH variable which defined by the activate.sh script. One of the many directories specified is qiime_software, so I put it there:

gwatts@futscherlab32gb:~/qiime_software$ ls

activate.sh chimeraslayer-4.29.2010-release gg_otus-4feb2011-release mpi4py-1.2.2-release pprospector-1.0.1-release r-2.12.0-release SQLAlchemy-0.7.1-release
ampliconnoise-1.25-release clearcut-1.0.9-release infernal-1.0.2-release muscle-3.8.31-release pycogent-1.5.1-release raxml-7.3.0-release tornado-2.2.1-release
biom-format-0.9.3-release core_set_aligned.fasta.imputed ipython-0.13-release MySQL-python-1.2.3-release pynast-1.1-release rdpclassifier-2.2-release uclust-1.2.22-release
blast-2.2.22-release cytoscape-2.7.0-release lanemask_in_1s_and_0s numpy-1.5.1-release python-2.7.1-release rtax-0.982-release usearch5.2.32_i86linux32
cdbtools-10.11.2010-release drisee-1.2-release matplotlib-1.1.0-release parsinsert-1.0.4-release pyzmq-2.1.9-release setuptools-0.6c11-release vienna-1.8.4-release
cdhit-3.1-release fasttree-2.1.3-release mothur-1.25.0-release pplacer-1.1-release qiime-1.5.0-release sphinx-1.0.4-release

Next, I chmod the permissions to make it executable:

gwatts@futscherlab32gb:~/qiime_software$ chmod +x usearch5.2.32_i86linux32
gwatts@futscherlab32gb:~/qiime_software$ ls -l
total 38812
-rw-rw-r-- 1 gwatts gwatts 3985 Sep 13 10:58 activate.sh
drwxr-xr-x 22 gwatts gwatts 4096 May 19 2011 ampliconnoise-1.25-release
drwxr-xr-x 6 gwatts gwatts 4096 May 1 19:21 biom-format-0.9.3-release
drwxr-xr-x 5 gwatts gwatts 4096 Sep 27 2009 blast-2.2.22-release
drwxrwxr-x 3 gwatts gwatts 4096 Sep 13 10:50 cdbtools-10.11.2010-release
drwxrwxr-x 3 gwatts gwatts 4096 Sep 13 10:53 cdhit-3.1-release
drwxr-xr-x 7 gwatts gwatts 4096 Apr 29 2010 chimeraslayer-4.29.2010-release
drwxr-xr-x 3 gwatts gwatts 4096 Sep 13 10:50 clearcut-1.0.9-release
-rw-rw-r-- 1 gwatts gwatts 37975992 Sep 13 10:51 core_set_aligned.fasta.imputed
drwx------ 14 gwatts gwatts 4096 Sep 13 10:51 cytoscape-2.7.0-release
drwxrwxr-x 2 gwatts gwatts 4096 Apr 2 15:34 drisee-1.2-release
drwx------ 2 gwatts gwatts 4096 Sep 13 10:52 fasttree-2.1.3-release
drwxrwxr-x 6 gwatts gwatts 4096 Feb 5 2011 gg_otus-4feb2011-release
drwxrwxr-x 3 gwatts gwatts 4096 Sep 13 10:52 infernal-1.0.2-release
drwxrwxr-x 5 gwatts gwatts 4096 Sep 13 10:54 ipython-0.13-release
-rw-rw-r-- 1 gwatts gwatts 7683 Sep 13 10:50 lanemask_in_1s_and_0s
drwxrwxr-x 3 gwatts gwatts 4096 Sep 13 10:56 matplotlib-1.1.0-release
drwxr-xr-x 3 gwatts gwatts 4096 Aug 6 07:20 mothur-1.25.0-release
drwxrwxr-x 3 gwatts gwatts 4096 Sep 13 10:54 mpi4py-1.2.2-release
drwx------ 2 gwatts gwatts 4096 Sep 13 10:52 muscle-3.8.31-release
drwxrwxr-x 3 gwatts gwatts 4096 Sep 13 10:54 MySQL-python-1.2.3-release
drwxrwxr-x 4 gwatts gwatts 4096 Sep 13 10:55 numpy-1.5.1-release
drwx------ 2 gwatts gwatts 4096 Sep 13 10:53 parsinsert-1.0.4-release
drwx------ 2 gwatts gwatts 4096 May 30 17:05 pplacer-1.1-release
drwxrwxr-x 4 gwatts gwatts 4096 Sep 13 10:56 pprospector-1.0.1-release
drwxrwxr-x 9 gwatts gwatts 4096 Sep 13 10:56 pycogent-1.5.1-release
drwxrwxr-x 4 gwatts gwatts 4096 Sep 13 10:57 pynast-1.1-release
drwxrwxr-x 6 gwatts gwatts 4096 Sep 13 10:54 python-2.7.1-release
drwxrwxr-x 3 gwatts gwatts 4096 Sep 13 10:54 pyzmq-2.1.9-release
drwxrwxr-x 12 gwatts gwatts 4096 Sep 13 16:18 qiime-1.5.0-release
drwxrwxr-x 5 gwatts gwatts 4096 Sep 13 10:56 r-2.12.0-release
drwxr-xr-x 2 gwatts gwatts 4096 Sep 13 10:50 raxml-7.3.0-release
drwxr-xr-x 6 gwatts gwatts 4096 Mar 31 2010 rdpclassifier-2.2-release
drwxr-xr-x 4 gwatts gwatts 4096 Jul 5 09:31 rtax-0.982-release
drwxrwxr-x 4 gwatts gwatts 4096 Sep 13 10:54 setuptools-0.6c11-release
drwxrwxr-x 4 gwatts gwatts 4096 Sep 13 10:54 sphinx-1.0.4-release
drwxrwxr-x 3 gwatts gwatts 4096 Sep 13 10:54 SQLAlchemy-0.7.1-release
drwxrwxr-x 3 gwatts gwatts 4096 Sep 13 10:54 tornado-2.2.1-release
drwxrwxr-x 2 gwatts gwatts 4096 Sep 13 10:53 uclust-1.2.22-release
-rwxrwxr-x 1 gwatts gwatts 1551097 Sep 13 17:07 usearch5.2.32_i86linux32
drwxrwxr-x 6 gwatts gwatts 4096 Sep 13 10:52 vienna-1.8.4-release

Now I need to run a script that used usearch, turns out there is a test script, test_usearch.py here: /mnt/datas/home/gwatts/qiime_software/qiime-1.5.0-release/tests/test_pycogent_backports, so:

gwatts@futscherlab32gb:~/qiime_software/qiime-1.5.0-release/tests/test_pycogent_backports$ python test_usearch.py > test_usearch_results.txt

Results in a lot of text going by, the last portion of which is:

======================================================================
ERROR: test_usearch_sort_by_abundance_filter (__main__.UsearchTests)
Properly sorts fasta by abundance, filters low count otus

----------------------------------------------------------------------
Traceback (most recent call last):

File "test_usearch.py", line 534, in test_usearch_sort_by_abundance_filter
working_dir=self.tmp_dir)
File "/mnt/datas/home/gwatts/qiime_software/qiime-1.5.0-release/lib/qiime/pycogent_backports/usearch.py", line 492, in usearch_sort_by_abundance
app = Usearch(params, WorkingDir=working_dir, HALT_EXEC=HALT_EXEC)
File "/mnt/datas/home/gwatts/qiime_software/pycogent-1.5.1-release/lib/python2.7/site-packages/cogent/app/util.py", line 166, in __init__
self._error_on_missing_application(params)
File "/mnt/datas/home/gwatts/qiime_software/pycogent-1.5.1-release/lib/python2.7/site-packages/cogent/app/util.py", line 420, in _error_on_missing_application
% command
ApplicationNotFoundError: Cannot find usearch. Is it installed? Is it in your path?

----------------------------------------------------------------------
Ran 25 tests in 0.053s

FAILED (errors=19)

What should I try now?

Jose Navas

unread,

Sep 13, 2012, 8:28:48 PM9/13/12

to qiime...@googlegroups.com

Can you psot the result of the following command?

echo $PATH

Thanks,

Jose

2012/9/13 GeorgeWatts <gwa...@azcc.arizona.edu>

--

GeorgeWatts

unread,

Sep 13, 2012, 8:31:25 PM9/13/12

to qiime...@googlegroups.com

D'oh, I think I see where this is going already...qiime_software is not one of the PATH directories it it?

gwatts@futscherlab32gb:~/qiime_software/qiime-1.5.0-release/tests/test_pycogent_backports$ echo $PATH
/mnt/datas/home/gwatts/qiime_software/qiime-1.5.0-release/bin:/mnt/datas/home/gwatts/qiime_software/r-2.12.0-release/bin:/mnt/datas/home/gwatts/qiime_software/pynast-1.1-release/bin:/mnt/datas/home/gwatts/qiime_software/pprospector-1.0.1-release/bin:/mnt/datas/home/gwatts/qiime_software/biom-format-0.9.3-release/scripts/:/mnt/datas/home/gwatts/qiime_software/pyzmq-2.1.9-release/bin:/mnt/datas/home/gwatts/qiime_software/ipython-0.13-release/bin:/mnt/datas/home/gwatts/qiime_software/tornado-2.2.1-release/bin:/mnt/datas/home/gwatts/qiime_software/python-2.7.1-release/bin:/mnt/datas/home/gwatts/qiime_software/mothur-1.25.0-release/.:/mnt/datas/home/gwatts/qiime_software/chimeraslayer-4.29.2010-release/ChimeraSlayer:/mnt/datas/home/gwatts/qiime_software/chimeraslayer-4.29.2010-release/NAST-iEr:/mnt/datas/home/gwatts/qiime_software/uclust-1.2.22-release/.:/mnt/datas/home/gwatts/qiime_software/rdpclassifier-2.2-release/.:/mnt/datas/home/gwatts/qiime_software/cdhit-3.1-release/.:/mnt/datas/home/gwatts/qiime_software/parsinsert-1.0.4-release/.:/mnt/datas/home/gwatts/qiime_software/muscle-3.8.31-release/.:/mnt/datas/home/gwatts/qiime_software/blast-2.2.22-release/bin:/mnt/datas/home/gwatts/qiime_software/infernal-1.0.2-release/bin:/mnt/datas/home/gwatts/qiime_software/pplacer-1.1-release/.:/mnt/datas/home/gwatts/qiime_software/fasttree-2.1.3-release/.:/mnt/datas/home/gwatts/qiime_software/vienna-1.8.4-release/.:/mnt/datas/home/gwatts/qiime_software/cytoscape-2.7.0-release/.:/mnt/datas/home/gwatts/qiime_software/ampliconnoise-1.25-release/Scripts:/mnt/datas/home/gwatts/qiime_software/ampliconnoise-1.25-release/bin:/mnt/datas/home/gwatts/qiime_software/drisee-1.2-release/.:/mnt/datas/home/gwatts/qiime_software/raxml-7.3.0-release/.:/mnt/datas/home/gwatts/qiime_software/cdbtools-10.11.2010-release/.:/mnt/datas/home/gwatts/qiime_software/clearcut-1.0.9-release/.:/mnt/datas/home/gwatts/qiime_software/rtax-0.982-release/.:/mnt/datas/home/gwatts/qiime_software/qiime-1.5.0-release/bin:/mnt/datas/home/gwatts/qiime_software/r-2.12.0-release/bin:/mnt/datas/home/gwatts/qiime_software/pynast-1.1-release/bin:/mnt/datas/home/gwatts/qiime_software/pprospector-1.0.1-release/bin:/mnt/datas/home/gwatts/qiime_software/biom-format-0.9.3-release/scripts/:/mnt/datas/home/gwatts/qiime_software/pyzmq-2.1.9-release/bin:/mnt/datas/home/gwatts/qiime_software/ipython-0.13-release/bin:/mnt/datas/home/gwatts/qiime_software/tornado-2.2.1-release/bin:/mnt/datas/home/gwatts/qiime_software/python-2.7.1-release/bin:/mnt/datas/home/gwatts/qiime_software/mothur-1.25.0-release/.:/mnt/datas/home/gwatts/qiime_software/chimeraslayer-4.29.2010-release/ChimeraSlayer:/mnt/datas/home/gwatts/qiime_software/chimeraslayer-4.29.2010-release/NAST-iEr:/mnt/datas/home/gwatts/qiime_software/uclust-1.2.22-release/.:/mnt/datas/home/gwatts/qiime_software/rdpclassifier-2.2-release/.:/mnt/datas/home/gwatts/qiime_software/cdhit-3.1-release/.:/mnt/datas/home/gwatts/qiime_software/parsinsert-1.0.4-release/.:/mnt/datas/home/gwatts/qiime_software/muscle-3.8.31-release/.:/mnt/datas/home/gwatts/qiime_software/blast-2.2.22-release/bin:/mnt/datas/home/gwatts/qiime_software/infernal-1.0.2-release/bin:/mnt/datas/home/gwatts/qiime_software/pplacer-1.1-release/.:/mnt/datas/home/gwatts/qiime_software/fasttree-2.1.3-release/.:/mnt/datas/home/gwatts/qiime_software/vienna-1.8.4-release/.:/mnt/datas/home/gwatts/qiime_software/cytoscape-2.7.0-release/.:/mnt/datas/home/gwatts/qiime_software/ampliconnoise-1.25-release/Scripts:/mnt/datas/home/gwatts/qiime_software/ampliconnoise-1.25-release/bin:/mnt/datas/home/gwatts/qiime_software/drisee-1.2-release/.:/mnt/datas/home/gwatts/qiime_software/raxml-7.3.0-release/.:/mnt/datas/home/gwatts/qiime_software/cdbtools-10.11.2010-release/.:/mnt/datas/home/gwatts/qiime_software/clearcut-1.0.9-release/.:/mnt/datas/home/gwatts/qiime_software/rtax-0.982-release/.:/mnt/datas/home/gwatts/qiime_software/qiime-1.5.0-release/bin:/mnt/datas/home/gwatts/qiime_software/r-2.12.0-release/bin:/mnt/datas/home/gwatts/qiime_software/pynast-1.1-release/bin:/mnt/datas/home/gwatts/qiime_software/pprospector-1.0.1-release/bin:/mnt/datas/home/gwatts/qiime_software/biom-format-0.9.3-release/scripts/:/mnt/datas/home/gwatts/qiime_software/pyzmq-2.1.9-release/bin:/mnt/datas/home/gwatts/qiime_software/ipython-0.13-release/bin:/mnt/datas/home/gwatts/qiime_software/tornado-2.2.1-release/bin:/mnt/datas/home/gwatts/qiime_software/python-2.7.1-release/bin:/mnt/datas/home/gwatts/qiime_software/mothur-1.25.0-release/.:/mnt/datas/home/gwatts/qiime_software/chimeraslayer-4.29.2010-release/ChimeraSlayer:/mnt/datas/home/gwatts/qiime_software/chimeraslayer-4.29.2010-release/NAST-iEr:/mnt/datas/home/gwatts/qiime_software/uclust-1.2.22-release/.:/mnt/datas/home/gwatts/qiime_software/rdpclassifier-2.2-release/.:/mnt/datas/home/gwatts/qiime_software/cdhit-3.1-release/.:/mnt/datas/home/gwatts/qiime_software/parsinsert-1.0.4-release/.:/mnt/datas/home/gwatts/qiime_software/muscle-3.8.31-release/.:/mnt/datas/home/gwatts/qiime_software/blast-2.2.22-release/bin:/mnt/datas/home/gwatts/qiime_software/infernal-1.0.2-release/bin:/mnt/datas/home/gwatts/qiime_software/pplacer-1.1-release/.:/mnt/datas/home/gwatts/qiime_software/fasttree-2.1.3-release/.:/mnt/datas/home/gwatts/qiime_software/vienna-1.8.4-release/.:/mnt/datas/home/gwatts/qiime_software/cytoscape-2.7.0-release/.:/mnt/datas/home/gwatts/qiime_software/ampliconnoise-1.25-release/Scripts:/mnt/datas/home/gwatts/qiime_software/ampliconnoise-1.25-release/bin:/mnt/datas/home/gwatts/qiime_software/drisee-1.2-release/.:/mnt/datas/home/gwatts/qiime_software/raxml-7.3.0-release/.:/mnt/datas/home/gwatts/qiime_software/cdbtools-10.11.2010-release/.:/mnt/datas/home/gwatts/qiime_software/clearcut-1.0.9-release/.:/mnt/datas/home/gwatts/qiime_software/rtax-0.982-release/.:/mnt/datas/home/gwatts/qiime_software/qiime-1.5.0-release/scripts/:/mnt/datas/home/gwatts/qiime_software/qiime-1.5.0-release/bin:/mnt/datas/home/gwatts/qiime_software/r-2.12.0-release/bin:/mnt/datas/home/gwatts/qiime_software/pynast-1.1-release/bin:/mnt/datas/home/gwatts/qiime_software/pprospector-1.0.1-release/bin:/mnt/datas/home/gwatts/qiime_software/biom-format-0.9.3-release/scripts/:/mnt/datas/home/gwatts/qiime_software/pyzmq-2.1.9-release/bin:/mnt/datas/home/gwatts/qiime_software/ipython-0.13-release/bin:/mnt/datas/home/gwatts/qiime_software/tornado-2.2.1-release/bin:/mnt/datas/home/gwatts/qiime_software/python-2.7.1-release/bin:/mnt/datas/home/gwatts/qiime_software/mothur-1.25.0-release/.:/mnt/datas/home/gwatts/qiime_software/chimeraslayer-4.29.2010-release/ChimeraSlayer:/mnt/datas/home/gwatts/qiime_software/chimeraslayer-4.29.2010-release/NAST-iEr:/mnt/datas/home/gwatts/qiime_software/uclust-1.2.22-release/.:/mnt/datas/home/gwatts/qiime_software/rdpclassifier-2.2-release/.:/mnt/datas/home/gwatts/qiime_software/cdhit-3.1-release/.:/mnt/datas/home/gwatts/qiime_software/parsinsert-1.0.4-release/.:/mnt/datas/home/gwatts/qiime_software/muscle-3.8.31-release/.:/mnt/datas/home/gwatts/qiime_software/blast-2.2.22-release/bin:/mnt/datas/home/gwatts/qiime_software/infernal-1.0.2-release/bin:/mnt/datas/home/gwatts/qiime_software/pplacer-1.1-release/.:/mnt/datas/home/gwatts/qiime_software/fasttree-2.1.3-release/.:/mnt/datas/home/gwatts/qiime_software/vienna-1.8.4-release/.:/mnt/datas/home/gwatts/qiime_software/cytoscape-2.7.0-release/.:/mnt/datas/home/gwatts/qiime_software/ampliconnoise-1.25-release/Scripts:/mnt/datas/home/gwatts/qiime_software/ampliconnoise-1.25-release/bin:/mnt/datas/home/gwatts/qiime_software/drisee-1.2-release/.:/mnt/datas/home/gwatts/qiime_software/raxml-7.3.0-release/.:/mnt/datas/home/gwatts/qiime_software/cdbtools-10.11.2010-release/.:/mnt/datas/home/gwatts/qiime_software/clearcut-1.0.9-release/.:/mnt/datas/home/gwatts/qiime_software/rtax-0.982-release/.:/mnt/datas/home/gwatts/qiime_software/qiime-1.5.0-release/scripts/:/mnt/datas/home/gwatts/src/Qiime/scripts/:/mnt/datas/home/gwatts/src/biom-format/scripts/:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games
gwatts@futscherlab32gb:~/qiime_software/qiime-1.5.0-release/tests/test_pycogent_backports$

Jose Navas

unread,

Sep 13, 2012, 8:34:05 PM9/13/12

to qiime...@googlegroups.com

Yes, that's it. You can move the usearch executable inside the /mnt/datas/home/gwatts/qiime_software/qiime-1.5.0-release/bin folder.

Then, in order to test it, you can execute:

usearch

directly from the command line.

Jose N.

2012/9/13 GeorgeWatts <gwa...@azcc.arizona.edu>

--

GeorgeWatts

unread,

Sep 13, 2012, 8:38:45 PM9/13/12

to qiime...@googlegroups.com

gwatts@futscherlab32gb:~/qiime_software/qiime-1.5.0-release/bin$ ls
add_qiime_labels.py count_seqs.py make_distance_boxplots.py parallel_assign_taxonomy_rdp.py single_rarefaction.py
add_taxa.py denoiser_preprocess.py make_distance_comparison_plots.py parallel_beta_diversity.py sort_otu_table.py
adjust_seq_orientation.py denoiser.py make_distance_histograms.py parallel_blast.py split_fasta_on_sample_ids.py
align_seqs.py denoiser_worker.py make_fastq.py parallel_identify_chimeric_seqs.py split_libraries_fastq.py
alpha_diversity.py denoise_wrapper.py make_library_id_lists.py parallel_multiple_rarefactions.py split_libraries.py
alpha_rarefaction.py dissimilarity_mtx_stats.py make_otu_heatmap_html.py parallel_pick_otus_blast.py split_otu_table_by_taxonomy.py
ampliconnoise.py distance_matrix_from_mapping.py make_otu_heatmap.py parallel_pick_otus_uclust_ref.py split_otu_table.py
assign_taxonomy.py ec2Clean make_otu_network.py per_library_stats.py start_parallel_jobs.py
beta_diversity.py ec2Dispatch make_otu_table.py pick_otus.py start_parallel_jobs_sc.py
beta_diversity_through_plots.py exclude_seqs_by_blast.py make_per_library_sff.py pick_otus_through_otu_table.py start_parallel_jobs_torque.py
beta_significance.py extract_seqs_by_sample_id.py make_phylogeny.py pick_reference_otus_through_otu_table.py submit_to_mgrast.py
blast_wrapper.py filter_alignment.py make_prefs_file.py pick_rep_set.py subsample_fasta.py
categorized_dist_scatterplot.py filter_distance_matrix.py make_qiime_py_file.py pick_subsampled_reference_otus_through_otu_table.py summarize_otu_by_cat.py
check_id_map.py filter_fasta.py make_qiime_rst_file.py plot_rank_abundance_graph.py summarize_taxa.py
clean_raxml_parsimony_tree.py filter_otus_by_sample.py make_rarefaction_plots.py plot_semivariogram.py summarize_taxa_through_plots.py
cluster_quality.py filter_otus_from_otu_table.py make_tep.py plot_taxa_summary.py supervised_learning.py
collate_alpha.py filter_samples_from_otu_table.py merge_mapping_files.py poller_example.py transform_coordinate_matrices.py
compare_3d_plots.py filter_tree.py merge_otu_maps.py poller.py tree_compare.py
compare_alpha_diversity.py fix_arb_fasta.py merge_otu_tables.py principal_coordinates.py trflp_file_to_otu_table.py
compare_categories.py identify_chimeric_seqs.py multiple_rarefactions_even_depth.py print_qiime_config.py trim_sff_primers.py
compare_distance_matrices.py identify_missing_files.py multiple_rarefactions.py process_iseq.py truncate_fasta_qual_files.py
conditional_uncovered_probability.py inflate_denoiser_output.py neighbor_joining.py process_qseq.py truncate_reverse_primer.py
consensus_tree.py insert_seqs_into_tree.py nmds.py process_sff.py unweight_fasta.py
convert_fastaqual_to_fastq.py jackknifed_beta_diversity.py otu_category_significance.py quality_scores_plot.py upgma_cluster.py
convert_otu_table_to_unifrac_sample_mapping.py make_2d_plots.py parallel_align_seqs_pynast.py relatedness.py usearch5.2.32_i86linux32
convert_unifrac_sample_mapping_to_otu_table.py make_3d_plots.py parallel_alpha_diversity.py shared_phylotypes.py validate_demultiplexed_fasta.py
core_qiime_analyses.py make_bootstrapped_tree.py parallel_assign_taxonomy_blast.py simsam.py
gwatts@futscherlab32gb:~/qiime_software/qiime-1.5.0-release/bin$ usearch
No command 'usearch' found, did you mean:
Command 'search' from package 'sphinxsearch' (universe)
Command 'rsearch' from package '389-ds-base' (universe)
Command 'ausearch' from package 'auditd' (universe)
usearch: command not found
gwatts@futscherlab32gb:~/qiime_software/qiime-1.5.0-release/bin$

Do I need to rename usearch5.2.32_i86linux32 to just usearch?

Jose Navas

unread,

Sep 13, 2012, 8:40:00 PM9/13/12

to qiime...@googlegroups.com

Yes. I'm sorry, I miss to tell you this :-)

2012/9/13 GeorgeWatts <gwa...@azcc.arizona.edu>

--

GeorgeWatts

unread,

Sep 13, 2012, 8:45:48 PM9/13/12

to qiime...@googlegroups.com

That did it:

gwatts@futscherlab32gb:~/qiime_software/qiime-1.5.0-release/bin$ mv usearch5.2.32_i86linux32 usearch
gwatts@futscherlab32gb:~/qiime_software/qiime-1.5.0-release/bin$ usearch

USEARCH 5.2.32
(C) Copyright 2010-11 Robert C. Edgar, all rights reserved.
http://drive5.com/usearch

Licensed to: redacted

Common commands
===============
Clustering de novo (default is global alignment):
usearch -cluster seqs.sorted.fasta -uc results.uc -id 0.97 [-usersort]
Specify -usersort if input is not sorted by length.
Not recommended for OTU clustering. See manual.

Database search (default is local alignment):
usearch -query q.fasta -evalue 0.01 -blast6out results.b6
-db db.fasta | -udb db.udb [-threads n] | -wdb db.wdb

Search + clustering of seqs that don't match (default is global alignment):
usearch -cluster seqs.sorted.fasta -db db.fasta -id 0.97 [-uc results.uc]
[-seedsout seeds.fasta] [-consout cons.fasta]

Create udb or wdb database index:
usearch -makeudb db.fasta -output db.udb
usearch -makewdb db.fasta -output db.wdb

Dereplication, removing identical full-length sequences (does not search reverse strand):
usearch -derep_fullseq -cluster input.fasta -seedsout nr.fasta [-bithash] [-sizeout]

Dereplication, removing identical sub-sequences:
usearch -derep_subseq -cluster input.fasta -seedsout nr.fasta
-w 32 -slots 40000003 [-sizeout]

Chimera detection (UCHIME ref. db. mode):
usearch -uchime q.fasta [-db db.fasta] [-chimeras ch.fasta]
[-nonchimeras good.fasta] [-uchimeout results.uch] [-uchimealns results.alns]

Chimera detection (UCHIME de novo mode):
usearch -uchime amplicons.fasta [-chimeras ch.fasta] [-nonchimeras good.fasta]
[-uchimeout results.uch] [-uchimealns results.alns]
Input is estimated amplicons with integer abundances specified using ";size=N".

Sort sequences by length:
usearch -sort seqs.fasta -output seqs.sorted.fasta
usearch -mergesort seqs.fasta -output seqs.sorted.fasta [-split S]
Use -mergesort if too big for -sort. S is partition size in Mb, default 1000.0.

Sort sequences by cluster size/abundance specified by ";size=N" in label:
usearch -sortsize seqs.fasta -output seqs.sorted.fasta [-minsize n]

Output files
============
All formats are supported for clustering and searching.
-uc file UCLUST format, tab-separated.
-blastout file Human-readable verbose format similar to BLAST.
-blast6out file Tab-separated, same as -outfmt 6 or -m8 option of NCBI BLAST.
-userout file Tab-separated, fields specified by -userfields (see manual).
-seedsout file FASTA file with cluster seeds, i.e. non-redundant version of input.
-consout file FASTA file with consensus sequence for each cluster.
-fastapairs file FASTA file with pair-wise alignments.

Search termination
==================
-maxaccepts N Max accepted targets, 0=ignore, default 1.
-maxrejects N Max rejected targets, 0=ignore, default 32.
-[no]usort [Do not] test database sequences in U-sorted order. If -nousort is
specified, the entire database is searched and termination options
are ignored. Default is -usort.

Accept/reject criteria
======================
Criteria are combined with AND.

-id F
Minimum identity, as a value 0.0 to 1.0, meaning 0% to 100% identity. No default value.
The -iddef option specifies definiton of identity (see manual).

-evalue E
Maximum E-value. Local alignments only. No default value.

-query[aln]fract F
Minimum fraction of the query sequence covered by alignment. Default 0.0.

-target[aln]fract F
Minimum fraction of the target sequence covered by alignment. Default 0.0.

-idprefix n / -idsuffix n
First (last) n letters of the query must be identical to the target. Default 0.

-leftjust / -rightjust
Left (right) terminal gaps cause reject. Recommended to use -idprefix if you
use -leftjust or -idsuffix if you use -rightjust.

Alignment style
===============
-global Default if -cluster is specified.
-local Default if -query is specified.

Compressed index
================
-slots n Size of compressed index table. Should be prime, e.g. 40000003.
Should also specify -w, typical is -w 16 to 32.

Misc.
=====
-quiet Do not write progress messages to standard error.
-log filename Write log file with information about parameters and performance.
-version Show program version number and exit.
-help This help.

See manual for more options.
gwatts@futscherlab32gb:~/qiime_software/qiime-1.5.0-release/bin$

Now, the mothur errors and the undefined test data (see my post on 4:04PM today above).

Jose Navas

unread,

Sep 13, 2012, 8:53:02 PM9/13/12

to qiime...@googlegroups.com

I'm working on the mothur and test data stuff. I'll get back to you ASAP.

Jose N.

2012/9/13 GeorgeWatts <gwa...@azcc.arizona.edu>

That did it:

--

Jose Navas

unread,

Sep 14, 2012, 2:04:50 AM9/14/12

to qiime...@googlegroups.com

George,

The app-deploy installs the wrong version of mothur. We will fix this for the next release. In order to install the supported version of mothur follow these steps:

1 - Change to the folder which contains mothur:
cd ~/qiime_software/mothur-1.25.0-release/

2 - Remove the contents of the folder:

rm *

3 - Get the supported version of mothur:

wget http://www.mothur.org/w/images/6/6d/Mothur.1.25.0.zip

4 - Extract the contents of the zipped file:

unzip Mothur.1.25.0.zip

5 - Change to the Mothur.source folder:

cd Mothur.source

6 - Open the 'makefile' file with a text editor and comment the line 31 (by adding a # at the beginning) and uncomment the line 40 (by removing the # at the beginning). Save and close the file.

7 - Then build mothur using the make command:

make

It will take some time, be patient. Sometimes, the build fails with this error: '/usr/bin/ld: cannot find -lncurses'. In this case, you should contact with your system administration and ask him to execute: 'sudo apt-get install libncurses5-dev '. Then, rerun the make command.

8 - Copy the mothur and uchime executables one folder up:

cp mothur ../

cp uchime ../

That's it.

You shouldn't be worried about the undefined test data. This test data are extra tests used by developers.

Let me know if it works!!

2012/9/13 Jose Navas <josenav...@gmail.com>

--
Jose Navas

Jai Ram Rideout

unread,

Sep 14, 2012, 12:14:03 PM9/14/12

to qiime...@googlegroups.com

Hi George,

Sorry to weigh in here a little late. In answer to your questions
about the sfffile and sffinfo failures: these are proprietary tools
that must be obtained from Roche/454, so we cannot bundle them with
the app-deploy script. You don't need to be worried about these
failures.

Glad to see the rest of your install is going well!

-Jai

> --
>
>
>

GeorgeWatts

unread,

Sep 14, 2012, 4:14:29 PM9/14/12

to qiime...@googlegroups.com

I did all of the following without issue:

1 - Change to the folder which contains mothur: cd ~/qiime_software/mothur-1.25.0-release/
2 - Remove the contents of the folder: rm *
3 - Get the supported version of mothur: wget http://www.mothur.org/w/images/6/6d/Mothur.1.25.0.zip
4 - Extract the contents of the zipped file: unzip Mothur.1.25.0.zip
5 - Change to the Mothur.source folder: cd Mothur.source
6 - Open the 'makefile' file with a text editor and comment the line 31 (by adding a # at the beginning) and uncomment the line 40 (by removing the # at the beginning). Save and close the file.
7 - Then build mothur using the make command: make

8 - Copy the mothur and uchime executables one folder up: cp mothur ../ and cp uchime ../

I then re-ran all_tests.py in the tests directory. The results at attached. The only remaining errors are related to qsub not being installed and sfffile and sffinfo. Jai has explained the sfffile and sffinfo errors are nothing I need to worry about in the post below and the unit tests that failed due to not having qsub I don't think I need. Finally, as you explained, I don't need to worry about theundefined test data.

So, it looks like I have a working install...now on to the actual analysis. Thanks for all the help!

friday_all_tests.txt

Jose Navas

unread,

Sep 14, 2012, 4:23:35 PM9/14/12

to qiime...@googlegroups.com

Glad it worked!!

Jose N.

2012/9/14 GeorgeWatts <gwa...@azcc.arizona.edu>

--

Richard Agans

unread,

Oct 13, 2015, 12:49:14 PM10/13/15

to Qiime Forum

Hello,
I have been following with this topic attempting to get Qiime to install on Ubuntu 12.04 (after unsuccessfully attempting with Ubuntu 15) as well. I have been unable to perform a minimal install as explained here http://qiime.org/install/install.html#native-base, for after I follow these steps I get "Command not found" when entering print_qiime_config.py -t.
I have also been unable to utilize the qiime_deploy as I am unable to connect to thebeast.colorado.edu/... I did notice today on the Forum that it appears this is no longer supported so may explain my troubles with that.

I have, so far, been able to replicate everything from this thread up until print_qiime_config.py -t again. When I enter this command it returns the following error:
ra@Serenity:~$ print_qiime_config.py -t #My home directory containing src with the installed directories

Traceback (most recent call last):

File "/home/ra/src/Qiime/scripts/print_qiime_config.py", line 15, in <module>
from qiime.util import make_option
File "/home/ra/src/Qiime/qiime/util.py", line 42, in <module>
from biom.table import DenseTable

One thing I do notice is in my PYTHONPATH, I don't see any python-code directory within the biom-format and am wondering if something has not been installed? Again, I have followed this thread from start till successful install (I don't use mothur so didn't read through that section much), and the only difference between my approach and the one here is that my HOME directory is different. It has been more than a few days and a couple clean installs of Ubuntu to get me to this thread and my hope is you will be my salvation!

Antonio González Peña

unread,

Oct 16, 2015, 10:29:24 AM10/16/15

to Qiime Forum

Sorry to hear about these issues. Have you had any luck?

Note that thebeast is offline and the new server is: ftp://ftp.microbio.me/pub/

> --
>
> ---
> You received this message because you are subscribed to the Google Groups
> "Qiime Forum" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to qiime-forum...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

--
Antonio

Richard Agans

unread,

Oct 16, 2015, 10:37:49 AM10/16/15

to qiime...@googlegroups.com

Dear Antonio,
I guess you could say I've had 'slight' success.
The success I have had is running Bio-linux on the same system. I am
able to run some of the qiime commands, however the output from them
does not appear to work with native qiime commands (we have a Linux
system with qiime in our office, but some of us want to be able to work
at home as well); so I still want to try and get a native qiime
operating on my own system.
I have yet to get Qiime to install on Ubuntu 14, but I also was
unaware of the new server. I imagine I can run the base install
replacing the ftp server for the beat server? If so I will give it
another try. Should I try and remove everything that I have already
tried to install? Should I do a clean install of Ubuntu to be sure?

Best regards,
-Richard

-Richard Agans, MSc
PhD Candidate
Dr. Paliy, Advisor
Wright State University

---
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus

Antonio González Peña

unread,

Oct 16, 2015, 11:16:24 AM10/16/15

to Qiime Forum

Substituting the server should do the trick. I don't have a strong
opinion about a clean install. In general, after installation all your
paths in your PATH and PYTHON_PATH should be updated. Note that we
have some suggestions about install here:
http://qiime.org/install/install.html#prepare-your-build-environment
and here: https://github.com/qiime/qiime-deploy#setting-up-qiime-deploy-on-ubuntu.
Let us know how it goes.

Antonio

Richard Agans

unread,

Oct 16, 2015, 1:16:32 PM10/16/15

to qiime...@googlegroups.com

Antonio,
Perhaps there is something simple but crucial I am overlooking here.
From your previous message I did the following:
#Using the directions from qiime-deploy
sudo pico /etc/apt/sources.list #Add the deb cran.rstudio for Ubuntu trusty
#saved file
sudo apt-get -y update # so far so good no problems
sudo apt-get --force-yes -y install python-dev libncurses5-dev

libssl-dev libzmq-dev libgsl0-dev openjdk-6-jdk libxml2 libxslt1.1

libxslt1-dev ant git subversion build-essential zlib1g-dev libpng12-dev

libfreetype6-dev mpich2 libreadline-dev gfortran unzip libmysqlclient18

libmysqlclient-dev ghc sqlite3 libsqlite3-dev libc6-i386 libbz2-dev
tcl-dev tk-dev r-base r-base-dev libatlas-dev libatlas-base-dev
liblapack-dev swig libhdf5-serial-dev # install qiime-deploy
pre-requisites
sudo apt-get install build-essential python-dev python-pip #setting up
the build environment, still no issues
python --version # returns Python 2.7.6 which appears to be compatible
since it's <3.0.0
pip install numpy #no issues yet
pip install qiime #runs through seemingly correct, no error or warning
messages that I can tell
print_qiime_config.py -t
#returns Traceback (most recent call last):

File "/home/ra/src/Qiime/scripts/print_qiime_config.py",
line 15, in <module>
from qiime.util import make_option
File "/home/ra/src/Qiime/qiime/util.py", line 42, in
<module>
from biom.table import DenseTable

ImportError: cannot import name DenseTable

#OK, maybe need to run installs as sudo
sudo pip install numpy
#returns The directory '/home/ra/.cache/pip/http' or its parent
directory is not owned by the current user and the cache has been
disabled. Please check the permission and owner of that directory. If
executing pip with sudo, you may want sudo's -H flag.
The directory '/home/ra/.cache/pip' or its parent directory is not owned
by the current user and the cache has been disabled. Please check the
permission and owner of that directory. If executing pip with sudo, you
may want sudo's -H flag.
Requirement already satisfied (use --upgrade to upgrade): numpy in
/usr/local/lib/python2.7/dist-packages

Is it possible that the dual boot I have set up with Bio-linux is
causing this issue? I have seen the traceback previously when trying to
set up qiime-deploy. Given that everyone else in the world is able to
get Qiime to install it leads me to think that I'm just missing
something simple due to my lack of experience with Linux systems.

The only other thing I can think to try (at least right now) is to
follow the guide from qiime.org/1.5.0/install/ubuntu_install.html since
that actually has a link to a server, so I would update the server and
go from there.

Again, my deepest thanks for your help,
-Richard

Antonio González Peña

unread,

Oct 16, 2015, 1:43:45 PM10/16/15

to Qiime Forum

Not the dual boot. The issue is how your paths are set and that it's
getting confused with older installs. Could you try doing:

mv /home/ra/src /home/ra/src_backup
print_qiime_config.py -t

Antonio

Richard Agans

unread,

Oct 16, 2015, 1:47:14 PM10/16/15

to qiime...@googlegroups.com

I get the message "bash:
/home/ra/src/Qiime/scripts/print_qiime_config.py: No such file or
directory".

Antonio González Peña

unread,

Oct 16, 2015, 1:56:38 PM10/16/15

to Qiime Forum

Could you try running the `print_qiime_config.py` on a new terminal?
If you get the same result, could you close all terminals and try:
`pip install -U qiime` and then try print_qiime_config.py? If you get
the same result could you send the output of `echo $PATH`?

Thanks

Antonio

Richard Agans

unread,

Oct 16, 2015, 2:53:43 PM10/16/15

to qiime...@googlegroups.com

Antonio,
Somehow I lost all terminal access with all this. I have reinstalled
12.04 and am currently running through the installation indicated from
qiime.org/1.5.0/install/ubuntu.html and will let you know what the end
result of that is.

Antonio González Peña

unread,

Oct 16, 2015, 3:30:49 PM10/16/15

to Qiime Forum

Thanks for letting me know. Suggest sticking to only one set of
instructions, if those don't work let us know and we will try to help.
Good luck.

Antonio

Richard Agans

unread,

Oct 19, 2015, 5:10:07 AM10/19/15

to qiime...@googlegroups.com

When I ran the pip install I did see a message that packages failed to deploy and qiime was among them. Below is the output from echo $PATH.

ra@Serenity:~$ echo$PATH
bash: echo/home/ra/qiime_software/chimeraslayer-4.29.2010-release/ChimeraSlayer:/home/ra/qiime_software/chimeraslayer-4.29.2010-release/NAST-iEr:/home/ra/qiime_software/uclust-1.2.22-release/.:/home/ra/qiime_software/r-2.12.0-release/bin:/home/ra/qiime_software/rdpclassifier-2.2-release/.:/home/ra/qiime_software/muscle-3.8.31-release/.:/home/ra/qiime_software/infernal-1.0.2-release/bin:/home/ra/qiime_software/parsinsert-1.0.4-release/.:/home/ra/qiime_software/blast-2.2.22-release/bin:/home/ra/qiime_software/fasttree-2.1.3-release/.:/home/ra/qiime_software/cytoscape-2.7.0-release/.:/home/ra/qiime_software/mothur-1.25.0-release/.:/home/ra/qiime_software/drisee-1.2-release/.:/home/ra/qiime_software/ampliconnoise-1.25-release/Scripts:/home/ra/qiime_software/ampliconnoise-1.25-release/bin:/home/ra/qiime_software/clearcut-1.0.9-release/.:/home/ra/qiime_software/cdbtools-10.11.2010-release/.:/usr/lib/lightdm/lightdm:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games: No such file or directory

Antonio

--

---
You received this message because you are subscribed to a topic in the Google Groups "Qiime Forum" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/qiime-forum/-fD44oQg4wE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to qiime-forum...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Antonio González Peña

unread,

Oct 21, 2015, 9:14:12 AM10/21/15

to Qiime Forum

Do you have the full error and can you send it?

Richard Agans

unread,

Oct 26, 2015, 7:42:32 PM10/26/15

to qiime...@googlegroups.com

DEPLOYMENT SUMMARY

Packages deployed successfully:

data-lanemask, cdbtools, clearcut, data-core, ampliconnoise, drisee, cytoscape, mothur, gg_otus, fasttree, blast, parsinsert, muscle, rdpclassifier, chimeraslayer, uclust, infernal, r

Packages skipped (assumed successful):

Packages failed to deploy:

rtax, vienna, pplacer, raxml, python, numpy, pyzmq, SQLAlchemy, tornado, matplotlib, ipython, setuptools, mpi4py, MySQL-python, biom-format, pycogent, sphinx, pynast, pprospector, qiime, cdhit

ra@Serenity:~$ cd qiime_software/

ra@Serenity:~/qiime_software$ print_qiime_config.py

print_qiime_config.py: command not found

ra@Serenity:~/qiime_software$

Jai Ram Rideout

unread,

Oct 28, 2015, 1:01:41 PM10/28/15

to qiime...@googlegroups.com

Hi Richard,

I'll help you get QIIME installed on your system but we'll need to take a few steps back. It sounds like you're wanting a native QIIME install on your Ubuntu machine. First let's get the minimal (base) QIIME install working and then we'll work on getting the other dependencies installed via qiime-deploy.

1. What version of Ubuntu are you installing QIIME on?

2. Delete any previous attempts at installing QIIME. Having these around can make this process very hard to debug so let's start from scratch.

3. Send me the output from running the following commands:

echo $PATH

echo $PYTHONPATH

4. Send me your ~/.bashrc file as an attachment (or send via private message/email if you don't want it public).

Jai

Richard Agans

unread,

Oct 28, 2015, 1:40:22 PM10/28/15

to qiime...@googlegroups.com

Jai,
Thanks a lot! I won't be at my machine for about 4-5 hours and then can send you the information you requested. For now I can answer the following:
1. I am installing on an Ubuntu 12.04 system (previously had Window OS, but I did a clean install for Ubuntu)
2. How do I delete previous Ubuntu tries?
3. I will try to output those to a txt file (using "echo PATH > filename.txt)
4. Where is the bashrc file usually located physically?

Many thanks,
-Richard

This email has been checked for viruses by Avast antivirus software.

www.avast.com

Richard Agans

unread,

Oct 29, 2015, 9:08:35 AM10/29/15

to qiime...@googlegroups.com

Jai,

I have attached the PATH output and copied the bashrc file into a text document. When I run echo $PYTHONPATH I get no return value, just an empty line.

I still have not figured out how to delete all the previous qiime installation packages/files, should I just do a clean install of the Ubuntu OS?

Thanks,

-Richard

bashrc.txt

path.txt

Jai Ram Rideout

unread,

Oct 29, 2015, 3:06:09 PM10/29/15

to qiime...@googlegroups.com

Hi Richard,

1. Sounds good re: Ubuntu version.

2. It depends on how you were trying to install QIIME previously. You were using qiime-deploy, that tool will create a directory containing your QIIME install, so you would want to delete those directories (likely just /home/ra/qiime_software). If you used "sudo apt-get install qiime", you can remove that with "sudo apt-get remove qiime". If you installed QIIME with "pip install qiime", run "pip uninstall qiime". You could also start over with a fresh Ubuntu install (we've tested mostly on Ubuntu 11.10, 12.04, and 14.04).

3/4. Re: PYTHONPATH, that's what I was hoping to see. Can you delete the last line in your .bashrc file? That was added by qiime-deploy.

Note that it may take awhile to get you set up with a complete native QIIME install (it's not trivial). Another option is using the QIIME VirtualBox on your home computer. That way you'll be able to use your operating system of choice (e.g., Windows) and run the QIIME VirtualBox within that. It comes with a nearly-complete QIIME install (just missing usearch due to licensing restrictions).

Let me know how you'd like to proceed,

Jai

Richard Agans

unread,

Oct 29, 2015, 3:18:58 PM10/29/15

to qiime...@googlegroups.com, jai.r...@gmail.com

Jai,
1. Great!
2. I have deleted the qiime_software directory, and running the other uninstall commands (just to be save), so we are good there now.
3/4. I have removed the activation line added by qiime-deploy.

I would definitely prefer to stay away from a virtual box if possible.

Best,
-Richard

Jai Ram Rideout

unread,

Oct 30, 2015, 5:34:37 PM10/30/15

to qiime...@googlegroups.com

Ok sounds good. Close your terminal and open a new one (necessary since you modified your .bashrc file). Next run:

sudo apt-get install build-essential python-dev python-pip

After that, send me the output from running these commands:

python --version

echo $PATH

echo $PYTHONPATH

Jai

Richard Agans

unread,

Nov 2, 2015, 8:27:50 AM11/2/15

to qiime...@googlegroups.com

Jai,

Thank you, I have included the terminal output below. As you will see running %PYTHONPATH does not appear to return anything.

ra@Serenity:~$ sudo apt-get install build-essential python-dev python-pip

[sudo] password for ra:

Reading package lists... Done

Building dependency tree

Reading state information... Done

build-essential is already the newest version.

The following packages were automatically installed and are no longer required:

libpgm-5.1-0 libzmq1

Use 'apt-get autoremove' to remove them.

The following extra packages will be installed:

libexpat1 libexpat1-dev libpython2.7 libssl-dev python-setuptools python2.7

python2.7-dev python2.7-minimal

Suggested packages:

python2.7-doc binfmt-support

The following NEW packages will be installed:

libexpat1-dev libssl-dev python-dev python-pip python-setuptools

python2.7-dev

The following packages will be upgraded:

libexpat1 libpython2.7 python2.7 python2.7-minimal

4 upgraded, 6 newly installed, 0 to remove and 260 not upgraded.

Need to get 36.0 MB/37.6 MB of archives.

After this operation, 48.0 MB of additional disk space will be used.

Do you want to continue [Y/n]?

Get:1 http://us.archive.ubuntu.com/ubuntu/ precise-updates/main libexpat1 amd64 2.0.1-7.2ubuntu1.2 [131 kB]

Get:2 http://us.archive.ubuntu.com/ubuntu/ precise-updates/main libpython2.7 amd64 2.7.3-0ubuntu3.8 [1,188 kB]

Get:3 http://us.archive.ubuntu.com/ubuntu/ precise-updates/main python2.7 amd64 2.7.3-0ubuntu3.8 [2,676 kB]

Get:4 http://us.archive.ubuntu.com/ubuntu/ precise-updates/main python2.7-minimal amd64 2.7.3-0ubuntu3.8 [1,743 kB]

Get:5 http://us.archive.ubuntu.com/ubuntu/ precise-updates/main libexpat1-dev amd64 2.0.1-7.2ubuntu1.2 [215 kB]

Get:6 http://us.archive.ubuntu.com/ubuntu/ precise-updates/main python2.7-dev amd64 2.7.3-0ubuntu3.8 [29.5 MB]

Get:7 http://us.archive.ubuntu.com/ubuntu/ precise-updates/main python-dev amd64 2.7.3-0ubuntu2.2 [1,090 B]

Get:8 http://us.archive.ubuntu.com/ubuntu/ precise/main python-setuptools all 0.6.24-1ubuntu1 [441 kB]

Get:9 http://us.archive.ubuntu.com/ubuntu/ precise/universe python-pip all 1.0-1build1 [95.1 kB]

Fetched 36.0 MB in 18s (1,907 kB/s)

(Reading database ... 151841 files and directories currently installed.)

Preparing to replace libexpat1 2.0.1-7.2ubuntu1.1 (using .../libexpat1_2.0.1-7.2ubuntu1.2_amd64.deb) ...

Unpacking replacement libexpat1 ...

Preparing to replace libpython2.7 2.7.3-0ubuntu3.5 (using .../libpython2.7_2.7.3-0ubuntu3.8_amd64.deb) ...

Unpacking replacement libpython2.7 ...

Preparing to replace python2.7 2.7.3-0ubuntu3.5 (using .../python2.7_2.7.3-0ubuntu3.8_amd64.deb) ...

Unpacking replacement python2.7 ...

Preparing to replace python2.7-minimal 2.7.3-0ubuntu3.5 (using .../python2.7-minimal_2.7.3-0ubuntu3.8_amd64.deb) ...

Unpacking replacement python2.7-minimal ...

Processing triggers for man-db ...

Processing triggers for desktop-file-utils ...

Processing triggers for bamfdaemon ...

Rebuilding /usr/share/applications/bamf.index...

Processing triggers for gnome-menus ...

Setting up python2.7-minimal (2.7.3-0ubuntu3.8) ...

Selecting previously unselected package libexpat1-dev.

(Reading database ... 151841 files and directories currently installed.)

Unpacking libexpat1-dev (from .../libexpat1-dev_2.0.1-7.2ubuntu1.2_amd64.deb) ...

Selecting previously unselected package libssl-dev.

Unpacking libssl-dev (from .../libssl-dev_1.0.1-4ubuntu5.31_amd64.deb) ...

Selecting previously unselected package python2.7-dev.

Unpacking python2.7-dev (from .../python2.7-dev_2.7.3-0ubuntu3.8_amd64.deb) ...

Selecting previously unselected package python-dev.

Unpacking python-dev (from .../python-dev_2.7.3-0ubuntu2.2_amd64.deb) ...

Selecting previously unselected package python-setuptools.

Unpacking python-setuptools (from .../python-setuptools_0.6.24-1ubuntu1_all.deb) ...

Selecting previously unselected package python-pip.

Unpacking python-pip (from .../python-pip_1.0-1build1_all.deb) ...

Processing triggers for doc-base ...

Processing 1 added doc-base file...

Processing triggers for man-db ...

Setting up libexpat1 (2.0.1-7.2ubuntu1.2) ...

Setting up python2.7 (2.7.3-0ubuntu3.8) ...

Setting up libpython2.7 (2.7.3-0ubuntu3.8) ...

Setting up libexpat1-dev (2.0.1-7.2ubuntu1.2) ...

Setting up libssl-dev (1.0.1-4ubuntu5.31) ...

Setting up python2.7-dev (2.7.3-0ubuntu3.8) ...

Setting up python-dev (2.7.3-0ubuntu2.2) ...

Setting up python-setuptools (0.6.24-1ubuntu1) ...

Setting up python-pip (1.0-1build1) ...

Processing triggers for libc-bin ...

ldconfig deferred processing now taking place

ra@Serenity:~$ python --version

Python 2.7.3

ra@Serenity:~$ echo $PATH

/usr/lib/lightdm/lightdm:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games

ra@Serenity:~$ $PYTHONPATH

ra@Serenity:~$

Best,

-Richard

Kyle Bittinger

unread,

Nov 2, 2015, 1:45:58 PM11/2/15

to Qiime Forum

Richard, I'll be taking over as QIIME forum master for this week. It looks like you've successfully uninstalled QIIME and are ready to start with a base install. Is that correct?

Best,

Kyle

Richard Agans

unread,

Nov 2, 2015, 1:50:04 PM11/2/15

to qiime...@googlegroups.com

Kyle,
Awesome, I think so. I sent Jai the results from the apt-get build essential, python-dev, and PATH/PYTHONPATH, etc. Did you also get those? If not should be able to send them to you as well. At this point, I'm completely lost with this.

Best,
-Richard

--

---
You received this message because you are subscribed to a topic in the Google Groups "Qiime Forum" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/qiime-forum/-fD44oQg4wE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to qiime-forum...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Kyle Bittinger

unread,

Nov 2, 2015, 4:39:10 PM11/2/15

to Qiime Forum

Richard,

I got that stuff.

I'm going to walk you through the instructions here:

http://qiime.org/install/install.html#then-just-pip-install-qiime

First of all, let's triple check that you do not have an active installation. You should be able to run this command and get an error:

print_qiime_config.py

Next, we have to install numpy before we can install QIIME. Run this command:

sudo pip install numpy

If that succeeded, you can install QIIME with this command:

sudo pip install qiime

After this, you should be have a basic system installed. Please run this command and send me the output:

print_qiime_config.py

Best,

Kyle

Richard Agans

unread,

Nov 2, 2015, 7:22:43 PM11/2/15

to qiime...@googlegroups.com

Kyle,

I have attached to text files of the command terminal during the numpy and qiime installs. There were a lot of warnings during the numpy install, but the message indicated a successful install. The Qiime install does not appear to have been successful however. Running the print_qiime_config.py indicates no command exists.

-Richard

--

---
You received this message because you are subscribed to a topic in the Google Groups "Qiime Forum" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/qiime-forum/-fD44oQg4wE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to qiime-forum...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

qiimeinstall.txt

numpy.txt

Kyle Bittinger

unread,

Nov 3, 2015, 8:42:01 AM11/3/15

to qiime...@googlegroups.com

Ok, I'm going to have you follow the instructions at the bottom of the error message. It appears that one of your python packages is too old for you to install matplotlib. Just to point out, this is a problem with the installation of matplotlib, which is used by QIIME to make plots.

Please run this command:

sudo easy_install -U distribute

Then re-try the QIIME installation command:

sudo pip install qiime

Then verify the installation:

print_qiime_config.py

Richard Agans

unread,

Nov 3, 2015, 12:38:30 PM11/3/15

to qiime...@googlegroups.com

Kyle,

OK, below is the output from sudo easy_install -U distribute

ra@Serenity:~$ sudo easy_install -U distribute

[sudo] password for ra:

Searching for distribute

Reading http://pypi.python.org/simple/distribute/

Best match: distribute 0.7.3

Downloading https://pypi.python.org/packages/source/d/distribute/distribute-0.7.3.zip#md5=c6c59594a7b180af57af8a0cc0cf5b4a

Processing distribute-0.7.3.zip

Running distribute-0.7.3/setup.py -q bdist_egg --dist-dir /tmp/easy_install-3GSQ5B/distribute-0.7.3/egg-dist-tmp-sOL7ux

warning: install_lib: 'build/lib.linux-x86_64-2.7' does not exist -- no Python modules to install

Adding distribute 0.7.3 to easy-install.pth file

Installed /usr/local/lib/python2.7/dist-packages/distribute-0.7.3-py2.7.egg

Processing dependencies for distribute

Searching for setuptools>=0.7

Reading http://pypi.python.org/simple/setuptools/

Best match: setuptools 18.5

Downloading https://pypi.python.org/packages/source/s/setuptools/setuptools-18.5.zip#md5=db877c1ea889a036e26321db71105a14

Processing setuptools-18.5.zip

Running setuptools-18.5/setup.py -q bdist_egg --dist-dir /tmp/easy_install-xea4LF/setuptools-18.5/egg-dist-tmp-ZQADMF

Adding setuptools 18.5 to easy-install.pth file

Installing easy_install script to /usr/local/bin

Installing easy_install-2.7 script to /usr/local/bin

Installed /usr/local/lib/python2.7/dist-packages/setuptools-18.5-py2.7.egg

Finished processing dependencies for distribute

Following this I output the results from sudo pip install qiime to the attached document qiime2.txt which appears to indicate a failure to install?

I did run print_qiime_config.py for fun and include the results from that below:

ra@Serenity:~$ print_qiime_config.py

Traceback (most recent call last):

File "/usr/local/bin/print_qiime_config.py", line 33, in <module>

raise ImportError("%s\n%s" % (e, core_dependency_missing_msg))

ImportError: No module named scipy

See the QIIME Installation Guide: http://qiime.org/install/install.html

Going to the guide I really don't see any troubleshooting resources. I also don't get why I seem to be having such issues with the install, since this is my personal machine (so have root access) with a clean Ubuntu install shouldn't it be easy to install QIIME? Am I doing something stupid that is resulting in this not working? I greatly appreciate your help and do apologize for these issues.

Best,

-Richard

qiime2.txt

Kyle Bittinger

unread,

Nov 3, 2015, 1:14:30 PM11/3/15

to Qiime Forum

Richard,

You are not doing anything stupid, you are just having problems installing the basic scientific computing packages for Python. Most of the time, this goes really smoothly, but sometimes it gets messed up in different ways. We may move to recommending the conda system for installing the basic scientific computing libraries in the future, for exactly this reason.

The error message you attached and the message from print_qiime_config.py indicate that scipy was not installed properly. Let's get scipy installed and working on your system.

Let's get rid of any half-baked scipy version that is left over from your previous installation. Please run:

sudo pip uninstall scipy

Let's make sure all the linear algebra libraries are present by doing a system install of scipy. Please run this command:

sudo apt-get install python-scipy

Now, the linear algebra libraries are present, but the version of scipy is too old. Let's update scipy using pip:

sudo pip install scipy

Now try to verify the QIIME install:

print_qiime_config.py

Richard Agans

unread,

Nov 3, 2015, 2:21:05 PM11/3/15

to qiime...@googlegroups.com

Kyle,

Running sudo pip uninstall scipy I receive "Cannot uninstall requirement scipy, not installed" with a log file location.
OK, it seems that no scipy is installed so no worries about that, just install. I have attached the output from running "sudo apt-get install python-scipy".

Looking at the output, it doesn't appear to me that there is any sort of error, awesome, so let's run "print_qiime_config.py". Uh-oh:

Traceback (most recent call last):

File "/usr/local/bin/print_qiime_config.py", line 40, in <module>

raise ImportError("%s\n%s" % (e, core_dependency_missing_msg))

ImportError: No module named skbio.util

See the QIIME Installation Guide: http://qiime.org/install/install.html

Does this mean I need to install skbio.util now?

-Richard

--

---
You received this message because you are subscribed to a topic in the Google Groups "Qiime Forum" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/qiime-forum/-fD44oQg4wE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to qiime-forum...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

scipy.txt

Kyle Bittinger

unread,

Nov 3, 2015, 2:29:58 PM11/3/15

to Qiime Forum

Richard, you forgot to do a pip install of scipy to update to the new version, after doing your system install:

sudo pip install scipy

And yes, you should "sudo pip install --update" any packages that appear in the error message like the one you posted.

Kyle Bittinger

unread,

Nov 3, 2015, 2:31:22 PM11/3/15

to Qiime Forum

And you would run this to install skbio:

sudo pip install scikit-bio

Richard Agans

unread,

Nov 3, 2015, 2:58:40 PM11/3/15

to qiime...@googlegroups.com

Kyle,

I'm lost, I did pass the sudo command to install scipy. I did rerun it and received:
"Requirement already satisfied (use --upgrade to upgrade): scipy in /usr/lib/python2.7/dist-packages
Cleaning up..."

OK, so it seems that scipy installed, let's pass --update with it
"ra@Serenity:~$ sudo pip install scipy--update
Downloading/unpacking scipy-update
Could not find any downloads that satisfy the requirement scipy-update
No distributions at all found for scipy-update
Storing complete log in /home/ra/.pip/pip.log"

Odd, but I did see above that scipy appears to be installed, so let's try to install skbio via sudo pip install scikit-bio:
A lot of messages ending in a rollback uninstall of scipy and error. I have re-passed the command and attached the output from that, the error message at the end appears to be the same. Reviewing the file, I do notice many NOT AVAILABLE messages, however I'm not sure if this causes an issue or not.

I am doing this in my home directory, do I need to change the working directory? I can't imagine how that would change anything, but at this point who knows!

-Richard

On Tue, Nov 3, 2015 at 2:31 PM, Kyle Bittinger <kylebi...@gmail.com> wrote:

And you would run this to install skbio:
sudo pip install scikit-bio

--

---
You received this message because you are subscribed to a topic in the Google Groups "Qiime Forum" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/qiime-forum/-fD44oQg4wE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to qiime-forum...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

skbio.txt

Kyle Bittinger

unread,

Nov 3, 2015, 3:54:54 PM11/3/15

to Qiime Forum

Richard,

Your working directory is not the problem. We need to get numpy and scipy sorted out on your system, then we can focus on the actual QIIME install.

Let's uninstall the pip versions of numpy and scipy while we do our work. Please run:

sudo pip uninstall scipy

sudo pip uninstall numpy

I googled your error message, and it may be that you need to install header files for the linear algebra libraries. Let's try to install those with the Ubuntu package manager. We will throw in a FORTRAN compiler for good measure because this also showed up on the StackOverflow thread. Please run:

sudo apt-get install libblas-dev liblapack-dev libatlas-base-dev gfortran

Now, please attempt to re-install numpy and scipy with pip. If the same error messages pop up, try using the --no-cache-dir option to make pip start from scratch.

Richard Agans

unread,

Nov 3, 2015, 4:50:57 PM11/3/15

to qiime...@googlegroups.com

Kyle,
I got successful uninstalls for scipy and numpy. Passed the sudo apt-get install..., and that seemed to work (at least I see now warnings or error messages in the output). I ran sudo pip install for scipy and numpy and get a message for each that the requirement is already satisfied in /usr/lib/python2.7/dist-packages.
Should I now try sudo pip install qiime?

-Richard

--

---
You received this message because you are subscribed to a topic in the Google Groups "Qiime Forum" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/qiime-forum/-fD44oQg4wE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to qiime-forum...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

-- 
-Richard Agans, MSc
PhD Candidate
Dr. Paliy, Advisor
Wright State University

Kyle Bittinger

unread,

Nov 4, 2015, 11:44:57 AM11/4/15

to Qiime Forum

Before we get to qiime, please make sure that numpy and scipy are correctly installed and up-to-date. I'm afraid that pip is picking up on the old system version, and want to make sure that you have the most recent version.

Please run:

pip install --upgrade numpy

pip install --upgrade scipy

Then run:

pip install qiime

Then test with this command:

print_qiime_config.py

Richard Agans

unread,

Nov 4, 2015, 12:18:40 PM11/4/15

to qiime...@googlegroups.com

Kyle,

It appears we have success! Passing 'print_qiime_config.py' returned the below:
ra@Serenity:~$ print_qiime_config.py

System information
==================
Platform: linux2

   Python version:    2.7.3 (default, Jun 22 2015, 19:33:41) [GCC 4.6.3]
Python executable:    /usr/bin/python

QIIME default reference information
===================================
For details on what files are used as QIIME's default references, see here:
https://github.com/biocore/qiime-default-reference/releases/tag/0.1.3

Dependency versions
===================
          QIIME library version:    1.9.1
           QIIME script version:    1.9.1
qiime-default-reference version:    0.1.3
                  NumPy version:    1.10.1
                  SciPy version:    0.16.1
                 pandas version:    0.17.0
             matplotlib version:    1.5.0
            biom-format version:    2.1.5
                   h5py version:    Not installed.
                   qcli version:    0.1.1
                   pyqi version:    0.3.2
             scikit-bio version:    0.2.3
                 PyNAST version:    1.2.2
                Emperor version:    0.9.51
                burrito version:    0.9.1
       burrito-fillings version:    0.1.1
              sortmerna version:    SortMeRNA version 2.0, 29/11/2014
              sumaclust version:    SUMACLUST Version 1.0.00
                  swarm version:    Swarm 1.2.19 [Nov 3 2015 12:05:18]
                          gdata:    Installed.

QIIME config values
===================
For definitions of these settings and to learn how to configure QIIME, see here:
http://qiime.org/install/qiime_config.html
http://qiime.org/tutorials/parallel_qiime.html

                     blastmat_dir:    None
      pick_otus_reference_seqs_fp:    /usr/local/lib/python2.7/dist-packages/qiime_default_reference/gg_13_8_otus/rep_set/97_otus.fasta
                         sc_queue:    all.q
      topiaryexplorer_project_dir:    None
     pynast_template_alignment_fp:    /usr/local/lib/python2.7/dist-packages/qiime_default_reference/gg_13_8_otus/rep_set_aligned/85_otus.pynast.fasta
                  cluster_jobs_fp:    start_parallel_jobs.py
pynast_template_alignment_blastdb:    None
assign_taxonomy_reference_seqs_fp:    /usr/local/lib/python2.7/dist-packages/qiime_default_reference/gg_13_8_otus/rep_set/97_otus.fasta
                     torque_queue:    friendlyq
                    jobs_to_start:    1
                       slurm_time:    None
            denoiser_min_per_core:    50
assign_taxonomy_id_to_taxonomy_fp:    /usr/local/lib/python2.7/dist-packages/qiime_default_reference/gg_13_8_otus/taxonomy/97_otu_taxonomy.txt
                         temp_dir:    /tmp/
                     slurm_memory:    None
                      slurm_queue:    None
                      blastall_fp:    blastall
                 seconds_to_sleep:    1

Did we finally get it working?

-Richard

--

---
You received this message because you are subscribed to a topic in the Google Groups "Qiime Forum" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/qiime-forum/-fD44oQg4wE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to qiime-forum...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Kyle Bittinger

unread,

Nov 4, 2015, 12:26:16 PM11/4/15

to Qiime Forum

Hot damn! Can I say that on the forum?

No seriously, I think it's time to analyze some sequences. I imagine that you may have some glitches early on -- please open a new forum topic if so!

Best,

Kyle

Richard Agans

unread,

Nov 4, 2015, 12:27:45 PM11/4/15

to qiime...@googlegroups.com

Kyle,
Will do, thank you and everyone else for the help! One last question, should I refrain from updating the Ubuntu version in the future, will that cause potential issues with QIIME?

Best,
-Richard

--

---
You received this message because you are subscribed to a topic in the Google Groups "Qiime Forum" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/qiime-forum/-fD44oQg4wE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to qiime-forum...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

-- 
-Richard Agans, MSc
PhD Candidate
Dr. Paliy, Advisor
Wright State University

Kyle Bittinger

unread,

Nov 4, 2015, 12:58:00 PM11/4/15

to Qiime Forum

Maybe, but please don't hold your computer at a specific version of Ubuntu forever. If you encounter problems after an upgrade, we'd like to know and help fix them.

--Kyle

Reply all

Reply to author

Forward