Trouble installing Qiime 1.9.1 on Ubuntu 12.04LTS

[benjaminm...@u.northwestern.edu]

Hello,

I'll say in advance that I appreciate any help that more skilled and experienced folks can offer.  I have included basic information about my difficulties below and will happily supply additional detailed outputs/warnings/etc if you will let me know which will be useful for troubleshooting.

I've had a working installation of Qiime 1.8.0 on my home PC running Ubuntu 12.04 for several years (a qiime-deploy based full native installation), but found that certain functions were not compatible with outputs I was getting from Qiime 1.9.0 installed at our lab, so I wanted to update.  However, I have had no luck whatsoever in getting a new installation to work, although I believe I am very carefully following the instructions on http://qiime.org/install/install.html and the qiime-deploy readme.

So far, I have been unable to even get the basic pip install to function. I have:

removed my original installation from its deploy location

removed all references to that deploy location from my PATH and PYTHONPATH in .bashrc

uninstalled and reinstalled build-essential, python-dev, and python-pip using apt-get utilities

uninstalled and reinstalled numpy and scipy using pip

attempted to install qiime using sudo pip install qiime

 - I have attached the log file to this question: there appears to be a problem with downloading scikit-bio causing my install to fail.  This seems to arise from scikit-bio requiring python 3, while I'm using python 2.7.3 by default, as Qiime requires python <3.  

 - I have made repeated attempts to do the base native install using pip following the instructions on the Qiime installation page, and sometimes it appears that the failure arises from an inability to find a usable version of simplegeneric, although I also pip installed that.  I believe all the outputs from my many attempts have been appended together in the log file I attached.

when I execute pip install qiime in ~/qiime/ I get what appears to be an installation in a directory named "build" which looks like it should have been a successful installation.  However, when I execute any of the scripts in build/qiime/qiime or build/qiime/scripts, I receive errors indicating that there are missing modules.  A couple cases in point:

python print_qiime_config.py -t returns:

Traceback (most recent call last):

  File "print_qiime_config.py", line 40, in <module>

    raise ImportError("%s\n%s" % (e, core_dependency_missing_msg))

ImportError: No module named skbio.util

python pick_open_reference_otus.py returns:

Traceback (most recent call last):

  File "pick_open_reference_otus.py", line 15, in <module>

    from qiime.util import (parse_command_line_parameters,

ImportError: No module named qiime.util

python split_libraries_fastq.py returns:

Traceback (most recent call last):

  File "split_libraries_fastq.py", line 19, in <module>

    from skbio.parse.sequences import parse_fastq

ImportError: No module named skbio.parse.sequences

So, it is clear that the pip installation was not successful.

I have also attempted to directly do a full install using qiime-deploy to ~/qiime_software.  Based on the instructions in the qiime-deploy readme I:

edited my etc/apt/sources.list to uncomment all the universe and multiverse repos and added
deb http://cran.rstudio.com/bin/linxu/ubuntu precise/ 

installed the qiime-deploy pre-reqs by copy/pasting the command for Ubuntu 12.04/14.04 directly from the readme; this appeared to complete successfully.  

Cloned the qiime-deploy and qiime-deploy-conf .git directories into my ~/

Executed python qiime-deploy.py ~/qiime_software -f ~/qiime-deploy-conf/qiime-1.9.1/qiime.conf --force-remove-failed-dirs

 - this appeared to work, with the exception of infernal, which was not installed.

Ran source ~/qiime_software/activate.sh

However, when I attempted to run print_qiime_config.py -t it was not recognized, so it's clear that the scripts were not added to my path.

I found the scripts manually, added the script directory (and the ~/qiime_software/ directory) to my path in .bashrc, and manually made all the scripts executable with chmod +x *.py, and sourced .bashrc.  I was now able to run the commands from any directory, but I received similar errors to those printed above, indicating that there were missing modules, including qiime.util.

I looked back through the qiime-deploy instructions, and saw that it now assumes a base native install using pip, so I guess that my failure to get that installation to work might explain this difficulty.  

I figure the first problem I need to troubleshoot is getting the pip install to work, and I can worry about completing the full native install later if it looks like I need any of the additional features.  I greatly appreciate any insight you may have into why this is not working for me.  Please let me know what additional system information, warning messages, or other outputs will be useful to look at, and I will post them here as soon as possible.

Many thanks,

Ben

[benjaminm...@u.northwestern.edu]

OK, well I feel pretty silly about how much trouble I had, and how simple the solution was; but I'm going to leave this post up here so that anyone else who runs into a similar problem might learn from my experience.

It turns out that the problem was ultimately with the version of pip I was using (!!!).  Despite the fact that I repeatedly uninstalled and reinstalled pip, it was continually calling an old version (1.0) that was lurking somewhere on my system (I'm guessing a version that was installed with distutils).

When I realized this, I did a version check on pip2 and saw that this was using v 8.1.2.  Just to be safe and have clean installs, I used pip2 to uninstall and reinstall numpy and scipy with commands along the lines of 

sudo -H pip2 uninstall numpy

sudo -H pip2 install numpy

(I also had to manually rm -r an old version of numpy that was hiding out in /usr/lib/python2.7/dist-packages).

After reinstalling those things, I ran 

sudo -H pip2 install h5py

and

sudo -H pip2 install qiime

checked the install with 

print_qiime_config.py -t

and everything seems to have worked.  I've been able to pick back up with my analyses where I'd left off.  I'll add another post if I encounter any further difficulties, but this seems to have cracked it!

Ben

[zhiying Guo]

Hello Ben,

Thanks for your post

Best,

Zhiying