updating issues

[Kelly]

Ok, so we made a new virtual box to be at the 64 bit version. Just to
show you this worked:
qiime@qiime-vm:~$ print_qiime_config.py -t

System information
==================
Platform: linux2
Python version: 2.6.4 (r264:75706, Nov 10 2010, 15:59:26) [GCC
4.4.3]
Python executable: /software/python-2.6.4-release/bin/python

Dependency versions
===================
PyCogent version: 1.5.0
NumPy version: 1.3.0
matplotlib version: 0.98.5.3
QIIME library version: 1.2.0
QIIME script version: 1.2.0
PyNAST version (if installed): 1.1
Denoiser version (if installed): 0.851

QIIME config values
===================
blastmat_dir: /software/blast-2.2.22-release/
data/
rdp_classifier_fp: /software/rdpclassifier-2.0.1-
release/rdp_classifier-2.0.jar
template_alignment_lanemask_fp: /software/lanemask_in_1s_and_0s
cluster_jobs_fp: None
pynast_template_alignment_fp: /software/
core_set_aligned.fasta.imputed
seconds_to_sleep: 60
pynast_template_alignment_blastdb: None
pyronoise_data_fp: /software/denoiser-0.851-release/
Data/LookUp.dat
jobs_to_start: 1
qiime_scripts_dir: /software/qiime-1.2.0-release/bin
working_dir: /tmp/
python_exe_fp: /software/python-2.6.4-release/bin/
python
temp_dir: /tmp/
blastall_fp: blastall


running checks:

FastTree is in path and version is supported ... ok
INFERNAL is in path and version is supported ... ok
blast is in path and version is supported ... ok
blastall_fp is set to a valid path ... ok
blastmat_dir is set to a valid path. ... ok
cdbtools is in path and version is supported ... ok
cd-hit is in path and version is supported ... ok
no obvious problems with ChimeraSlayer install ... ok
clearcut is in path and version is supported ... ok
cluster_jobs_fp is set to a valid path and is executable ... ok
denoiser is in path and version is supported ... ok
dotur is in path and version is supported ... ok
local qiime_config has no extra params ... ok
mothur is in path and version is supported ... ok
muscle is in path and version is supported ... ok
pynast_template_alignment_blastdb is set to a valid path ... ok
pynast_template_alignment is set to a valid path ... ok
pyrnoise2 is in path and version is supported ... ok
python_exe_fp is set to a working python env ... ok
python is in path and version is supported ... ok
qiime_scripts_dir is set to a valid path ... ok
raxmlHPC is in path and version is supported ... ok
rdp_classifier is set to a valid path ... ok
template_alignment_lanemask is set to a valid path ... ok
uclust is in path and version is supported ... ok
working_dir is set to a valid path ... ok

----------------------------------------------------------------------
Ran 26 tests in 0.762s

OK
qiime@qiime-vm:~$


However, when actually trying to work with my data in the shared
folder, nothing seemed to work. I was trying to do the jackknife
stuff that required me to update versions in the first place. I
figured that may still be a problem so I then put in a command that I
know worked for me in the past and it still didn't work. This is the
command and error message:

qiime@qiime-vm:~/Desktop/Shared_Folder$ make_3d_plots.py -i ./
Split_Library_Output/beta_div/euclidean_otu_table.txt -m
kelly_map_modified.txt -p prefs_outa.txt -o 3d_plotsb/
Usage: make_3d_plots.py [options] {-i/--coord_fname COORD_FNAME -m/--
map_fname MAP_FNAME}

[] indicates optional input (order unimportant)
{} indicates required input (order unimportant)

This script automates the construction of 3D plots (kinemage format)
from the PCoA output file generated by principal_coordinates.py (e.g.
P1 vs. P2 vs. P3, P2 vs. P3 vs. P4, etc., where P1 is the first
component).

Example usage:
Print help message and exit
make_3d_plots.py -h

Default Usage: If you just want to use the default output, you can
supply the principal coordinates file (i.e., resulting file from
principal_coordinates.py) and a user-generated mapping file, where the
default coloring will be based on the SampleID as follows
make_3d_plots.py -i beta_div_coords.txt -m Mapping_file.txt

Additionally, the user can supply their mapping file ("-m") and a
specific category to color by ("-b") or any combination of categories.
When using the -b option, the user can specify the coloring for
multiple mapping labels, where each mapping label is separated by a
comma, for example: -b 'mapping_column1,mapping_column2'. The user can
also combine mapping labels and color by the combined label that is
created by inserting an '&&' between the input columns, for example: -
b 'mapping_column1&&mapping_column2'.


If the user would like to color all categories in their metadata
mapping file, they can pass 'ALL' to the '-b' option, as follows
make_3d_plots.py -i beta_div_coords.txt -m Mapping_file.txt -b ALL

As an alternative, the user can supply a preferences (prefs) file,
using the -p option. The prefs file allows the user to give specific
samples their own columns within a given mapping column. This file
also allows the user to perform a color gradient, given a specific
mapping column.

If the user wants to color by using the prefs file (e.g. prefs.txt),
they can use the following code
make_3d_plots.py -i beta_div_coords.txt -m Mapping_file.txt -p
prefs.txt

Output Directory: If you want to give an specific output directory
(e.g. "3d_plots"), use the following code
make_3d_plots.py -i principal_coordinates-output_file --o 3d_plots/

Background Color Example: If the user would like to color the
background white they can use the '-k' option as follows
make_3d_plots.py -i beta_div_coords.txt -m Mapping_file.txt -b ALL -k
white

Jackknifed Principal Coordinates (w/ confidence intervals): If you
have created jackknifed PCoA files, you can pass the folder containing
those files, instead of a single file. The user can also specify the
opacity of the ellipses around each point "--ellipsoid_opacity", which
is a value from 0-1. Currently there are two metrics "--
ellipsoid_method" that can be used for generating the ellipsoids,
which are 'IQR' and 'sdev'. The user can specify all of these options
as follows
make_3d_plots.py -i jackknifed_pcoas/ -m Mapping_file.txt -b
'mapping_column1,mapping_column1&&mapping_column2' --
ellipsoid_opacity=0.5 --ellipsoid_method=IQR

Bi-Plots: If the user would like to see which taxa are more prevalent
in different areas of the PCoA plot, they can generate Bi-Plots, by
passing a principal coordinates file or folder "-i", a mapping file "-
m", and a summarized taxa file "-t" from summarize_taxa.py. Can be
combined with jacknifed principal coordinates.
make_3d_plots.py -i pcoa.txt -m Mapping_file.txt -t
otu_table_level3.txt

make_3d_plots.py: error: Every line of every coord file must have the
same number of columns.
qiime@qiime-vm:~/Desktop/Shared_Folder$

[Jesse Stombaugh]

Hello Kelly,

It appears you are passing an OTU table to the make_3d_plots.py script.  The input file for this script is a principal coordinates file, so you should run principal_coordinates.py first, then use the output of that script for make_3d_plots.py.

Principal Coordinates Link: http://qiime.sourceforge.net/scripts/principal_coordinates.html

-Jesse 

--
Jesse Stombaugh, Ph.D.
Research Associate
University of Colorado, Boulder

[Kelly Skinner]

I did do the principal coordinates (and a bunch of other stuff) first.  I just pulled out a command that worked before to run a test run.  Shouldn't all the data I did in the 32 bit version be recognized in the 64 bit version?

---

Date: Fri, 28 Jan 2011 11:03:42 -0700
Subject: Re: [qiime-forum 1.2.0-dev] updating issues
From: jisto...@gmail.com
To: qiime...@googlegroups.com

[Jesse Stombaugh]

Yes, everything should work the same, but can you post the command that is actually failing, so we can try to determine where the error is taking place.

-Jesse

[Jesse Stombaugh]

Hello again,

The error you show in the earlier email is due to a bad input file. This leads me to believe that it is not the output of principal_coordinates.py, which means there could be 1 of 2 options.: 1) the input file in the command is not a principal coordinates file (e.g. an OTU table); or 2) The command is improperly written (contains tabs or spaces in the wrong place), which can happen if you copy and paste commands.

Hope this helps,

  Jesse

[Kelly]

This is the command that I originally tried in the updated version and
the error I got. I was trying this command based on what I learned
from the Forum on what to try to get the jackknife stuff to work. I
went back and tried the commands I listed on Friday because they had
worked for me in version 1.1.

qiime@qiime-vm:~/Desktop/Shared_Folder$ python qiime/scripts/
jackknifed_beta_diversity.py -i ./Split_Library_Output/beta_div/
euclidean_otu_table.txt -t seqs.fna_rep_set_aligned_pfiltered.tre -p
custom_parameters_jack.txt -o wf_jack -e 5 -v -m
kelly_map_modified.txt
python: can't open file 'qiime/scripts/jackknifed_beta_diversity.py':
[Errno 2] No such file or directory

On Jan 28, 3:19 pm, Jesse Stombaugh <jistomba...@gmail.com> wrote:
> Hello again,
>
> The error you show in the earlier email is due to a bad input file. This
> leads me to believe that it is not the output of principal_coordinates.py,
> which means there could be 1 of 2 options.: 1) the input file in the command
> is not a principal coordinates file (e.g. an OTU table); or 2) The command
> is improperly written (contains tabs or spaces in the wrong place), which
> can happen if you copy and paste commands.
>
> Hope this helps,
>   Jesse
>

> On Fri, Jan 28, 2011 at 1:55 PM, Jesse Stombaugh <jistomba...@gmail.com>wrote:
>
>
>
>
>
> > Yes, everything should work the same, but can you post the command that is
> > actually failing, so we can try to determine where the error is taking
> > place.
>
> > -Jesse
>

> > On Fri, Jan 28, 2011 at 1:52 PM, Kelly Skinner <kellyne...@hotmail.com>wrote:
>
> >>  I did do the principal coordinates (and a bunch of other stuff) first.  I
> >> just pulled out a command that worked before to run a test run.  Shouldn't
> >> all the data I did in the 32 bit version be recognized in the 64 bit
> >> version?
>

> >> ------------------------------

> >> Date: Fri, 28 Jan 2011 11:03:42 -0700
> >> Subject: Re: [qiime-forum 1.2.0-dev] updating issues

> >> From: jistomba...@gmail.com

> >> To: qiime...@googlegroups.com
>
> >> Hello Kelly,
>
> >> It appears you are passing an OTU table to the make_3d_plots.py script.
> >>  The input file for this script is a principal coordinates file, so you
> >> should run principal_coordinates.py first, then use the output of that
> >> script for make_3d_plots.py.
>
> >> Principal Coordinates Link:
> >>http://qiime.sourceforge.net/scripts/principal_coordinates.html
>
> >> -Jesse
>

> University of Colorado, Boulder- Hide quoted text -
>
> - Show quoted text -

[Greg Caporaso]

Kelly,
In this command call it looks like your trying to access the script incorrectly.

> qiime@qiime-vm:~/Desktop/Shared_Folder$ python qiime/scripts/
> jackknifed_beta_diversity.py -i ./Split_Library_Output/beta_div/
> euclidean_otu_table.txt -t seqs.fna_rep_set_aligned_pfiltered.tre -p
> custom_parameters_jack.txt -o wf_jack -e 5 -v -m
> kelly_map_modified.txt
> python: can't open file 'qiime/scripts/jackknifed_beta_diversity.py':
> [Errno 2] No such file or directory

Try running:

jackknifed_beta_diversity.py -h

If that prints the help text then everything is working correctly --
the difference is that I removed the:

python qiime/scripts

from the beginning as the VirtualBox should know where to find that script.

If that does work, modify your command as follows (be sure to clean up
any line breaks that get pasted in the command from this message):

jackknifed_beta_diversity.py -i ./Split_Library_Output/beta_div/
euclidean_otu_table.txt -t seqs.fna_rep_set_aligned_pfiltered.tre -p
custom_parameters_jack.txt -o wf_jack -e 5 -v -m
kelly_map_modified.txt

Greg

[Kelly]

the jackknifed_beta_diversity.py -h part seemed to work ok, so I tried
the text you suggested and this is what I got:

qiime@qiime-vm:~$ cd Desktop
qiime@qiime-vm:~/Desktop$ cd Shared_Folder
qiime@qiime-vm:~/Desktop/Shared_Folder$ jackknifed_beta_diversity.py -
i ./Split_Library_Output/beta_div/euclidean_otu_table.txt -t
seqs.fna_rep_set_aligned_pfiltered.tre -pcustom_parameters_jack.txt -o

wf_jack -e 5 -v -mkelly_map_modified.txt

Traceback (most recent call last):
File "/software/qiime-1.2.0-release/bin/
jackknifed_beta_diversity.py", line 149, in <module>
main()
File "/software/qiime-1.2.0-release/bin/
jackknifed_beta_diversity.py", line 111, in main
% opts.parameter_fp
IOError: Can't open parameters file (custom_parameters_jack.txt). Does
it exist? Do you have read access?
qiime@qiime-vm:~/Desktop/Shared_Folder$

[Greg Caporaso]

What happens if you run:

ls -al ~/Desktop/Shared_Folder/custom_parameters_jack.txt

It looks like you're passing a file that doesn't exist for -p. If it
does exist can you post the output from the ls command?

Greg

[Kelly]

qiime@qiime-vm:~/Desktop/Shared_Folder$ ls -al ~/Desktop/Shared_Folder/
custom_parameters_jack.txt
ls: cannot access /home/qiime/Desktop/Shared_Folder/
custom_parameters_jack.txt: No such file or directory
qiime@qiime-vm:~/Desktop/Shared_Folder$

here is some of the earlier thread (read from the bottom up to go in
order) as to why I am trying this script:

Hello Kelly,


You may need to append the "python" tag and location of your qiime/
scripts dir as follows:


python qiime/scsripts/jackknifed_beta_diversity.py -i ./

Split_Library_Output/beta_div/euclidean_otu_table.txt -t
seqs.fna_rep_set_aligned_pfiltered.tre -p custom_parameters_jack.txt -
o wf_jack -e 5 -v -m kelly_map_modified.txt

-Jesse


On Tue, Jan 25, 2011 at 11:53 AM, Kelly <kelly...@hotmail.com>
wrote:

Thanks, I just tried this command and got this error:

qiime@qiime-vm:~/Desktop/Shared_Folder$ jackknifed_beta_diversity.py -

i ./Split_Library_Output/beta_div/euclidean_otu_table.txt -t
seqs.fna_rep_set_aligned_pfiltered.tre -p custom_parameters_jack.txt -
o wf_jack -e 5 -v -m kelly_map_modified.txt

jackknifed_beta_diversity.py: command not found


On Jan 25, 11:53 am, Jesse Stombaugh <jistomba...@gmail.com> wrote:
> Hello Kelly,
>
> The jacknifed_beta_diversity.py script is the one you should probably refer
> to:
>

> http://qiime.sourceforge.net/scripts/jackknifed_beta_diversity.html#i...

>
> The commands you have listed are correct, except there should not be a space
> between "--" and "master_pcoa...". When calling these scripts you can use
> either the single character like "-m" or you can use the full parameter name
> such as "--master_pcoa=...", however; for the command you are trying to run,
> there is not a single character shorthand for master_pcoa.
>
> -Jesse
>

> On Tue, Jan 25, 2011 at 10:45 AM, Kelly <kellyne...@hotmail.com> wrote:
> > I'm having a hard time finding in the tutorial where making jackknifed
> > principal coordinates is explained. I see it mentioned on how to use
> > them once you have them. Speaking of which, the tutorial uses the
> > following command and I'd like to know if it is correctly written:
>
> > make_3d_plots.py -i jackknifed_pcos/ -m Mapping_file.txt -t
> > otu_table_level3.txt -- master_pcoa=unweighted_unifrac_pc.txt
>
> > I don't recognize the -- or = symbols being used earlier during the
> > tutorial, so I thought I'd ask about them, too.

>


On Feb 1, 8:21 am, Greg Caporaso <gregcapor...@gmail.com> wrote:

[Greg Caporaso]

Hey Kelly,

Sorry for all the confusion with this.

> qiime@qiime-vm:~/Desktop/Shared_Folder$ ls -al ~/Desktop/Shared_Folder/custom_parameters_jack.txt
> ls: cannot access /home/qiime/Desktop/Shared_Folder/custom_parameters_jack.txt: No such file or directory

This indicates that the parameters files is not there. Have you
created a parameters file? If you've used one with other workflow
scripts, or in other versions of QIIME, that should work here as well.

Again, sorry for all of the confusion, but I think we are very close
to sorting this out.

Greg

[Kelly]

I don't think I made any parameters file. I was trying this first
script, then Jesse suggested the latter script, which is where that -p
custom_parameters part came from - then it became clear I needed to
update and afterwards I just went back to the latest script
suggestion.

qiime@qiime-vm:~/Desktop/Shared_Folder$ make_3d_plots.py -i ./
Split_Library_Output/beta_div/euclidean_otu_table.txt -m

kelly_map_modified.txt -t ./Split_Library_Output/Class7.txt --
master_pcoa=unweighted_unifrac_pc.txt -o bi_plot/

qiime@qiime-vm:~/Desktop/Shared_Folder$ jackknifed_beta_diversity.py -
i ./Split_Library_Output/beta_div/euclidean_otu_table.txt -t
seqs.fna_rep_set_aligned_pfiltered.tre -p custom_parameters_jack.txt -
o wf_jack -e 5 -v -m kelly_map_modified.txt