add_qiime_labels

[divyapr...@gmail.com]

Hi every one 

Hope Every one will be alright 

I am running a command add_qiime _labels  and i am getting the following error

add_qiime_labels.py -i T_A_1_joined/  T_A_2_joined/  T_C_joined/  T_F_joined/  T_O_joined/ -m validtaed_mapping_file/Book1.csv_corrected.txt -c InputFastaFileName -o Termite_combined_sequences

Error in add_qiime_labels.py: Positional argument detected: T_A_2_joined/

 Be sure all parameters are identified by their option name.

 (e.g.: include the '-i' in '-i INPUT_DIR')

If you need help with QIIME, see:

http://help.qiime.org

 

Please sought it 

[divyapr...@gmail.com]

HI All 

i am done with add_qiime .labels. But where i am fixed now is calculating

Alpha diversity

 alpha_diversity.py -i Termite_otus/otu_table.biom -o Alpha_diversity -t Termite_otus/rep_set.tre -m PD_whole_tree

I only get a file but not any graph .

Secondly when i run the command  alpha_rarefraction.py 

i am getting the following error  

alpha_rarefaction.py -i Termite_otus/otu_table.biom -m validtaed_mapping_file/Book1.csv_corrected.txt -t Termite_otus/rep_set.tre -o Termite_rarefraction_Curve

Traceback (most recent call last):

  File "/usr/local/bin/alpha_rarefaction.py", line 161, in <module>

    main()

  File "/usr/local/bin/alpha_rarefaction.py", line 158, in main

    retain_intermediate_files=retain_intermediate_files)

  File "/usr/local/lib/python2.7/dist-packages/qiime/workflow/downstream.py", line 342, in run_alpha_rarefaction

    close_logger_on_success=close_logger_on_success)

  File "/usr/local/lib/python2.7/dist-packages/qiime/workflow/util.py", line 122, in call_commands_serially

    raise WorkflowError(msg)

qiime.workflow.util.WorkflowError: 

*** ERROR RAISED DURING STEP: Alpha diversity on rarefied OTU tables

Command run was:

 alpha_diversity.py -i Termite_rarefraction_Curve/rarefaction/ -o Termite_rarefraction_Curve/alpha_div/  -t Termite_otus/rep_set.tre

Command returned exit status: 1

Stdout:

Stderr

Traceback (most recent call last):

  File "/usr/local/bin/alpha_diversity.py", line 131, in <module>

    main()

  File "/usr/local/bin/alpha_diversity.py", line 115, in main

    opts.tree_path)

  File "/usr/local/lib/python2.7/dist-packages/qiime/alpha_diversity.py", line 379, in multiple_file_alpha

    output_fp, tree_path)

  File "/usr/local/lib/python2.7/dist-packages/qiime/alpha_diversity.py", line 326, in single_file_alpha

    result_path=outfilepath, log_path=None)

  File "/usr/local/lib/python2.7/dist-packages/qiime/util.py", line 259, in __call__

    result = self.getResult(*args, **kwargs)

  File "/usr/local/lib/python2.7/dist-packages/qiime/alpha_diversity.py", line 199, in getResult

    sample_names=otu_table.ids())

  File "/usr/local/lib/python2.7/dist-packages/qiime/util.py", line 259, in __call__

    result = self.getResult(*args, **kwargs)

  File "/usr/local/lib/python2.7/dist-packages/qiime/alpha_diversity.py", line 128, in getResult

    new_sample_names, result = self.Metric(tree, envs, **self.Params)

  File "/usr/local/lib/python2.7/dist-packages/cogent/maths/unifrac/fast_unifrac.py", line 227, in PD_whole_tree

    branch_lengths, nodes, t = _fast_unifrac_setup(t, envs)

  File "/usr/local/lib/python2.7/dist-packages/cogent/maths/unifrac/fast_unifrac.py", line 194, in _fast_unifrac_setup

    raise ValueError, "No valid samples/environments found. Check whether tree tips match otus/taxa present in samples/environments"

ValueError: No valid samples/environments found. Check whether tree tips match otus/taxa present in samples/environments

Please help me out here.

[divyapr...@gmail.com]

Kindly please help out here as i am stuck in my analysis.

[Colin Brislawn]

Here is the important line from the error:

ValueError: No valid samples/environments found. Check whether tree tips match otus/taxa present in samples/environments

Looks like your .tre file does not match your .biom file. Are these from the same run of pick_otu.py? If they are different, then this script will not work well.

Colin

[divyapr...@gmail.com]

Good Morning Colin 

Yes they are run from the same pick_de_novo_otu.py.

After running this command, the out put are as:

1. pynast_aligned_seqs.

2.rep_set

3.uclust_assigned_taxonomy

4. uclust_picked_otus

5.log .txt

6.otu_table.biom

7.rep_set.tre

 These are the outputs of the command.

Please suggest here what to do.

[Colin Brislawn]

Thanks for explaining the full pipeline. 

Can you upload the log file and the .tre file? Also, can you use the biom summarize-table with this .biom file and post the output? Here is an example command to get you started. 

biom summarize-table -i rich_sparse_otu_table.biom -o rich_sparse_otu_table_summary.txt

I wonder if something went wrong with the processing (which I can see from the log file). Or maybe I can see something strange with the .biom file summary or with the tree file.

Thanks! Let see if we can solve this,

Colin

[divyapr...@gmail.com]

Yes Colin

 i am attaching the same.

[Colin Brislawn]

Thanks. Ok, so your .biom table has 5 samples total. Is that the right number of samples?

Next, I can see the .biom table has all de novo OTU, which matches the pipeline you described. Can you also post the first few lines of Termite_otus/otu_table.biom using this command:

head -c 300 Termite_otus/otu_table.biom 

Colin

[Colin Brislawn]

Nevermind, I found the problem!

Your .biom file has  24,708 OTUs in it. Your .tre file has only 13,276 OTUs in it. Because some OTUs are missing from the .tre file, this command will not work.

My guess is that not all of your OTUs were able to be aligned when running align_seqs.py. The solution is to filter out the OTUs that are not part of the tree, or use alpha_diversity.py -m chao1,observed_otus which do not need a .tre file. 

Sorry for my confusion. Let me know if this helps,

Colin

[divyapr...@gmail.com]

Thanks 

While running the alpha_diversity.py -m chao1,observed_otus

Running the alpha_diversity command this is the error that is generated.

alpha_diversity.py -i Termite_otus/otu_table.biom -o Alpha_diversity -m 'chao1' 'observed_species' 'PD_whole_tree'

Error in alpha_diversity.py: Positional argument detected: observed_species

 Be sure all parameters are identified by their option name.

 (e.g.: include the '-i' in '-i INPUT_DIR')

Secondly please let me know how to filter the otus form the biom table. Actually what i mean is that while filtering the otus from the table 

1. how can i know that those otus are filtered that are not the part of the tree.

[Colin Brislawn]

Hello again,

You will have to run the full command, and include an input. Try this: 

alpha_diversity.py -i Termite_otus/otu_table.biom -o Alpha_diversity -t Termite_otus/rep_set.tre -m chao1,observed_species

When I have some more time tomorrow, I can discuss how best to filter out that .biom file to match your .tre file.

Colin

[divyapr...@gmail.com]

Hello Again 

Sorry for inconvenience caused by me 

running the following full command:

alpha_diversity.py -i Termite_otus/otu_table.biom -o Alpha_diversity -t Termite_otus/rep_set.tre -m chao1, PD_whole_tree

Error in alpha_diversity.py: option -m: invalid choice: '' (choose from 'ace','berger_parker_d','brillouin_d','chao1','chao1_ci','dominance','doubles','enspie','equitability','esty_ci','fisher_alpha','gini_index','goods_coverage','heip_e','kempton_taylor_q','margalef','mcintosh_d','mcintosh_e','menhinick','michaelis_menten_fit','observed_otus','observed_species','osd','simpson_reciprocal','robbins','shannon','simpson','simpson_e','singles','strong','PD_whole_tree')

If you need help with QIIME, see:

http://help.qiime.org

Secondly however running with out the -m the file i got is simple text file 

how ever no graphical representation is done 

i mean not graph is generated.

Attached is the file 

[divyapr...@gmail.com]

More over kindly please do help tomorrow regarding how to best filter the otus that match the the otus in the tre. 

[Colin Brislawn]

Hello again,

In this command, there is a space.

alpha_diversity.py -i Termite_otus/otu_table.biom -o Alpha_diversity -t Termite_otus/rep_set.tre -m chao1, PD_whole_tree

There is a space between chao1, and PD_whole tree.                                             Right here ^

That space is causing the error. Keep in mind that using PD_whole_tree will require a .tre file with matching OTUs, which we do not have right now.

This script does not make graphs, just this file. You can open the file using a text editor or using Excel, then make a graph with Excel or Google Sheets. If you want to make a graphs with qiime, try this script:

http://qiime.org/scripts/alpha_rarefaction.html 

I noticed that the alpha_diversity file you posted includes the results of PD_whole_tree. How did you get that?

Colin

[divyapr...@gmail.com]

Hi Colin 

I simply ran the command

alpha_diversity.py -i Termite_otus/otu_table.biom -o Alpha_diversity -t Termite_otus/rep_set.tre -m chao1, PD_whole_tree

 and it gave the results.

[divyapr...@gmail.com]

Hi Colin

When I have some more time tomorrow, I can discuss how best to filter out that .biom file to match your .tre file.

please discuss it here 

[Colin Brislawn]

Good morning,

I just checked some of my analysis and past projects, and realize that I don't know how to filter a .biom table to remove OTUs that are not in a tree. The various methods and software I have used have never needed this type of filter. 

Maybe another person knows how to do this filter. Keep in mind that the qiime developers are now working on Qiime 2, and there may be a Qiime 2 script in order to do this.

Colin

[divyapr...@gmail.com]

HI Colin 

Can i contact to Tony Walters

[divyapr...@gmail.com]

Hi again

 Sorry for disturbing 

Can i use this command: (http://qiime.org/scripts/filter_otus_from_otu_table.html)  and (http://qiime.org/scripts/filter_fasta.html) 

 then using a filtered rep_set.fna file as a way to filter the tree with the filter_tree.py (http://qiime.org/scripts/filter_tree.html) script  and finally usually that rep_set.tre in case of alpha_rarefaction.py

[TonyWalters]

You can't go backwards from the tree to filter and OTU table to match the tree. There isn't a way to filter a fasta file or an OTU table with a tree as input, sorry.

I'd recommend you go back to your raw data and start the process over-you shouldn't have tips that aren't matching your OTUs unless something happened along the way, and it's probably the best use of everyone's time to just redo it instead of trying to come up with some hack to fix whatever happened to your tree.

[divyapr...@gmail.com]

Thanks To

Actually i have only got one single sample as my own. For the analysis to be convenient i downloaded some of the sequences that were available publicly so as to process my analysis .