beta diversity question

[louisville 1]

Hi Qiime 1 experts,

I had the following errors when I try to run Beta diversity through plots. The PcoA folders are empty although the weighted and unweighted texts were generated. I used the corrected mapping file. Thanks for the help.

Bassam

qiime@qiime-190-virtual-box:~$ beta_diversity_through_plots.py -i '/home/qiime/FSU_closed_stage3/total_otus_filtered' -o bdiv_season -t '/home/qiime/FSU_closed_stage3/97_otus.tree' -m '/home/qiime/Season_mappingfile/Mapping_file season_corrected.txt' -e 100

/usr/local/lib/python2.7/dist-packages/skbio/stats/ordination/_principal_coordinate_analysis.py:107: RuntimeWarning: The result contains negative eigenvalues. Please compare their magnitude with the magnitude of some of the largest positive eigenvalues. If the negative ones are smaller, it's probably safe to ignore them, but if they are large in magnitude, the results won't be useful. See the Notes section for more details. The smallest eigenvalue is -0.305793519583 and the largest is 19.9304207689.

  RuntimeWarning

Traceback (most recent call last):

  File "/usr/local/bin/beta_diversity_through_plots.py", line 153, in <module>

    main()

  File "/usr/local/bin/beta_diversity_through_plots.py", line 150, in main

    status_update_callback=status_update_callback)

  File "/usr/local/lib/python2.7/dist-packages/qiime/workflow/downstream.py", line 183, in run_beta_diversity_through_plots

    close_logger_on_success=close_logger_on_success)

  File "/usr/local/lib/python2.7/dist-packages/qiime/workflow/util.py", line 122, in call_commands_serially

    raise WorkflowError(msg)

qiime.workflow.util.WorkflowError: 

*** ERROR RAISED DURING STEP: Make emperor plots, weighted_unifrac)

Command run was:

 make_emperor.py -i bdiv_season/weighted_unifrac_pc.txt -o bdiv_season/weighted_unifrac_emperor_pcoa_plot/ -m /home/qiime/Season_mappingfile/Mapping_file season_corrected.txt 

Command returned exit status: 2

Stdout:

Stderr

Error in make_emperor.py: option -m: file does not exist: '/home/qiime/Season_mappingfile/Mapping_file'

[louisville 1]

Hi, I asked a question about beta diversity but I didn’t get answer. Should the mapping file match exactly the number of the samples in the biome when you run bets diversity through plots script in order to generate pcoa graph?
Thanks
Bassam

[TonyWalters]

hello,

For your second question, the mapping and pcoa data do not have to be exact matches, but you do want the pcoa data to be a subset of the mapping data (i.e, no pcoa data that are not found in your mapping file).

For your first question, it looks like a problem with the specific mapping file path, not the contents of the mapping file:

Error in make_emperor.py: option -m: file does not exist: '/home/qiime/Season_mappingfile/Mapping_file'

right now you have this path with a space in it:

'/home/qiime/Season_mappingfile/Mapping_file season_corrected.txt'

I would rename the file so it's a space with an underscore:

'/home/qiime/Season_mappingfile/Mapping_file_season_corrected.txt'

and rerun it using the filepath with the underscore in it.

[louisville 1]

Hi Tony,

Thanks for the reply. I followed your instructions but I still get the same error (see below). I combined 3 biomes files into one biome containing all the experimental analysis. Should I merge the 3 tree files also? the mapping file is attached. Erros are highlighting the mapping files samples?. I don't know what kind of solution to adopt?. Please help. Thanks so much

Bassam

qiime@qiime-190-virtual-box:~$ merge_otu_maps.py -i '/home/qiime/Season_mappingfile/Mapping_file_season_corrected.txt' '/home/qiime/academic_mappingfile/Mapping_file_academic_corrected.txt' -o combined_mapping_file

Error in merge_otu_maps.py: Positional argument detected: /home/qiime/academic_mappingfile/Mapping_file_academic_corrected.txt

 Be sure all parameters are identified by their option name.

 (e.g.: include the '-i' in '-i INPUT_DIR')

If you need help with QIIME, see:

http://help.qiime.org

qiime@qiime-190-virtual-box:~$ merge_otu_maps.py -i '/home/qiime/academic_mappingfile/Mapping_file_academic_corrected.txt', '/home/qiime/Season_mappingfile/Mapping_file_season_corrected.txt' -o combined_mapping_file

Error in merge_otu_maps.py: No filepaths match pattern/name ''. All patterns must be matched at least once.

If you need help with QIIME, see:

http://help.qiime.org

qiime@qiime-190-virtual-box:~$ beta_diversity_through_plots.py -i '/home/qiime/FSU_closed_stage3/FSU_total_merged' -o bdiv_season -t '/home/qiime/FSU_closed_stage3/97_otus.tree' -m '/home/qiime/Season_mappingfile/Mapping_file_season_corrected.txt' -e 100

/usr/local/lib/python2.7/dist-packages/skbio/stats/ordination/_principal_coordinate_analysis.py:107: RuntimeWarning: The result contains negative eigenvalues. Please compare their magnitude with the magnitude of some of the largest positive eigenvalues. If the negative ones are smaller, it's probably safe to ignore them, but if they are large in magnitude, the results won't be useful. See the Notes section for more details. The smallest eigenvalue is -0.333071355145 and the largest is 20.1739013122.

  RuntimeWarning

Traceback (most recent call last):

  File "/usr/local/bin/beta_diversity_through_plots.py", line 153, in <module>

    main()

  File "/usr/local/bin/beta_diversity_through_plots.py", line 150, in main

    status_update_callback=status_update_callback)

  File "/usr/local/lib/python2.7/dist-packages/qiime/workflow/downstream.py", line 183, in run_beta_diversity_through_plots

    close_logger_on_success=close_logger_on_success)

  File "/usr/local/lib/python2.7/dist-packages/qiime/workflow/util.py", line 122, in call_commands_serially

    raise WorkflowError(msg)

qiime.workflow.util.WorkflowError: 

*** ERROR RAISED DURING STEP: Make emperor plots, weighted_unifrac)

Command run was:

 make_emperor.py -i bdiv_season/weighted_unifrac_pc.txt -o bdiv_season/weighted_unifrac_emperor_pcoa_plot/ -m /home/qiime/Season_mappingfile/Mapping_file_season_corrected.txt 

Command returned exit status: 2

Stdout:

Stderr

Error in make_emperor.py: The metadata mapping file has fewer sample identifiers than the coordinates file. Verify you are using a mapping file that contains at least all the samples contained in the coordinates file(s). You can force the script to ignore these samples by passing the '--ignore_missing_samples' flag.

qiime@qiime-190-virtual-box:~$ beta_diversity_through_plots.py -i '/home/qiime/FSU_closed_stage3/FSU_total_merged' -o bdiv_season -t '/home/qiime/FSU_closed_stage3/97_otus.tree' -m '/home/qiime/Season_mappingfile/Mapping_file_season_corrected.txt' -e 100 '--ignore_missing_samples' flag

Error in beta_diversity_through_plots.py: no such option: --ignore_missing_samples

If you need help with QIIME, see:

http://help.qiime.org

qiime@qiime-190-virtual-box:~$ beta_diversity_through_plots.py -i '/home/qiime/FSU_closed_stage3/FSU_total_merged' -o bdiv_season -t '/home/qiime/FSU_closed_stage3/97_otus.tree' -m '/home/qiime/Season_mappingfile/Mapping_file_season_corrected.txt' -e 100 '--ignore_missing_samples' 

Error in beta_diversity_through_plots.py: no such option: --ignore_missing_samples

If you need help with QIIME, see:

http://help.qiime.org

qiime@qiime-190-virtual-box:~$ beta_diversity_through_plots.py -i '/home/qiime/FSU_closed_stage3/FSU_total_merged' -o bdiv_season -t '/home/qiime/FSU_closed_stage3/97_otus.tree' -m '/home/qiime/Season_mappingfile/Mapping_file_season_corrected.txt' -e 100

/usr/local/lib/python2.7/dist-packages/skbio/stats/ordination/_principal_coordinate_analysis.py:107: RuntimeWarning: The result contains negative eigenvalues. Please compare their magnitude with the magnitude of some of the largest positive eigenvalues. If the negative ones are smaller, it's probably safe to ignore them, but if they are large in magnitude, the results won't be useful. See the Notes section for more details. The smallest eigenvalue is -0.326119476948 and the largest is 20.2568528421.

  RuntimeWarning

Traceback (most recent call last):

  File "/usr/local/bin/beta_diversity_through_plots.py", line 153, in <module>

    main()

  File "/usr/local/bin/beta_diversity_through_plots.py", line 150, in main

    status_update_callback=status_update_callback)

  File "/usr/local/lib/python2.7/dist-packages/qiime/workflow/downstream.py", line 183, in run_beta_diversity_through_plots

    close_logger_on_success=close_logger_on_success)

  File "/usr/local/lib/python2.7/dist-packages/qiime/workflow/util.py", line 122, in call_commands_serially

    raise WorkflowError(msg)

qiime.workflow.util.WorkflowError: 

*** ERROR RAISED DURING STEP: Make emperor plots, weighted_unifrac)

Command run was:

 make_emperor.py -i bdiv_season/weighted_unifrac_pc.txt -o bdiv_season/weighted_unifrac_emperor_pcoa_plot/ -m /home/qiime/Season_mappingfile/Mapping_file_season_corrected.txt 

Command returned exit status: 2

Stdout:

Stderr

Error in make_emperor.py: The metadata mapping file has fewer sample identifiers than the coordinates file. Verify you are using a mapping file that contains at least all the samples contained in the coordinates file(s). You can force the script to ignore these samples by passing the '--ignore_missing_samples' flag.

qiime@qiime-190-virtual-box:~$ 

[louisville 1]

Hi Tony,

At this point, I don't know how to handle this situation?. I tried several times but the PcoA folders are empty. Just a reminder, I merged 3 biomes into one and I want to run Beta diversity through plots. Can I send you the biome and the mapping files to try them. Thanks for your help.

Bass

[TonyWalters]

Here is the error that you're posting now:

Stderr

Error in make_emperor.py: The metadata mapping file has fewer sample identifiers than the coordinates file. Verify you are using a mapping file that contains at least all the samples contained in the coordinates file(s). You can force the script to ignore these samples by passing the '--ignore_missing_samples' flag.

Here is the prior error you posted:

Stderr

Error in make_emperor.py: option -m: file does not exist: '/home/qiime/Season_mappingfile/Mapping_file'

So it did fix the issue with the filepath being incorrect, but it revealed a new error, which is part of what I was saying earlier: "For your second question, the mapping and pcoa data do not have to be exact matches, but you do want the pcoa data to be a subset of the mapping data (i.e, no pcoa data that are not found in your mapping file)."

I would strongly recommend that you amend your mapping files to fill in all metadata for your samples. If you wish to bypass it (which may make completely meaningless results, depending upon just how many samples are not matching in your mapping data), you can follow the message in the error:

"*** ERROR RAISED DURING STEP: Make emperor plots, weighted_unifrac)

Command run was:

 make_emperor.py -i bdiv_season/weighted_unifrac_pc.txt -o bdiv_season/weighted_unifrac_emperor_pcoa_plot/ -m /home/qiime/Season_mappingfile/Mapping_file_season_corrected.txt 

Command returned exit status: 2

Stdout:

Stderr

Error in make_emperor.py: The metadata mapping file has fewer sample identifiers than the coordinates file. Verify you are using a mapping file that contains at least all the samples contained in the coordinates file(s). You can force the script to ignore these samples by passing the '--ignore_missing_samples' flag."

which is to say, rerun the make_emperor.py command with the flag it tells you.

make_emperor.py -i bdiv_season/weighted_unifrac_pc.txt -o bdiv_season/weighted_unifrac_emperor_pcoa_plot/ -m /home/qiime/Season_mappingfile/Mapping_file_season_corrected.txt --ignore_missing_samples

[louisville 1]

Hi Tony,

Thanks for your help. It worked when I included all the samples in the mapping file (100% matching between mapping file and biome samples). Is there anyway, to present the figure in a better publication quality presentation?. Also, if I want to highlight some groups and not all on the figure?. Thanks so much for your guidance.

Bassam