adonis error

[AB]

Hello,

I am trying to run an adonis test for a colleague and am receiving the error below:

command: cd "/mnt/data/joanne/";  R --slave --args --source_dir /usr/local/lib/python2.7/dist-packages/qiime/support_files/R -d bray_curtis_merged/bray_curtis_merged_USNA_SAV.txt -m map_jac.txt -c Group -o adonis -n 999  < /usr/local/lib/python2.7/dist-packages/qiime/support_files/R/adonis.r
stdout:

stderr:
Error in model.frame.default(formula, data, drop.unused.levels = TRUE) :
  invalid type (NULL) for variable 'qiime.data$map[[opts$category]]'
Calls: adonis -> model.frame -> model.frame.default
Execution halted

Attached is the mapping file and distance matrix.  I was able to run both of these files in an anosim test, so I am unsure how to interpret the error.

Thanks,
Lisa

[Lisa A. Beirn]

I just tried filling in the empty cells in the mapping file with NA as reported on another thread- this time it appears as if the analysis works, but it generates an empty output folder.

Thanks,

Lisa

--

---
You received this message because you are subscribed to a topic in the Google Groups "Qiime Forum" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/qiime-forum/0sb_QDSE3FA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to qiime-forum...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Lisa A. Beirn

Ph.D. Candidate

Rutgers, The State University of New Jersey

Dept. of Plant Biology & Pathology

59 Dudley Road

New Brunswick, NJ 08901
Office: (848)932-6312

Email: lbe...@scarletmail.rutgers.edu

[Lisa A. Beirn]

Oops. My mistake, there was an extra space in one of the files. Please disregard previous messages.

Thanks!

[Debora]

Hello!

I am having the same problem with files from a colleague, who wants to run adonis and permdisp tests.

This is the error with the adonis method:

stderr:
Error in model.frame.default(formula, data, drop.unused.levels = TRUE) :
  invalid type (NULL) for variable 'qiime.data$map[[opts$category]]'
Calls: adonis -> model.frame -> model.frame.default
Execution halted

and this is the one with the PERMDISP method:

stderr:
Error in x - c : non-conformable arrays
Calls: betadisper -> Resids
Execution halted

I have checked the mapping file but there are no spaces, I have tried to use the corrected file after validate_mapping_file.py but no success. I have to mention that we are running the same exact commands and analyses with different mapping files and data and on some of them it is working without problems.

What can be the reason for the error here?

I hope someone can help us, I am out of ideas!

Thanks a lot!
Debora

[Kyle Bittinger]

I notice that some of your sample ID's start with a number.  Sometimes, when R reads in a table with where column names start with a number, it puts an "X." in front of the number.  This could cause the type of problem you see.

Could you create a new mapping file and try testing with a small input FASTA file to see if this is the issue?

[zabe...@ucdavis.edu]

I'm running into the same problem.  A normal part of my workflow involves CSS-normalizing my OTU table, then using ADONIS to see if the normalization messed up my data.  I've done this analysis multiple times on other datasets without any problems.  

 

#normalize
 normalize_table.py -i ~/Zach/USDA2/OTU_picking/USDA2_otu_table_0.0005.biom -o ~/Zach/USDA2/OTU_picking/CSS_USDA2_otu_table_0.0005.biom -a CSS

#generate distance matrix
beta_diversity_through_plots.py -i ~/Zach/USDA2/OTU_picking/CSS_USDA2_otu_table_0.0005.biom -o ~/Zach/USDA2/core_analyses/CSS/ -m ~/Zach/USDA2/USDA2_mapfile.txt -a -O 12 -t ~/Zach/USDA2/OTU_picking/rep_set.tre
 
#summarize table, add normalized library sizes to mapping file
biom summarize-table -i ~/Zach/USDA2/OTU_picking/CSS_USDA2_otu_table_0.0005.biom -o ~/Zach/USDA2/OTU_picking/SUMMARY_CSS_USDA2_otu_table_0.0005.txt
 
#trying to run ADONIS
compare_categories.py -i ~/Zach/USDA2/core_analyses/0.0005/bdiv_even1763/weighted_unifrac_dm.txt -o ~/Zach/USDA2/core_analyses/0.0005/bdiv_even1763/CSS_check/ -c CSS_counts -m ~/Zach/USDA2/USDA2_mapfile.txt --method adonis
 
#Error in model.frame.default(formula, data, drop.unused.levels = TRUE) :  

  invalid type (NULL) for variable 'qiime.data$map[[opts$category]]'

Calls: adonis -> model.frame -> model.frame.default

Execution halted

 

ANOSIM works fine, but ADONIS and Moran's I refuse to cooperate.  I've attached my DM, OTU table, and mapping file.  There should not be any blank spaces in the mapping file, and none of my sample IDs start with numbers.  If anyone has some insight into the problem, please let me know.  

[zech xu]

Hello,

I looked at the code and your files and found the mapping files were read in inappropriately. It seems there are a problem parsing your mapping file. For a quick fix, I de-comment your header line in your mapping file and run the following command and then I get the result as following. Note you can't just run "compare_categories.py -i weighted_unifrac_dm.txt -o abc -c CSS_counts -m USDA2_mapfile.txt --method adonis" after the removal of the comment char. You will see the error of "qiime.parse.QiimeParseError: No header line was found in mapping file". So the best thing to do is probably to fix your mapping file and run "validate_mapping_file.py" to verify your mapping file is correct. Or you can work around it like I did here.

$ R --slave --args --source_dir /Users/zech/anaconda/envs/qiime_1.9/lib/python2.7/site-packages/qiime/support_files/R -d weighted_unifrac_dm.txt -m USDA2_mapfile.txt -c CSS_counts -o abc -n 999  < /Users/zech/anaconda/envs/qiime_1.9/lib/python2.7/site-packages/qiime/support_files/R/adonis.r

$ cat abc/adonis_results.txt

Call:
adonis(formula = as.dist(qiime.data$distmat) ~ qiime.data$map[[opts$category]],      permutations = opts$num_permutations)

Permutation: free
Number of permutations: 999

Terms added sequentially (first to last)

                                Df SumsOfSqs MeanSqs F.Model      R2 Pr(>F)
qiime.data$map[[opts$category]]  1    0.6165 0.61645  10.693 0.15343  0.001 ***
Residuals                       59    3.4014 0.05765         0.84657
Total                           60    4.0178                 1.00000
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

[zabe...@ucdavis.edu]

I checked my mapping file with validate_mapping_file.py, and the only error it gave was for the 'NA' values in the barcode column.  However, I was able to use your workaround to get a result, so thanks!

[Christian Gray]

Hi,

Not sure if I should start another thread, but I am having this same problem, but am unable to get the workaround to work. Here is the error when I run the original adonis command:

command:

compare_categories.py --method adonis -i /Users/christian/Documents/Run_Sequence/otus_Christian_16s_Q29_no_na/cdout29NoNa2760/bdiv_even2760/weighted_unifrac_dm.txt  -m ../../16sMapMerged.txt -c Species -o 16s_adonis_out -n 999

Error:

Traceback (most recent call last):

  File "/macqiime/anaconda/bin/compare_categories.py", line 189, in <module>

    main()

  File "/macqiime/anaconda/bin/compare_categories.py", line 185, in main

    categories, opts.num_permutations, out_dir)

  File "/macqiime/anaconda/lib/python2.7/site-packages/qiime/compare_categories.py", line 146, in compare_categories

    rex(command_args, '%s.r' % method)

  File "/macqiime/anaconda/lib/python2.7/site-packages/qiime/util.py", line 1968, in __call__

    (exit_status, command, stdout, stderr))

burrito.util.ApplicationError: Unacceptable application exit status: 1

command: cd "/Users/christian/Documents/Run_Sequence/otus_Christian_16s_Q29_no_na/cdout29NoNa2760/";  R --slave --args --source_dir /macqiime/anaconda/lib/python2.7/site-packages/qiime/support_files/R -d /Users/christian/Documents/Run_Sequence/otus_Christian_16s_Q29_no_na/cdout29NoNa2760/bdiv_even2760/weighted_unifrac_dm.txt -m ../../16sMapMerged.txt -c Species -o 16s_adonis_out -n 999  < /macqiime/anaconda/lib/python2.7/site-packages/qiime/support_files/R/adonis.r

stdout:

stderr:

Error in model.frame.default(formula, data, drop.unused.levels = TRUE) : 

  invalid type (NULL) for variable 'qiime.data$map[[opts$category]]'

Calls: adonis -> model.frame -> model.frame.default

Execution halted

And here is the work around command I tried to run:

R --slave --args --source_dir /Users/christian/miniconda3/pkgs/qiime-1.9.1-np110py27_0/lib/python2.7/site-packages/qiime/support_files/R/ -d /Users/christian/Documents/Run_Sequence/otus_Christian_16s_Q29_no_na/cdout29NoNa2760/bdiv_even2760/weighted_unifrac_dm.txt -m ../../16sMapMerged.txt -c Species -o 16s_adonis_out -n 999  < /Users/christian/miniconda3/pkgs/qiime-1.9.1-np110py27_0/lib/python2.7/site-packages/qiime/support_files/R/adonis.r

And here is the error:

 < /Users/christian/miniconda3/pkgs/qiime-1.9.1-np110py27_0/lib/python2.7/site-packages/qiime/support_files/R/adonis.r

Error in model.frame.default(formula, data, drop.unused.levels = TRUE) : 

  invalid type (NULL) for variable 'qiime.data$map[[opts$category]]'

Calls: adonis -> model.frame -> model.frame.default

Execution halted

MacQIIME nat-oitwireless-inside-vapornet100-a-10459:cdout29NoNa2760 $ R --slave --args --source_dir /Users/christian/miniconda3/pkgs/qiime-1.9.1-np110py27_0/lib/python2.7/site-packages/qiime/support_files/R/ -d /Users/christian/Documents/Run_Sequence/otus_Christian_16s_Q29_no_na/cdout29NoNa2760/bdiv_even2760/weighted_unifrac_dm.txt -m ../../16sMapMerged.txt -c Species -o 16s_adonis_out -n 999  < /Users/christian/miniconda3/pkgs/qiime-1.9.1-np110py27_0/lib/python2.7/site-packages/qiime/support_files/R/adonis.r

Error in model.frame.default(formula, data, drop.unused.levels = TRUE) : 

  invalid type (NULL) for variable 'qiime.data$map[[opts$category]]'

Calls: adonis -> model.frame -> model.frame.default

Execution halted

In addition, here is the error when I try to run compare_categories.py with permdisp

Traceback (most recent call last):

  File "/macqiime/anaconda/bin/compare_categories.py", line 189, in <module>

    main()

  File "/macqiime/anaconda/bin/compare_categories.py", line 185, in main

    categories, opts.num_permutations, out_dir)

  File "/macqiime/anaconda/lib/python2.7/site-packages/qiime/compare_categories.py", line 146, in compare_categories

    rex(command_args, '%s.r' % method)

  File "/macqiime/anaconda/lib/python2.7/site-packages/qiime/util.py", line 1968, in __call__

    (exit_status, command, stdout, stderr))

burrito.util.ApplicationError: Unacceptable application exit status: 1

command: cd "/Users/christian/Documents/Run_Sequence/otus_Christian_16s_Q29_no_na/cdout29NoNa2760/";  R --slave --args --source_dir /macqiime/anaconda/lib/python2.7/site-packages/qiime/support_files/R -d /Users/christian/Documents/Run_Sequence/otus_Christian_16s_Q29_no_na/cdout29NoNa2760/bdiv_even2760/weighted_unifrac_dm.txt -m ../../16sMapMerged.txt -c Species -o 16s_permdisp_out -n 999  < /macqiime/anaconda/lib/python2.7/site-packages/qiime/support_files/R/permdisp.r

stdout:

stderr:

Error in x - c : non-conformable arrays

Calls: betadisper -> Resids

Execution halted

I have attached my mapping file. Thanks so much. Let me know if it is more appropriate for me to start a new thread!

Thank you!

[Evan Bolyen]

Hi Christian,

It looks like there is an `NA` value for the `NANA` sample for the `Species` column. This will be converted the NULL in R which is what we're seeing in your exception.

You can rename the Species value for the NANA sample to something else, but you probably don't want to include it in your analysis (it appears to be a blank sample which will be its own group of 1).
In which case, you'll want to filter your distance matrix with `filter_distance_matrix.py` to remove that sample.
(If you remove that sample, you may also want to remove NCNA and LonNA which also looks like NA samples.)

Hope that helps!
-Evan

[Christian Gray]

Hi,

Thank you for the advice. I gave that a try, but was still unable to get it to run. I filtered out all of the NA samples, tried to run adonis, and got the same error. I also tried to run it when I filtered them out of the mapping file itself, and also did not have any luck. There should not be any samples now that have a NA in the category Species. Here is the error I got from the most recent command:

command: compare_categories.py --method adonis -i ./bdiv_even2760/filtered_unweighted_unifrac_dm.txt -m ../16sMapMerged_filtered.txt -c Species -o 16s_adonis_out -n 999

Traceback (most recent call last):

  File "/macqiime/anaconda/bin/compare_categories.py", line 189, in <module>

    main()

  File "/macqiime/anaconda/bin/compare_categories.py", line 185, in main

    categories, opts.num_permutations, out_dir)

  File "/macqiime/anaconda/lib/python2.7/site-packages/qiime/compare_categories.py", line 146, in compare_categories

    rex(command_args, '%s.r' % method)

  File "/macqiime/anaconda/lib/python2.7/site-packages/qiime/util.py", line 1968, in __call__

    (exit_status, command, stdout, stderr))

burrito.util.ApplicationError: Unacceptable application exit status: 1

command: cd "/Users/stephanieschutz/Google Drive/Christian_Thesis/16s_analysis/cdout29NoNa2760/";  R --slave --args --source_dir /macqiime/anaconda/lib/python2.7/site-packages/qiime/support_files/R -d ./bdiv_even2760/filtered_unweighted_unifrac_dm.txt -m ../16sMapMerged_filtered.txt -c Species -o 16s_adonis_out -n 999  < /macqiime/anaconda/lib/python2.7/site-packages/qiime/support_files/R/adonis.r

stdout:

stderr:

Error in model.frame.default(formula, data, drop.unused.levels = TRUE) : 

  invalid type (NULL) for variable 'qiime.data$map[[opts$category]]'

Calls: adonis -> model.frame -> model.frame.default

Execution halted

Do you have any other suggestions? I have attached my distance matrix. 

Thank you so much!

[Jai Ram Rideout]

Hi Christian,

That's strange, I ran the command you posted with the filtered distance matrix and original mapping file you attached, and the command completed successfully for me. Here's the command I ran:

compare_categories.py --method adonis -i filtered_unweighted_unifrac_dm.txt -m 16sMapMerged.txt -c Species -o 16s_adonis_out -n 999

I have the same version of MacQIIME as you (1.9.1). Perhaps we have different versions of R or the vegan package? I have R 3.1.1 and vegan 2.2.1.

Anyways, I've uploaded the results so that you can move forward with your analyses.

Best,

Jai

[Christian Gray]

That must be it. Unfortunately, I have to run this a couple more times. I've looked for quite some time, but I have not been able to find a way to install the old versions of either R or vegan. Although it may be easiest to have someone else run the next 2 distance matrices. If you wouldn't mind, I would be very appreciative. Unless you have a good way to install the same packages that you have -- I am not very familiar with all of this unfortunately.

Thanks for your help Jai!

Christian

[Greg Caporaso]

Hi Christian, 

Would you be able to download the QIIME 2 VirtualBox and run the commands there? That will have the correct versions of R and vegan in it. 

Best,

Greg