Importing biom files into phyloseq/Rstudio

[Diana]

Hi,

I am new to Rstudio and phyloseq and am trying to import the biom file for further analysis using phyloseq

My file is in json format and I have used it successfully for all my other analyses in qiime 1.9.1.  

Here is the output I get with Rstudio

> otu_and_tax.ps <- import_biom("~/../Desktop/Shared_Folder/otu_json_w_tax.biom")
Error in fromJSON(content, handler, default.size, depth, allowComments,  : 
  invalid JSON input

I then converted the json biom file to a .txt file, but this does not seem to work either.

otu_and_tax.ps <- import_qiime("~/../Desktop/Shared_Folder/otu_tax.txt")
Processing otu/tax file...
Reading file into memory prior to parsing...
Detecting first header line...
Header is on line 2  
Converting input file to a table...
Defining OTU table... 
Parsing taxonomy table...
Error in names(taxlist) <- rownames(x) : 
  'names' attribute [296] must be the same length as the vector [0]
In addition: Warning message:
In `[.data.table`(x, , `:=`(`Consensus Lineage`, NULL)) :
  Adding new column 'Consensus Lineage' then assigning NULL (deleting it).

I've attached the two files.
I am not sure where to go from here. Any suggestions are much appreciated. 

Thanks,
Diana

[Jenya Kopylov]

Hi Diana,

Have you searched through phyloseq github issues regarding importing BIOM files? Here is one that is throwing the same error as yours, perhaps it can give some clues. 

Jenya

[Diana]

Hi Jenya,

Thanks for the link.  I am able to import the mapping file (it is a .txt in same location as the OTU biom and .txt files) into Rstudio - I am using the same file path and readLine function works fine with that script.  When I try to do the same with the OTU biom or .txt file the errors come up.

Diana 

[Diana]

Jenya,

Here is what I have been seeing 

> readLines("~/../Desktop/Shared_Folder/otu_json_w_tax.biom")
[1] "‰HDF

Not sure why the output is %HDF as I have been using json file format (I first had a .uc file from vsearch, then converted that to .txt file, then converted to json with script:   qiime@qiime-190-virtual-box:~/Desktop/RSL_NewAnalyses/vsearch_output$ biom convert -i otu_table.txt -o otu_json.biom --table-type="OTU table" --to-json   ;  I then added the taxonomy information with:   qiime@qiime-190-virtual-box:~/Desktop/RSL_NewAnalyses/vsearch_output$ biom add-metadata -i otu_json.biom -o otu_json_w_tax.biom --observation-metadata-fp repset_assign_tax/repset_denovo_nochimeras_OTUS_tax_assignments.txt --sc-separated taxonomy --observation-header OTUID,taxonomy ) 

To get around this I just converted the biom file to .txt   (biom convert -i otu_json_w_tax.biom -o otu_tax.txt --to-tsv --header-key taxonomy --output-metadata-id "ConsensusLineage")  I then deleted the "#constructed from biom file" header in the first row of the file. 


The readLines function recognizes the path to this txt. file 

> readLines("~/../Desktop/Shared_Folder/otu_tax.txt")
  [1] "#OTU ID\t3\t1G\t12\t8R\t8\t15R\t10R\t2R\t17\t19N\t6\t13N\t1R\t15G\t11G\t10N\t20G\t19G\t14\t14R\t18R\t3G\t2\t5N\t11N\t10\t19\t16\t20\t17N\t12N\t13\t15\t18\t1\t4\t8G\t16G\t9G\t11R\t7\t9N\t11\t17G\t9\t13R\t4G\t5\t17R\tConsensusLineage"                                                                                                                                                                                                                                                                                                     
  [2] "OTU_31\t1119.0\t2.0\t0.0\t1013.0\t945.0\t0.0\t1.0\t505.0\t423.0\t616.0\t1323.0\t0.0\t0.0\t0.0\t1.0\t0.0\t564.0\t216.0\t0.0\t0.0\t308.0\t239.0\t1048.0\t634.0\t0.0\t0.0\t556.0\t0.0\t869.0\t432.0\t2.0\t1.0\t0.0\t765.0\t0.0\t1.0\t422.0\t0.0\t653.0\t0.0\t3.0\t1210.0\t1.0\t501.0\t1501.0\t1.0\t0.0\t959.0\t401.0\tk__Bacteria; p__Firmicutes; c__Clostridia; o__Clostridiales; f__Ruminococcaceae; g__Ruminococcus"                                                                                                                         
etc...                     

When I try to import the output is:

otu_and_tax.ps <- import_qiime("~/../Desktop/Shared_Folder/otu_tax.txt")
Processing otu/tax file...
Reading file into memory prior to parsing...
Detecting first header line...
Header is on line 2  
Converting input file to a table...
Defining OTU table... 
Parsing taxonomy table...
Error in names(taxlist) <- rownames(x) : 
  'names' attribute [296] must be the same length as the vector [0]
In addition: Warning message:
In `[.data.table`(x, , `:=`(`Consensus Lineage`, NULL)) :
  Adding new column 'Consensus Lineage' then assigning NULL (deleting it).

I opened the .txt file in excel and seems ok, so not sure what the error means but will keep trying.  Sorry for the lengthy explanation and thanks for any insights. 

Diana

[Diana]

I was able to successfully import biom file!  I converted a second time to json format as below: 

biom convert -i otu_json_w_tax.biom -o otu_json2_w_tax.biom --table-type="OTU table" --to-json

[Jenya Kopylov]

Hi Diana,

The default output format of the BIOM table in QIIME 1.9.1 is HDF5 (not JSON), likely this is the reason that Rstudio could not read the format until you converted it to JSON.

Let us know if you have other questions, glad you figured it out!

Jenya

[Ru Wang]

hello! 

I am now having the same error with Daina. However, after I tried the biom convert command in qiime, the error remains the same. can you give me some guides about this? any help would be appreciated

Best,

Ru. 

[Jenya Kopylov]

Hi Ru,

Could you please paste:

1.  output of print_qiime_config.py

2.  the biom convert command you launched and the output error

This will help me deduce the issue quicker, thanks!

Jenya