converting txt to biom file

[Samir Ranjitkar]

Hi,

I have converted the biom file into txt file and then open in excel file. i did some editing in excel file and now want to convert back  to biom file. I first convert the excel file into tab delimited text file. Then I run the following command in QIIME (1.8 version) but got the error as follows;

MacQIIME Macintosh-2:new_maize $ biom convert -i ucrsss_open_reference_maize/test.txt -o ucrsss_open_reference_maize/test.biom --table-type "otu table" --process-obs-metadata taxonomy

Traceback (most recent call last):
  File "/macqiime/bin/pyqi", line 5, in <module>
    pkg_resources.run_script('pyqi==0.3.1', 'pyqi')
  File "/macqiime/lib/python2.7/site-packages/setuptools-0.9.8-py2.7.egg/pkg_resources.py", line 540, in run_script
  File "/macqiime/lib/python2.7/site-packages/setuptools-0.9.8-py2.7.egg/pkg_resources.py", line 1455, in run_script
  File "/macqiime/lib/python2.7/site-packages/pyqi-0.3.1-py2.7.egg/EGG-INFO/scripts/pyqi", line 177, in <module>
    optparse_main(cmd_obj, argv[1:])
  File "/macqiime/lib/python2.7/site-packages/pyqi-0.3.1-py2.7.egg/pyqi/core/interfaces/optparse/__init__.py", line 276, in optparse_main
    result = optparse_cmd(local_argv[1:])
  File "/macqiime/lib/python2.7/site-packages/pyqi-0.3.1-py2.7.egg/pyqi/core/interface.py", line 40, in __call__
    cmd_result = self.CmdInstance(**cmd_input)
  File "/macqiime/lib/python2.7/site-packages/pyqi-0.3.1-py2.7.egg/pyqi/core/command.py", line 137, in __call__
    result = self.run(**kwargs)
  File "/macqiime/lib/python2.7/site-packages/biom/commands/table_converter.py", line 217, in run
    raise CommandError(convert_error_msg)
pyqi.core.exception.CommandError: Input does not look like a classic table. Did you forget to specify that a classic table file should be created from a BIOM table file?

I have attached both the excel and text file for your reference.

Please advice me.

Best regards

Samir

[Jose Navas]

Hi Samir,

Your test.txt file is not a classic OTU table. You have a column for each of the taxonomy levels. However, the classic OTU table format should have only a single column called "taxonomy" which contains a single string ";"-separated with all the taxonomic levels.

Cheers,

--

---
You received this message because you are subscribed to the Google Groups "Qiime Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to qiime-forum...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Jose Navas

[Samir Ranjitkar]

Dear Jose Antonio,

Thanks for your prompt suggestion.  I made the changes in the excel file as you have suggested and converted to text delimited file and run the code again but I am getting the similar error message as follows;

MacQIIME macintosh-2:new_maize $ biom convert -i ucrsss_open_reference_maize/testnew.txt -o ucrsss_open_reference_maize/testnew.biom --table-type "otu table" --process-obs-metadata taxonomy

Traceback (most recent call last):
  File "/macqiime/bin/pyqi", line 5, in <module>
    pkg_resources.run_script('pyqi==0.3.1', 'pyqi')
  File "/macqiime/lib/python2.7/site-packages/setuptools-0.9.8-py2.7.egg/pkg_resources.py", line 540, in run_script
  File "/macqiime/lib/python2.7/site-packages/setuptools-0.9.8-py2.7.egg/pkg_resources.py", line 1455, in run_script
  File "/macqiime/lib/python2.7/site-packages/pyqi-0.3.1-py2.7.egg/EGG-INFO/scripts/pyqi", line 177, in <module>
    optparse_main(cmd_obj, argv[1:])
  File "/macqiime/lib/python2.7/site-packages/pyqi-0.3.1-py2.7.egg/pyqi/core/interfaces/optparse/__init__.py", line 276, in optparse_main
    result = optparse_cmd(local_argv[1:])
  File "/macqiime/lib/python2.7/site-packages/pyqi-0.3.1-py2.7.egg/pyqi/core/interface.py", line 40, in __call__
    cmd_result = self.CmdInstance(**cmd_input)
  File "/macqiime/lib/python2.7/site-packages/pyqi-0.3.1-py2.7.egg/pyqi/core/command.py", line 137, in __call__
    result = self.run(**kwargs)
  File "/macqiime/lib/python2.7/site-packages/biom/commands/table_converter.py", line 217, in run
    raise CommandError(convert_error_msg)
pyqi.core.exception.CommandError: Input does not look like a classic table. Did you forget to specify that a classic table file should be created from a BIOM table file?

I have also attached the corrected excel file and text delimited file for your reference. Please suggest me again.

Best regards

Samir Ranjitkar

[Jose Navas]

Hi Samir,

You still have "tab" characters in your OTU table. In order to be correct it has to be a single string (without the " characters) and no tabs and/or spaces between the different taxonomic levels, just a ";".

E.g. the first row of your OTU table should be like this:

357765 0.000604961 0 0 0 k__Bacteria;p__Firmicutes;c__Clostridia;o__Clostridiales;f__;g__;s__

Hope this helps!

--

---
You received this message because you are subscribed to the Google Groups "Qiime Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to qiime-forum...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Jose Navas

[Samir Ranjitkar]

Dear Jose Antonio,

Thanks again. I removed the space as you have suggested but I still get the same kind of error message as follows;

MacQIIME macintosh-2:new_maize $ biom convert -i ucrsss_open_reference_maize/testnewnew.txt -o ucrsss_open_reference_maize/testnewnew.biom --table-type "otu table" --process-obs-metadata taxonomy

Traceback (most recent call last):

  File "/macqiime/bin/pyqi", line 5, in <module>
    pkg_resources.run_script('pyqi==0.3.1', 'pyqi')
  File "/macqiime/lib/python2.7/site-packages/setuptools-0.9.8-py2.7.egg/pkg_resources.py", line 540, in run_script
  File "/macqiime/lib/python2.7/site-packages/setuptools-0.9.8-py2.7.egg/pkg_resources.py", line 1455, in run_script
  File "/macqiime/lib/python2.7/site-packages/pyqi-0.3.1-py2.7.egg/EGG-INFO/scripts/pyqi", line 177, in <module>
    optparse_main(cmd_obj, argv[1:])
  File "/macqiime/lib/python2.7/site-packages/pyqi-0.3.1-py2.7.egg/pyqi/core/interfaces/optparse/__init__.py", line 276, in optparse_main
    result = optparse_cmd(local_argv[1:])
  File "/macqiime/lib/python2.7/site-packages/pyqi-0.3.1-py2.7.egg/pyqi/core/interface.py", line 40, in __call__
    cmd_result = self.CmdInstance(**cmd_input)
  File "/macqiime/lib/python2.7/site-packages/pyqi-0.3.1-py2.7.egg/pyqi/core/command.py", line 137, in __call__
    result = self.run(**kwargs)
  File "/macqiime/lib/python2.7/site-packages/biom/commands/table_converter.py", line 217, in run
    raise CommandError(convert_error_msg)

pyqi.core.exception.CommandError: Input does not look like a classic table. Did you forget to specify that a classic table file should be created from a BIOM table file?

I again attached the excel and tab delimited file for your reference. Please suggest me.

Best regards

Samir Ranjitkar

 

On Thursday, 28 August 2014 15:45:23 UTC+2, Samir Ranjitkar wrote:

[Jose Navas]

Hi Samir,

There was still tab characters in your txt file. I recommend to fix the file in a text editor, rather than on Excel.

As a reference, here is the fixed testnewnew.txt table; so you can see the differences between your file and the one that I'm attaching.

Cheers,

--

---
You received this message because you are subscribed to the Google Groups "Qiime Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to qiime-forum...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Jose Navas

[Samir Ranjitkar]

Dear Jose Antonio,

Finally it worked. Thanks a lot for your kind support and prompt suggestion.

Best regards

Samir Ranjitkar 

On Thursday, 28 August 2014 15:45:23 UTC+2, Samir Ranjitkar wrote:

[Jose Navas]

Glad it worked!

Happy QIIME-ing!

--

---
You received this message because you are subscribed to the Google Groups "Qiime Forum" group.
To unsubscribe from this group and stop receiving emails from it, send an email to qiime-forum...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Jose Navas

[Erin McKenney]

Hello,

I am also having trouble converting a taxon table to .BIOM format.

I have checked the invisibles, and made sure that the format mirrors other taxa_summary tables that I have been able to convert successfully.

Could you please help me to format this table correctly?

Thanks,

Erin

[John Chase]

Hello, 

I was able to convert the small table to biom with the following command:

biom convert -i Taxon-table.txt -o output-table.biom --table-type "OTU table" --to-hdf5

However the large table has duplicate IDs so I was not able to convert it because there are duplicate observations. I would recommend removing these or changing them and running it again.

John

[Erin McKenney]

Thanks John,

I identified the duplication and converted my table to biom.

When I tried to summarize_taxa_through_plots , I got this ValueError: BIOM table does not contain any observation metadata (e.g., taxonomy). You can add metadata to it using the 'biom add-metadata' command.

I tried to add taxonomy using the following command:

$ biom add-metadata -i taxon-table.biom -o taxon-table+md.biom --observation-metadata-fp Taxon-table_QIIME.txt --observation-header Taxon

but when I tried to summarize_taxa_through_plots using the taxon-table+md.biom as input, I got this error:

Traceback (most recent call last):

  File "/macqiime/QIIME/bin/summarize_taxa_through_plots.py", line 132, in <module>

    main()

  File "/macqiime/QIIME/bin/summarize_taxa_through_plots.py", line 129, in main

    status_update_callback=status_update_callback)

  File "/macqiime/lib/python2.7/site-packages/qiime/workflow/downstream.py", line 722, in run_summarize_taxa_through_plots

    close_logger_on_success=close_logger_on_success)

  File "/macqiime/lib/python2.7/site-packages/qiime/workflow/util.py", line 116, in call_commands_serially

    raise WorkflowError, msg

qiime.workflow.util.WorkflowError: 

*** ERROR RAISED DURING STEP: Summarize Taxonomy

Command run was:

 /macqiime/bin/python /macqiime/QIIME/bin//summarize_taxa.py -i /Users/erinmckenney/Documents/Projects/Hgriseus/QIIME/taxon-table+md.biom -o /Users/erinmckenney/Documents/Projects/Hgriseus/QIIME/taxa_summary 

Command returned exit status: 1

Stdout:

Stderr

Traceback (most recent call last):

  File "/macqiime/QIIME/bin//summarize_taxa.py", line 203, in <module>

    main()

  File "/macqiime/QIIME/bin//summarize_taxa.py", line 190, in main

    md_identifier)

  File "/macqiime/lib/python2.7/site-packages/qiime/summarize_taxa.py", line 41, in make_summary

    md_identifier)

  File "/macqiime/lib/python2.7/site-packages/qiime/summarize_taxa.py", line 92, in sum_counts_by_consensus

    "Metadata category '%s' not in OTU %s. Can't continue. Did you pass the correct metadata identifier?" % (md_identifier,otu_id)

KeyError: u"Metadata category 'taxonomy' not in OTU k__Bacteria;p__Actinobacteria;c__Acidimicrobiia;o__Acidimicrobiales;f__koll13;g__;s__. Can't continue. Did you pass the correct metadata identifier?"

Could you help me to add the taxonomy metadata so that my biom table is formatted correctly for downstream analyses?

Thanks!

-Erin

[John Chase]

Hi Erin, 

Would you mind attaching the "taxon-table.biom " file as well? I think the error you are seeing is that there are no IDs in the Taxon-table but I would like to look at the OTU table as well.

Thanks, 

John

--

[Erin McKenney]

Hi John,

I've attached taxon-table.biom.

Thanks,

Erin

[John Chase]

Hi, 

Yes, I believe the issue is that the taxonomy is not associated with an observation ID in the taxonomy map. Take a look a the attached file for an example of what I am talking about. This is the same data shown on the biom website regarding adding metadata.

John 

[Erin McKenney]

I see.

The taxonomic data I am using summarizes Ion Torrent sequencing results, so I don't have observation IDs.

Thanks,
Erin