Error Generating Summary table

[Cody Sheik]

Hi All, 

Trying to generate a summary table. Background here: I have a single assembly from 10 samples. I binned with MetaBat instead of Concoct, going to see how they compare. Commands leading up to summarize. I'm running things on a cluster with CentOS operating system. I can query the database, like for the CPR genome prediction but can't generate a summary.

anvi-merge Mapped_Reads/*-ANVIO_PROFILE/RUNINFO.cp -o SAMPLES-MERGED -c contigs.db --skip-hierarchical-clustering --skip-concoct-binning

anvi-import-collection BinNames.txt -p SAMPLES-MERGED/PROFILE.db -c contigs.db --contigs-mode -C MetaBat 

anvi-summarize -p SAMPLES-MERGED/PROFILE.db -c contigs.db -o SAMPLES-SUMMARY -C MetaBat

Thanks in advance! 

-C

Error encountered:

 

$anaconda2/envs/anvio/lib/python2.7/site-packages/numpy/core/_methods.py:59: RuntimeWarning: Mean of empty slice.

  warnings.warn("Mean of empty slice.", RuntimeWarning)

$anaconda2/envs/anvio/lib/python2.7/site-packages/numpy/core/_methods.py:70: RuntimeWarning: invalid value encountered in double_scalars

  ret = ret.dtype.type(ret / rcount)

Traceback (most recent call last):

  File "$anaconda2/envs/anvio/bin/anvi-summarize", line 4, in <module>

    __import__('pkg_resources').run_script('anvio==2.0.2', 'anvi-summarize')

  File "build/bdist.linux-x86_64/egg/pkg_resources/__init__.py", line 719, in run_script

  File "build/bdist.linux-x86_64/egg/pkg_resources/__init__.py", line 1504, in run_script

  File "$anaconda2/envs/anvio/lib/python2.7/site-packages/anvio-2.0.2-py2.7-linux-x86_64.egg/EGG-INFO/scripts/anvi-summarize", line 45, in <module>

    summary.process()

  File "$anaconda2/envs/anvio/lib/python2.7/site-packages/anvio-2.0.2-py2.7-linux-x86_64.egg/anvio/summarizer.py", line 107, in process

    self.init_collection_profile(collection_dict)

  File "$anaconda2/envs/anvio/lib/python2.7/site-packages/anvio-2.0.2-py2.7-linux-x86_64.egg/anvio/dbops.py", line 828, in init_collection_profile

    bin_coverages_in_sample = sum([coverage_table_data[split_name][sample] for split_name in collection[bin_id]])

KeyError: 'contig-116_341833_split_00001'

[David Needham]

Hello,

I'm sorry, I can't really help. : (

 I, too, have received an error while calling anvi-summarize after importing bins from Metabat (see below). I can see the new bins with anvi-refine, but saving doesn't work (summary folder is empty).

 

anvi-merge dura_*/RUNINFO.cp ano_*/RUNINFO.cp -c contigs.db -o MERGED_PROFILE --skip-hierarchical-clustering --skip-concoct-binning &

anvi-import-collection metabat_for_anvio -p MERGED_PROFILE/PROFILE.db -c contigs.db -C metabat --contigs-mode

anvi-summarize -p MERGED_PROFILE/PROFILE.db -c contigs.db -o metabat_summarize -C metabat

Traceback (most recent call last):

  File "/usr/local/bin/anvi-summarize", line 45, in <module>

    summary.process()

  File "/usr/local/lib/python2.7/dist-packages/anvio/summarizer.py", line 107, in process

    self.init_collection_profile(collection_dict)

  File "/usr/local/lib/python2.7/dist-packages/anvio/dbops.py", line 828, in init_collection_profile

    bin_coverages_in_sample = sum([coverage_table_data[split_name][sample] for split_name in collection[bin_id]])

KeyError: 'k255_2690624_split_00001'

Best,

David

[A. Murat Eren]

Hi Cody, hi David,

I apologize for the late response. I was traveling. I will try to reproduce it using what I have, if I can't, I will ask for your help and leave it for tomorrow.

Best wishes,

--

A. Murat Eren (meren)
http://merenlab.org :: gpg

--
Anvi'o Paper: https://peerj.com/articles/1319/
Project Page: http://merenlab.org/projects/anvio/
Code Repository: https://github.com/meren/anvio
---
You received this message because you are subscribed to the Google Groups "Anvi'o" group.
To unsubscribe from this group and stop receiving emails from it, send an email to anvio+un...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/anvio/45f4852f-75af-4158-851d-c73bf247e41b%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[A. Murat Eren]

I tried a lot of things, but I simply can't reproduce this one :/

I would be very happy if either of you can share their contigs database and profile database so I can try to reproduce the issue. I apologize for this. You can simply share the merged PROFILE.db, Metabat input file, and the CONTIGS.db via Dropbox, and I can try to find out what is wrong.

Best,

--

A. Murat Eren (meren)
http://merenlab.org :: gpg

[A. Murat Eren]

Abort! Abort!

While reading the code a bit more carefully I figured what is wrong. No need to send me anything!

I will be back with a solution.

Best,

--

A. Murat Eren (meren)
http://merenlab.org :: gpg

[A. Murat Eren]

OK. Here is the problem, and solution:

https://github.com/meren/anvio/issues/380

If you update your repository and re-install anvi'o from master, this problem should go away.

If you happen to test this, please let me know if it works or not.

Thank you very much for reporting this, Cody (and I apologize for the inconvenience on behalf of Tom, because, you know)!

--

A. Murat Eren (meren)
http://merenlab.org :: gpg

[David Needham]

Hi Meren,

We updated, but unfortunately I am still receiving an error (see below). Do I need to re-perform the merging step or should these commands have been sufficient? Would you like to see the files to try to reproduce the error?

Thanks!

David

I ran:

anvi-script-upgrade-profile-db-v15-to-v16 megahit_contigs_fa_db_MERGED_PROFILE_2500/PROFILE.db

anvi-script-update-runinfo-variable megahit_contigs_fa_db_MERGED_PROFILE/RUNINFO.mcp —variable "blank" —set-bool false

anvi-import-collection metabat_for_anvio -p megahit_contigs_fa_db_MERGED_PROFILE_2500/PROFILE.db -c megahit.contigs.fa.db -C metabat_2500_superspecific --contigs-mode

anvi-summarize -p megahit_contigs_fa_db_MERGED_PROFILE_2500/PROFILE.db -c megahit.contigs.fa.db -o metabat_summarize_2500_superspecific -C metabat_2500_superspecific

Traceback (most recent call last):

  File "/usr/local/bin/anvi-summarize", line 4, in <module>

    __import__('pkg_resources').run_script('anvio==2.0.2', 'anvi-summarize')

  File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 726, in run_script

    self.require(requires)[0].run_script(script_name, ns)

  File "/usr/local/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 1484, in run_script

    exec(code, namespace, namespace)

  File "/usr/local/lib/python2.7/dist-packages/anvio-2.0.2-py2.7-linux-x86_64.egg/EGG-INFO/scripts/anvi-summarize", line 45, in <module>

    summary.process()

  File "/usr/local/lib/python2.7/dist-packages/anvio-2.0.2-py2.7-linux-x86_64.egg/anvio/summarizer.py", line 103, in process

    collection_dict, bins_info_dict = self.init_collection_profile(self.collection_name)

  File "/usr/local/lib/python2.7/dist-packages/anvio-2.0.2-py2.7-linux-x86_64.egg/anvio/dbops.py", line 842, in init_collection_profile

    bin_coverages_in_sample = sum([coverage_table_data[split_name][sample] for split_name in collection[bin_id]])

KeyError: 'k255_2690624_split_00001'

To view this discussion on the web visit https://groups.google.com/d/msgid/anvio/CAK5_LacDtb16F43Vv8OZNX2RgpwEhXro4uVqCCoYXzNvvjqTQg%40mail.gmail.com.

[A. Murat Eren]

Hi David,

I guess I should see the files :/ Thank you very much in advance.

I will need the contigs db, profile db, and the collection you have imported please.

Best wishes,

--

A. Murat Eren (meren)
http://merenlab.org :: gpg

To view this discussion on the web visit https://groups.google.com/d/msgid/anvio/CAK8YXU%3Dp%3D5N6EDJnXoC9weE5JU0jnq2Luh%2B%2By6tWdzm5NHhOhw%40mail.gmail.com.

[A. Murat Eren]

Thank you very much for the test files, David!

After another set of commits, I think we are good to address this issue. Please update your local copy, and let me know if there is an issue (also press refresh the first time you see the interactive interface with this updated copy, because just yesterday Ozcan improved the speed of the interactive interface about 400%!).

Also, a note for everyone who are trying to summarize imported collections to identify poor bins: you can also anvi-interactive to do that by running it in 'collection mode', which is a new feature. All you need to do is this:

anvi-interactive -p PROFILE.db -c CONTIGS.db -C COLLECTION_NAME

Best,

--

A. Murat Eren (meren)
http://merenlab.org :: gpg

[David Needham]

Hi Meren,

It is working for us! Thanks!!

David

To view this discussion on the web visit https://groups.google.com/d/msgid/anvio/CAK5_Ladtj7461gQWyEYMxJEsQBnTwnSdrj-J2McAv6xzVGSudQ%40mail.gmail.com.

[Cody Sheik]

Hi Meren and David, 

Sorry for the late response but seems to be working for us now! I appreciate the time you all spent figuring this out! Sorry for not answering earlier but was hampered by field work and power outages!

Best, 

Cody

[A. Murat Eren]

I'm glad to hear that it's working. Thank you both very much.

--

A. Murat Eren (meren)
http://merenlab.org :: gpg

To view this discussion on the web visit https://groups.google.com/d/msgid/anvio/bd3172d9-0367-4c00-abaa-5af8c94dc98f%40googlegroups.com.