anvi-profile has trouble with indexed bam files

[Kiera Sullivan]

Hi all!

I am attempting to profile some bam files using anvi'o (v6.2) and I am having some trouble with the anvi-profile step. I generated the bam files using bowtie2, then sorted/indexed them using anvi-init-bam. This generated a file index ending in .bam.bai.

However, when I try to run anvi-profile with my contigs database and the bam file, I get an error message. I tried running this script with both the .bam and .bam.bai files to see what would happen, and here are the error messages I got from each:

.bam: Config Error: It seems the BAM file is not indexed. See 'anvi-init-bam' script.

.bam.bai: [E::hts_hopen] Failed to open file /home/ksullivan/nob-genomes/metzyme/BIN41.bam-sorted.bam.bai

[E::hts_open_format] Failed to open file "/home/ksullivan/nob-genomes/metzyme/BIN41.bam-sorted.bam.bai" : Exec format error

Traceback (most recent call last):

  File "/home/ksullivan/miniconda3/envs/anvio-6.2/bin/anvi-profile", line 93, in <module>

    main(args)

  File "/home/ksullivan/miniconda3/envs/anvio-6.2/lib/python3.6/site-packages/anvio/terminal.py", line 748, in wrapper

    program_method(*args, **kwargs)

  File "/home/ksullivan/miniconda3/envs/anvio-6.2/bin/anvi-profile", line 33, in main

    profiler.BAMProfiler(args)._run()

  File "/home/ksullivan/miniconda3/envs/anvio-6.2/lib/python3.6/site-packages/anvio/profiler.py", line 267, in _run

    self.init_profile_from_BAM()

  File "/home/ksullivan/miniconda3/envs/anvio-6.2/lib/python3.6/site-packages/anvio/profiler.py", line 443, in init_profile_from_BAM

    self.bam = bamops.BAMFileObject(self.input_file_path)

  File "pysam/libcalignmentfile.pyx", line 742, in pysam.libcalignmentfile.AlignmentFile.__cinit__

  File "pysam/libcalignmentfile.pyx", line 941, in pysam.libcalignmentfile.AlignmentFile._open

OSError: [Errno 8] could not open alignment file `/home/ksullivan/nob-genomes/metzyme/BIN41.bam-sorted.bam.bai`: Exec format error

If anyone has any suggestions on how to fix this and profile my bam files, please let me know! 

[A. Murat Eren]

Hi Kiera,

Can you please install anvi'o v7 and try this again? Sorry for suggesting this as I know it is not fun to deal with installations, but I promise the installation of v7 is very straightforward, and will likely solve your issue.

Best,

--

A. Murat Eren (Meren) | he/him
http://merenlab.org :: twitter

--
Anvi'o Paper: https://peerj.com/articles/1319/
Project Page: http://merenlab.org/projects/anvio/
Code Repository: https://github.com/meren/anvio
---
You received this message because you are subscribed to the Google Groups "Anvi'o" group.
To unsubscribe from this group and stop receiving emails from it, send an email to anvio+un...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/anvio/7b520749-770f-48a2-9125-596e0251c466n%40googlegroups.com.

[Kiera Sullivan]

Hi Meren,

Thanks so much for the quick response! 

I downloaded v7, but I still get the same messages when I try to run anvi-profile. I can't quite figure out what's going on, but I think there might be an issue with the format of my bam file? Would re-indexing/sorting it using the new version of anvi'o help with this?

Best,

Kiera

[A. Murat Eren]

It is certainly worth trying to sort/index your raw BAM files using the new version. Which mapping software did you use to generate these BAM files, Kiera?

--

A. Murat Eren (Meren) | he/him
http://merenlab.org :: twitter

To view this discussion on the web visit https://groups.google.com/d/msgid/anvio/3f9d42c6-c4d6-43ab-b093-34f82bb7b27bn%40googlegroups.com.

[Evan Kiefl]

To me, it seems that the bam is corrupt, or it's format is incompatible with pysam. Also curious how the bam was generated.

To view this discussion on the web visit https://groups.google.com/d/msgid/anvio/3f9d42c6-c4d6-43ab-b093-34f82bb7b27bn%40googlegroups.com.

[Kiera Sullivan]

I generated the bam file using bowtie2 and samtools, but even after sorting/indexing the raw file using the new version of anvi'o I am still having trouble getting it to work. I am planning to start from scratch and generate a new bam file to hopefully resolve this - is there a different program that might be more compatible with anvi'o that either of you would recommend to do this?

Thanks again for all the help!

[Ana Belen Garcia Martin]

Hi Kiera, hi everyone!

I used bwa for bam profiling using anvi'o v6.2 without any problems. I would give a trial to bwa and further analysis using anvi'o v7.

I hope this helps.

Stay healthy!

Ana Garcia

To view this discussion on the web visit https://groups.google.com/d/msgid/anvio/ec6e6ed0-1c17-4b3a-9f86-aa11e1258e61n%40googlegroups.com.

[Kiera Sullivan]

Hi all,

Thanks for the suggestions! Good news first - I started from scratch and remade my bam file using bowtie2, which at least helped me get past the first error message I was getting. Unfortunately I am still having some issues with the profiling step. I have used anvi'o to reformat every fasta file involved in making the contigs database and the bam file, but now I am getting this message:

Config Error: At least one contig name in your BAM file does not match contig names stored in

              the contigs database. For instance, this is one contig name found in your BAM  

              file: 'c_000000000220', and this is another one found in your contigs database:

              'c_000000000151'. You may be using an contigs database for profiling that has  

              nothing to do with the BAM file you are trying to profile, or you may have     

              failed to fix your contig names in your FASTA file prior to mapping, which is  

              described here: http://goo.gl/Q9ChpS 

I am a bit confused since the contigs appear to be named following the same format and I have double checked to make sure that I am using the right contigs database, so I really don't know why the contig names don't match up. Could this still have something to do with the format of the bam file? I am still very new to anvi'o (and coding in general), so I apologize if this is something that has a simple answer.  

[A. Murat Eren]

Hi Kiera,

On Fri, Feb 19, 2021 at 1:46 PM Kiera Sullivan <ksul...@ucsb.edu> wrote:

I am a bit confused since the contigs appear to be named following the same format and I have double checked to make sure that I am using the right contigs database, so I really don't know why the contig names don't match up. Could this still have something to do with the format of the bam file? I am still very new to anvi'o (and coding in general), so I apologize if this is something that has a simple answer.

There is a simple and 100% accurate answer for this :) Either you have not used the same contigs.fa file for Bowtie2 mapping and for anvi-gen-contigs-database, OR, you are not using the correct contigs.db to run anvi-profile for a given BAM file.

Best,

[Kiera Sullivan]

Ah, thank you so much! I did some digging and found where the problem was and now everything works great :)