Loading large reference genome

[Marisa Miller]

Hello,

  I am trying to load a large reference genome (hexaploid wheat genome, ~10G file) and it is taking quite a while. The memory usage of IGV goes up to 9320M when trying to load. I am wondering if there are any strategies to make the loading time faster or to increase memory to IGV? My laptop has 16G total memory.

   Thank you,

      Marisa Miller

[Jim Robinson]

Hi,

When you say genome what are you referring to exactly?  A fasta file,  a gff or gtf file,  or something else?

The fasta file is not the issue, as it is indexed.  It is most likely the annotation file (gff, gtf, bed).   If that is the case,  there are 2 strategies.  (1) reduce the size of the file by extracting a subset of the key features, what you keep depends on the file type to some extent.   Or (2) don't create a ".genome" file,  instead load the fasta and an INDEXED annotation file separately.    I recommend tabix for indexing the annotation file, its available in the samtools package, but you can also use IGV.   To do it in igv first load the fasta (Genomes > Load genome from file...   > select fasta),  then open the igvtools window from the Tools menu.   Use the pulldown to select index and select your annotation file.   Running this will produce an index file with an "idx" extension.   Keep this file co-located with the annotation file, then load the annotation file from the file menu as you normally would.   The index will be found automatically.

Jim

--

---
You received this message because you are subscribed to the Google Groups "igv-help" group.
To unsubscribe from this group and stop receiving emails from it, send an email to igv-help+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/igv-help/b4a5309c-b58b-46bb-9280-d5423333068e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Marisa Miller]

Hi Jim,

  I had originally created a .genome file using a fasta file (sorted and indexed using samtools sort and faidx), and I have an annotation file in .gtf format. As you suggested I indexed the .gtf file using samtools, and am currently trying to load the genome (the .fa file) and then load the annotation file separately. However, when I try to load the .fa file (~10GB) IGV just times out and the file never loads. Do you have any suggestions for loading this file?

   Thanks,

     Marisa

[Jim Robinson]

Is the fasta index is in the same directory as the fasta file,  with the standard name (fast file name + .fai)?    How are you loading the fasta file (exact menu option, send a screenshot if needed).

To view this discussion on the web visit https://groups.google.com/d/msgid/igv-help/939f391c-c094-4b4b-a545-22522e426e71%40googlegroups.com.

[Marisa Miller]

Hi Jim,
  It is is the same directory and the file name is the exact same as the .fa file but with the addition of .fai to the end (so .fa.fai). I am loading the file (not a .genome file) from Genomes > load genome from file.

    Thanks,
     Marisa

You received this message because you are subscribed to a topic in the Google Groups "igv-help" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/igv-help/3lL09EkGrAo/unsubscribe.
To unsubscribe from this group and all its topics, send an email to igv-help+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/igv-help/548C776A.8080205%40broadinstitute.org.

[Jim Robinson]

There's nothing obvious that would cause that, the size of the file is not an issue.  How many sequences are in this genome (chromosomes, contings, whatever)?

To view this discussion on the web visit https://groups.google.com/d/msgid/igv-help/548D9DD7.7030809%40gmail.com.

[Marisa Miller]

Hi Jim,
  In the fasta file, it looks like there is 10776707 (at least based on grep -c ">").

I have another question for you as well. I have successfully loaded the wheat mitochondrial genome into IGV (http://www.ncbi.nlm.nih.gov/nuccore/81176508), but when I try to load the .gff file (converted from genbank to gff) I cannot see any annotation appear. Do you have any suggestions on how to get the .gff file to load properly?

     Thanks,
       Marisa

To view this discussion on the web visit https://groups.google.com/d/msgid/igv-help/548DFEE8.1070600%40broadinstitute.org.

[Jim Robinson]

That might be the issue,  how large is the fai file?  IGV wasn't really designed for this many contigs.

To view this discussion on the web visit https://groups.google.com/d/msgid/igv-help/548F31DC.8070408%40gmail.com.

[Marisa Miller]

The fai file is ~500MB.

To view this discussion on the web visit https://groups.google.com/d/msgid/igv-help/548F4524.6060603%40broadinstitute.org.

[Jim Robinson]

That's huge.  Again, IGV isn't designed for this case,  however other people have used it with some success.   I suggest you start IGV with as much memory as possible.  How are you launching it now?

Jim

To view this discussion on the web visit https://groups.google.com/d/msgid/igv-help/548F56F1.8020402%40gmail.com.

[Marisa Miller]

I am on a Mac so I have tried using the Mac app, as well as the "launch with 10 GB" java web start. I have 16GB total memory on my computer, so I'm not sure it will be enough for this task.

   Thanks,
    Marisa

To view this discussion on the web visit https://groups.google.com/d/msgid/igv-help/548F5759.3090705%40broadinstitute.org.

[Jim Robinson]

The "launch with 10 GB"  should definitely be more than enough memory to load the fasta. 

I am a bit confused, however.   The genome referenced below has a single sequence,  is this what you are trying to load?

I am on a Mac so I have tried using the Mac app, as well as the "launch with 10 GB" java web start. I have 16GB total memory on my computer, so I'm not sure it will be enough for this task.

   Thanks,
    Marisa

....  mitochondrial genome into IGV (http://www.ncbi.nlm.nih.gov/nuccore/81176508),

[Marisa Miller]

Hi Jim,
  I am working with both the nuclear and mt genomes of wheat. The one you linked below is a different genome than the large nuclear one (ftp://ftpmips.helmholtz-muenchen.de/plants/wheat/IWGSC/genome_assembly/genome_arm_assemblies_CLEANED/).

The question I asked a couple of messages before was about loading the .gff file for the mt genome. When I convert the genbank file for the mt genome into a .gff file and try to load it, I still am not able to see any of the annotation in IGV.

   Thanks,
     Marisa

--

---
You received this message because you are subscribed to a topic in the Google Groups "igv-help" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/igv-help/3lL09EkGrAo/unsubscribe.
To unsubscribe from this group and all its topics, send an email to igv-help+u...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/igv-help/548F5AD3.4020508%40broadinstitute.org.

[Jim Robinson]

OK.  RE the mt genome,  just use the genbank file (extension .gbk, rename if neccessary).   It includes both sequence and annotation.  Load it from the genome menu.

RE the wheat assembly,  IGV might not be the right tool for unfinished assemblies like this with that many contigs. 

Jim

You received this message because you are subscribed to the Google Groups "igv-help" group.

To unsubscribe from this group and stop receiving emails from it, send an email to igv-help+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/igv-help/548F5F8C.8060308%40gmail.com.

[Marisa Miller]

Hi Jim,

Sorry to reply to the same e-mail after quite a while, but I am just getting back to using IGV and have run into an issue.

When I load a .gbk mitochondrial genome from the genome menu (I do not create a .genome file) and then try to load a sorted and indexed bam file I am getting the error " 1_cs_mt.ab.sorted.bam
does not contain any sequence names which match the current genome." I know this can be fixed by placing an alias file in users/admin/igv/genomes, which I have done. However, this is not fixing the issue.

Do you have any recommendations for how to fix this problem? I have attached all the relevant files here for your information.

Thank you,
Marisa

I've linked 1 file to this email:

1_cs_mt.ab.sorted.bam(17.1 MB)Boxhttps://app.box.com/s/xab7zwkf9ekteyxikl96aq20qy09afhr

Mozilla Thunderbird makes it easy to share large files over email.

To view this discussion on the web visit https://groups.google.com/d/msgid/igv-help/548F742E.4080904%40broadinstitute.org.

[Jim Robinson]

Change the sequence name in your gbk file to match the name in your bam file.

Jim

To view this discussion on the web visit https://groups.google.com/d/msgid/igv-help/54EE098A.8080402%40gmail.com.