.bai file not found

[Jorge Amigo Lechuga]

we're having trouble when loading a remote bam file through http. we get the following error:
Error encountered querying alignments: htsjdk.samtools.util.RuntimeIOException: java.io.FileNotFoundException: File not found: http://www.randomarchive.com/sample.bam.bai
in fact that's true, since all the .bai files we have are generated by GATK and their name is exactly the same one as the .bam, but changing .bam for .bai, and not appending .bai as probably expected by IGV. but we didn't have no problem until version 2.3.54 (2.3.52 works perfectly), so we think that somewhere in the code IGV is not looking for .bai files other than .bam.bai
may this behaviour be corrected in upcoming versions?

[Jim Robinson]

Hi,
Thanks for the report.  Unfortunately I can't reproduce the problem,  could you try loading the following using "File > Load from URL" and tell me if it succeeds, nor not.

http://www.broadinstitute.org/igvdata/1KG/b37/data/NA06984/alignment/NA06984.mapped.ILLUMINA.bwa.CEU.low_coverage.20120522.bam

Also,  could you send your igv.log file?  Its located in an "igv" directory under the user home.

--

---
You received this message because you are subscribed to the Google Groups "igv-help" group.
To unsubscribe from this group and stop receiving emails from it, send an email to igv-help+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/igv-help/41459a96-e9b6-46ad-ab08-9d0892cdc39e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jorge Amigo Lechuga]

hi Jim,

thanks for such a quick response. your url loads perfectly, so I can't reproduce the problem with it.

the problem with our file is strange, because the bam file loads fine, but when searching for the INO80C gene it gives that error mentioned. here is the igv.log content in case you're able to see why this is happening:
INFO [2015-05-28 20:20:24,913] [Main.java:85]  Startup  IGV Version 2.3.54 (71)05/28/2015 11:13 AM
INFO [2015-05-28 20:20:24,913] [Main.java:86]  Java 1.7.0_80
INFO [2015-05-28 20:20:24,914] [Main.java:87]  Default User Directory: D:\Sistema\mis_documentos
INFO [2015-05-28 20:20:24,914] [Main.java:88]  OS: Windows 8.1
ERROR [2015-05-28 20:20:26,294] [Main.java:151]  Error checking version
java.net.SocketTimeoutException: Read timed out
    at java.net.SocketInputStream.socketRead0(Native Method)
    at java.net.SocketInputStream.read(Unknown Source)
    at java.net.SocketInputStream.read(Unknown Source)
    at java.io.BufferedInputStream.fill(Unknown Source)
    at java.io.BufferedInputStream.read1(Unknown Source)
    at java.io.BufferedInputStream.read(Unknown Source)
    at sun.net.www.http.HttpClient.parseHTTPHeader(Unknown Source)
    at sun.net.www.http.HttpClient.parseHTTP(Unknown Source)
    at sun.net.www.protocol.http.HttpURLConnection.getInputStream(Unknown Source)
    at java.net.HttpURLConnection.getResponseCode(Unknown Source)
    at org.broad.igv.util.HttpUtils.openConnection(HttpUtils.java:694)
    at org.broad.igv.util.HttpUtils.openConnection(HttpUtils.java:600)
    at org.broad.igv.util.HttpUtils.openConnection(HttpUtils.java:596)
    at org.broad.igv.util.HttpUtils.getContentsAsString(HttpUtils.java:124)
    at org.broad.igv.ui.Main.checkVersion(Main.java:124)
    at org.broad.igv.ui.Main.initApplication(Main.java:99)
    at org.broad.igv.ui.Main.main(Main.java:65)
INFO [2015-05-28 20:20:37,052] [GenomeManager.java:130]  Loading genome: http://igv.broadinstitute.org/genomes/hg19.genome
INFO [2015-05-28 20:20:38,686] [GenomeManager.java:177]  Genome loaded.  id= hg19
INFO [2015-05-28 20:20:38,843] [CommandListener.java:91]  Listening on port 60151
INFO [2015-05-28 20:20:44,522] [IGV.java:1639]  Loading 1 resources.
INFO [2015-05-28 20:20:44,523] [TrackLoader.java:113]  Loading resource, path http://www.randomserver.com/sample.processed.bam
ERROR [2015-05-28 20:20:51,325] [HttpUtils.java:813]  Error while testing byte range File not found: http://www.randomserver.com/sample.processed.bam.bai
INFO [2015-05-28 20:20:51,416] [HttpUtils.java:803]  Range-byte request succeeded
ERROR [2015-05-28 20:20:58,657] [AlignmentTileLoader.java:239]  Error loading alignment data
htsjdk.samtools.util.RuntimeIOException: java.io.FileNotFoundException: File not found: http://www.randomserver.com/sample.processed.bam.bai
    at htsjdk.samtools.AbstractBAMFileIndex$IndexStreamBuffer.readFully(AbstractBAMFileIndex.java:653)
    at htsjdk.samtools.AbstractBAMFileIndex$IndexStreamBuffer.readInteger(AbstractBAMFileIndex.java:686)
    at htsjdk.samtools.AbstractBAMFileIndex.readInteger(AbstractBAMFileIndex.java:433)
    at htsjdk.samtools.AbstractBAMFileIndex.<init>(AbstractBAMFileIndex.java:69)
    at htsjdk.samtools.DiskBasedBAMFileIndex.<init>(DiskBasedBAMFileIndex.java:42)
    at htsjdk.samtools.BAMFileReader.getIndex(BAMFileReader.java:256)
    at htsjdk.samtools.BAMFileReader.createIndexIterator(BAMFileReader.java:727)
    at htsjdk.samtools.BAMFileReader.query(BAMFileReader.java:413)
    at htsjdk.samtools.SamReader$PrimitiveSamReaderToSamReaderAdapter.query(SamReader.java:498)
    at htsjdk.samtools.SamReader$PrimitiveSamReaderToSamReaderAdapter.query(SamReader.java:503)
    at org.broad.igv.sam.reader.BAMHttpReader.query(BAMHttpReader.java:139)
    at org.broad.igv.sam.AlignmentTileLoader.loadTile(AlignmentTileLoader.java:124)
    at org.broad.igv.sam.AlignmentDataManager.loadInterval(AlignmentDataManager.java:327)
    at org.broad.igv.sam.AlignmentDataManager$1.run(AlignmentDataManager.java:292)
    at org.broad.igv.util.LongRunningTask.call(LongRunningTask.java:57)
    at java.util.concurrent.FutureTask.run(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)
Caused by: java.io.FileNotFoundException: File not found: http://www.randomserver.com/sample.processed.bam.bai
    at org.broad.igv.util.HttpUtils.openConnection(HttpUtils.java:720)
    at org.broad.igv.util.HttpUtils.openConnection(HttpUtils.java:600)
    at org.broad.igv.util.HttpUtils.openConnection(HttpUtils.java:596)
    at org.broad.igv.util.stream.IGVSeekableHTTPStream.openInputStreamForRange(IGVSeekableHTTPStream.java:155)
    at org.broad.igv.util.stream.IGVSeekableHTTPStream.read(IGVSeekableHTTPStream.java:92)
    at java.io.BufferedInputStream.fill(Unknown Source)
    at java.io.BufferedInputStream.read1(Unknown Source)
    at java.io.BufferedInputStream.read(Unknown Source)
    at htsjdk.samtools.seekablestream.SeekableBufferedStream.read(SeekableBufferedStream.java:100)
    at htsjdk.samtools.AbstractBAMFileIndex$IndexStreamBuffer.readFully(AbstractBAMFileIndex.java:651)
    ... 18 more
INFO [2015-05-28 20:20:58,661] [MessageUtils.java:60]  <html>Error encountered querying alignments: htsjdk.samtools.util.RuntimeIOException: java.io.FileNotFoundException: File not found: http://www.randomserver.com/sample.processed.bam.bai
INFO [2015-05-28 20:21:05,220] [HttpUtils.java:803]  Range-byte request succeeded

regards,
jorge.

[Jim Robinson]

OK, thanks for the log.   Just to be sure,  if you paste this url in a web browser does it download it?

http://www.randomserver.com/sample.processed.bam.bai

--

---
You received this message because you are subscribed to the Google Groups "igv-help" group.
To unsubscribe from this group and stop receiving emails from it, send an email to igv-help+u...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/igv-help/45dcb812-b8ff-401c-8d15-2c26994ef247%40googlegroups.com.

[Jorge Amigo Lechuga]

no it doesn't. we have both
http://www.randomserver.com/sample.processed.bam
and
http://www.randomserver.com/sample.processed.bai
but not
http://www.randomserver.com/sample.processed.bam.bai
since the .bai index is automatically generated by GATK, which is the software we process the original .bam file with.

the absence of the .bam.bai file but the presence of the just .bai file instead was not a problem till version 2.3.54, so I'm just guessing there must be some new code that forces the requirement of a .bam.bai file which I would kindly ask to be removed or, at least, to be complemented.

[Jim Robinson]

Ahh, yes, apologies I missed that critical detail.   Yes there was a change,  we now use the "htsjdk"  to read the index file over the network,  in the past we downloaded it locally.   I'm guessing they don't have the code to try both as IGV does.    Anyway now that I know the problem  I can work on a fix.

Having these 2 conventions (.bai and .bam.bai) is no end of problems,  the  proponents of each  just couldn't agree for whatever reason and so we're stuck with it.

--

---
You received this message because you are subscribed to the Google Groups "igv-help" group.
To unsubscribe from this group and stop receiving emails from it, send an email to igv-help+u...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/igv-help/f1de2adb-856a-4e67-8837-b4eed75d98f3%40googlegroups.com.

[Jim Robinson]

Hi,

I just pushed a fix for this, it will be released tonight as 2.3.55.   Apologies for the inconvenience.

Jim

--

---
You received this message because you are subscribed to the Google Groups "igv-help" group.
To unsubscribe from this group and stop receiving emails from it, send an email to igv-help+u...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/igv-help/f1de2adb-856a-4e67-8837-b4eed75d98f3%40googlegroups.com.

[Jorge Amigo Lechuga]

great to hear that. I'm sure there'll be several other users that will benefit from this fix as we are. thanks a lot for this fruitful (and amazingly fast) communication.

[Jorge Amigo Lechuga]

just tried version 2.3.55, which works like a charm. thanks again!