trouble loading from URL on opening IGV 2.3.71 from HTTPS?

493 views
Skip to first unread message

Lisa Simirenko

unread,
Mar 24, 2016, 6:03:50 PM3/24/16
to igv-help
We have recently changed the URL for the server where we host our sequencing results, and have all been experiencing issues with IGV ever since.  I am wondering if it could be because we changed the protocol from HTTP to HTTPS? Eventually we are able to load our analysis through the new URLs, but it often takes restarting IGV. 

1) One issue is that IGV caches URLs and now the older ones are wrong, but even if we clear the cache, and successfully load a new one, with the new URLs (using the "Load from URL" option), the next time we open IGV there is an error loading the last one opened - even with the correct URL.

Upon re-opening IGV I get an error message that says:
The genome file [https://myserver/my/path/references.fasta] could not be read. Would you like to remove the selected entry?

here are the logs:

INFO [2016-03-24 14:44:25,669] [Main.java:99]  Startup  IGV Version 2.3.71 (100)03/21/2016 12:43 PM

INFO [2016-03-24 14:44:25,670] [Main.java:100]  Java 1.8.0_77

INFO [2016-03-24 14:44:25,670] [DirectoryManager.java:72]  Fetching user directory... 

INFO [2016-03-24 14:44:25,999] [Main.java:101]  Default User Directory: /Users/simi

INFO [2016-03-24 14:44:26,000] [Main.java:102]  OS: Mac OS X

INFO [2016-03-24 14:44:30,294] [GenomeManager.java:145]  Loading genome: https://myserver/analysis/P20160314/references/seqs/sequence/seqs.fasta

INFO [2016-03-24 14:44:30,512] [HttpUtils.java:283]  HEAD request failed for url: https://myserver/analysis/P20160314/references/seqs/sequence/seqs.fasta.fai.  Trying GET

INFO [2016-03-24 14:44:30,584] [HttpUtils.java:283]  HEAD request failed for url: https://myserver/analysis/P20160314/references/seqs/sequence/seqs.fasta.  Trying GET

ERROR [2016-03-24 14:44:30,620] [HttpUtils.java:312]  Error fetching content length

org.broad.igv.exceptions.HttpResponseException: Bad Request <br> <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">

<html><head>

<title>400 Bad Request</title>

</head><body>

<h1>Bad Request</h1>

<p>Your browser sent a request that this server could not understand.<br />

</p>

</body></html>


at org.broad.igv.util.HttpUtils.openConnection(HttpUtils.java:749)

at org.broad.igv.util.HttpUtils.openConnection(HttpUtils.java:618)

at org.broad.igv.util.HttpUtils.openConnectionHeadOrGet(HttpUtils.java:287)

at org.broad.igv.util.HttpUtils.getHeaderField(HttpUtils.java:291)

at org.broad.igv.util.HttpUtils.getContentLength(HttpUtils.java:305)

at org.broad.igv.util.ParsingUtils.getContentLength(ParsingUtils.java:236)

at org.broad.igv.feature.genome.FastaIndexedSequence.<init>(FastaIndexedSequence.java:57)

at org.broad.igv.feature.genome.GenomeManager.loadFastaFile(GenomeManager.java:300)

at org.broad.igv.feature.genome.GenomeManager.loadGenome(GenomeManager.java:167)

at org.broad.igv.feature.genome.GenomeManager.loadGenome(GenomeManager.java:129)

at org.broad.igv.ui.IGVCommandBar$4.run(IGVCommandBar.java:250)

at org.broad.igv.util.LongRunningTask.submit(LongRunningTask.java:57)

at org.broad.igv.ui.IGVCommandBar.loadGenomeListItem(IGVCommandBar.java:297)

at org.broad.igv.ui.IGVCommandBar.access$500(IGVCommandBar.java:83)

at org.broad.igv.ui.IGVCommandBar$GenomeBoxActionListener.actionPerformed(IGVCommandBar.java:308)

at javax.swing.JComboBox.fireActionEvent(JComboBox.java:1258)

at javax.swing.JComboBox.setSelectedItem(JComboBox.java:586)

at org.broad.igv.ui.IGVCommandBar.selectGenome(IGVCommandBar.java:541)

at org.broad.igv.ui.IGV$StartupRunnable.run(IGV.java:2472)

at org.broad.igv.util.LongRunningTask.submit(LongRunningTask.java:57)

at org.broad.igv.ui.IGV.startUp(IGV.java:2415)

at org.broad.igv.ui.Main.open(Main.java:238)

at org.broad.igv.ui.Main.main(Main.java:85)

INFO [2016-03-24 14:44:55,545] [CommandListener.java:106]  Listening on port 60151

    


2) Sometimes we also get this error when "loading from URL":

        Unexpected error: org.broad.igv.exceptions.HttpResponseException: Bad Request

Bad Request     


Here is the log:
        Caused by: org.broad.igv.exceptions.HttpResponseException: Bad Request <br> <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>400 Bad Request</title>
</head><body>
<h1>Bad Request</h1>
<p>Your browser sent a request that this server could not understand.<br />
</p>
</body></html>

at org.broad.igv.util.HttpUtils.openConnection(HttpUtils.java:749)
at org.broad.igv.util.HttpUtils.openConnection(HttpUtils.java:618)
at org.broad.igv.util.HttpUtils.openConnection(HttpUtils.java:614)
at org.broad.igv.util.HttpUtils.openConnectionStream(HttpUtils.java:220)
at org.broad.igv.util.HttpUtils.openConnectionStream(HttpUtils.java:214)
at org.broad.igv.util.ParsingUtils.openInputStreamGZ(ParsingUtils.java:111)
at org.broad.igv.ui.IGV.restoreSessionSynchronous(IGV.java:1420)
... 6 more

 

3) We also noticed a bug when using the “home” and “end” keys to scroll left and right between frames when zoomed in on the IGV sequence window.  If I press these keys too quickly in succession it un-maximizes the IGV program window and begins further messing with the window size on subsequent quick succession key presses (alternating between full-height and the un-maximized state). ( http://www.broadinstitute.org/software/igv/Navigate - “Home and End keys (scroll by screen width)” )

After a while the behavior has gotten worse.  Now just moving the scroll bar with the mouse is causing the resizing behavior.


4) Minor annoyance: We used to be able to “File>Load from URL”, enter the URL, and press “Enter” to load the new URL.  Now that the optional Index URL text box was added it is no longer possible to load the File URL by pressing enter.  The “OK” button is mandatory.


1 & 2 seem related and are our most important issues. I can't tell if it is something on our server end, or in IGV. We didn't have these issues before the changes were made to our server, but we are able to eventually load our data through the server as it is currently configured. At the same time, our sysadmins are very security conscious and we don't have full knowledge of how they are routing requests.Again the major change to the url on our server was the change from HTTP to HTTPS, could that be causing our issues?

Thanks for any help!

Lisa

Jim Robinson

unread,
Mar 24, 2016, 7:02:51 PM3/24/16
to igv-...@googlegroups.com
Hi Lisa,

RE the server issues,  it looks like the main issue is that HEAD requests are being rejected by the server.  Could you check or verify that with the sys admins?  After you get an answer we can go from there.

I don't understand (3) when you say "After a while the behavior has gotten worse...".   Do you mean to say that you get this behavior without using the home/end keys?   Pressing those in rapid succession generates a slew of http requests for sequence,  which then must be canceled.   I will look into making this more robust,  but please expand on the meaning of  "After a while..."

RE (4),  I will see if I can give the OK button focus so enter will work as before.

Jim
--

---
You received this message because you are subscribed to the Google Groups "igv-help" group.
To unsubscribe from this group and stop receiving emails from it, send an email to igv-help+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/igv-help/042ff8c3-0c69-422c-9cd4-20a403ca44f2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Jim Robinson

unread,
Mar 25, 2016, 10:26:54 AM3/25/16
to igv-help
RE issue #1,  I don't understand the reference to "caching the URL".   I'm not sure what you are referring to here,  is it ".genome" files?  


I added 3 git issues for 2-4 which you can comment on and follow if you like.  

Lisa Simirenko

unread,
Mar 25, 2016, 3:29:30 PM3/25/16
to igv-help
Thanks - yes that was an awkward email - sorry if it wasn't clear. I am writing it for other people that use IGV more than I do.
re #3, I that makes sense that rapidly  pressing the home/end keys would be an issue - I think the user read the manual, and expected it to be a robust way to scroll through the sequence.
We use IVGV to verify synthetic sequences we construct, so the users here just want to quickly scroll through areas that there might be errors or dips. The "After a while..." was because the user noticed the issue, and after pressing on the home/end keys and getting the odd behavior (minimizing and maximizing the IGV window), just mousing over the window caused the window to resize, without pressing the home/end buttons.

I will check with the admins about the HEAD requests! That sounds like something they would start blocking, and not mentions to me.
Thanks!
Lisa

Jim Robinson

unread,
Mar 25, 2016, 3:40:01 PM3/25/16
to igv-...@googlegroups.com
OK, thanks for the clarification.   I had similar issues with rapid scrolling for the javascript igv.   The fix there, which I will likely implement in IGV desktop, is to simply ignore key presses if a load is in progress (until the current screen loads).   This will prevent the cascade of asynchronous requests.

RE "head requests",  let me know what the sys admins say and we'll take the next step re debugging.  
--

---
You received this message because you are subscribed to the Google Groups "igv-help" group.
To unsubscribe from this group and stop receiving emails from it, send an email to igv-help+u...@googlegroups.com.

Lisa Simirenko

unread,
Mar 25, 2016, 5:17:09 PM3/25/16
to igv-help
Great! Thanks!

So my sysadmin said that the HEAD requests are not being blocked, but he also gave me a snippet from the logs and it seems that IGV is trying to get a bed.idx file that doesn't exist on our server. I also just consulted with my power user again.
So we don't have a problem loading the data using "load from URL" with the URL pointing to this file:


The problem is when we open IGV and it tries to load the last thing we looked at - or if we select the file from the "genome" pulldown to load it again.

Here is the log snippet:
root@gpweb35:/var/log/httpd# grep 'synbioqc.*HEAD' access_log | tail
synbioqc.jgi.doe.gov 128.3.88.177 - - [25/Mar/2016:12:12:16 -0700] "HEAD /MiSeqAnalysis/AYBBH/20160315_AYAHX/2073_1103381_AYAHX/bwa_dir/callable.bed HTTP/1.1" 200 -
synbioqc.jgi.doe.gov 128.3.88.177 - - [25/Mar/2016:12:12:16 -0700] "HEAD /MiSeqAnalysis/AYBBH/20160315_AYAHX/2073_1103381_AYAHX/bwa_dir/callable.bed.idx HTTP/1.1" 404 -
synbioqc.jgi.doe.gov 128.3.88.177 - - [25/Mar/2016:12:12:16 -0700] "HEAD /MiSeqAnalysis/AYBBH/20160315_AYAHX/2073_1103381_AYAHX/bwa_dir/callable.bed.idx HTTP/1.1" 404 -
synbioqc.jgi.doe.gov 128.3.88.177 - - [25/Mar/2016:12:12:16 -0700] "HEAD /MiSeqAnalysis/AYBBH/20160315_AYAHX/2073_1103381_AYAHX/bwa_dir/callable.bed.idx.gz HTTP/1.1" 404 -
synbioqc.jgi.doe.gov 128.3.88.177 - - [25/Mar/2016:12:12:16 -0700] "HEAD /MiSeqAnalysis/AYBBH/20160315_AYAHX/2073_1103381_AYAHX/bwa_dir/aligned_reads.bam.bai HTTP/1.1" 200 -

The 404 file not found are for these files in the same directory:

callable.bed.idx
callable.bed.idx.gz 

And these do not exist:

synbio@genepool13:/global/dna/shared/synbio/MiSeqAnalysis/AYBBH/20160315_AYAHX/2073_1103381_AYAHX/bwa_dir$ ls -l

total 8448

-rw-r----- 1 synbio synbio 8599676 Mar 17 22:51 aligned_reads.bam

-rw-r----- 1 synbio synbio      96 Mar 17 22:51 aligned_reads.bam.bai

-rw-r----- 1 synbio synbio     139 Mar 17 22:51 call_summary.txt

-rw-r----- 1 synbio synbio      32 Mar 17 22:51 callable.bed

-rw-r----- 1 synbio synbio    7779 Mar 17 22:51 snps.gatk.vcf

-rw-r----- 1 synbio synbio     330 Mar 17 22:51 snps.gatk.vcf.idx


So that leads me to believe that when it is failing IGV just expects to find a .bed.idx  or .bed.idx.gz file in the same directory with the same name as the bed. 

-Lisa

Jim Robinson

unread,
Mar 25, 2016, 5:44:10 PM3/25/16
to igv-...@googlegroups.com
IGV probes for the .idx and .idx.gz file to see if they exist by making a head request.  These are optional index files and it should cause no problem if they are not.    This is normal behavior, it might cause entries in the log but is not an error.

So I've lost track a bit of what the problem is. 

Jim

Lisa Simirenko

unread,
Mar 25, 2016, 6:39:44 PM3/25/16
to igv-help
Just as I was reading this another of my users had the issue, but this time it was while loading through the "Load from URL".

She was loading a URL that points to a similar xml as the one I sent you earlier.
The error message is:

Unexpected error: org.broad.igv.exceptions.HttpResponseException: Bad Request

Bad Request

Your browser sent a request that this server could not understand.


the igv.log says this:


RROR [2016-03-25 15:26:46,295] [IGV.java:1462]  Error loading session session : <br>&nbsp;&nbsp;https://synbioqc.jgi.doe.gov/analysis/P20160314/results/pool7.igv.xml<br>Bad Request <br> <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">


<html><head>
<title>400 Bad Request</title>
</head><body>
<h1>Bad Request</h1>
<p>Your browser sent a request that this server could not understand.<br />
</p>
</body></html>

org.broad.igv.exceptions.HttpResponseException: Bad Request <br> <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>400 Bad Request</title>
</head><body>
<h1>Bad Request</h1>
<p>Your browser sent a request that this server could not understand.<br />
</p>
</body></html>

    at org.broad.igv.util.HttpUtils.openConnection(HttpUtils.java:749)
    at org.broad.igv.util.HttpUtils.openConnection(HttpUtils.java:618)
    at org.broad.igv.util.HttpUtils.openConnection(HttpUtils.java:614)
    at org.broad.igv.util.HttpUtils.openConnectionStream(HttpUtils.java:220)
    at org.broad.igv.util.HttpUtils.openConnectionStream(HttpUtils.java:214)
    at org.broad.igv.util.ParsingUtils.openInputStreamGZ(ParsingUtils.java:111)
    at org.broad.igv.ui.IGV.restoreSessionSynchronous(IGV.java:1420)

    at org.broad.igv.ui.IGV$8.run(IGV.java:1397)
    at org.broad.igv.util.LongRunningTask.call(LongRunningTask.java:72)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    at java.lang.Thread.run(Thread.java:745)
INFO [2016-03-25 15:26:46,295] [MessageUtils.java:74]  <html>Unexpected error: org.broad.igv.exceptions.HttpResponseException: Bad Request <br> <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">


<html><head>
<title>400 Bad Request</title>
</head><body>
<h1>Bad Request</h1>
<p>Your browser sent a request that this server could not understand.<br />
</p>
</body></html>

.<br>See igv.log for more details


Unfortunately I don't have access to the server logs.


The data did finally load after re-opening IGV a couple times. So it is weirdly intermittent. I assumed from the server logs that the problem was the 404's for the bed.idx, but it sounds like that should not be fatal. I will see if I can get the server logs for this particular request. Thank you for the prompt responses!


Lisa

Lisa Simirenko

unread,
Mar 25, 2016, 7:36:49 PM3/25/16
to igv-help
OK, this might be a problem on our end- and if it is, I sincerely apologize!
This is what my sysadmin wrote back to me:

Here is the access log for that:

synbioqc.jgi.doe.gov 128.3.89.191 - - [25/Mar/2016:15:26:46 -0700] "GET /analysis/P20160314/results/pool7.igv.xml HTTP/1.1" 400 226

It returned a 400 status, which means the request was malformed, but
doesn't give any details.

The error logs notes this:

[Fri Mar 25 15:26:46 2016] [error] Hostname synbioQC.jgi.doe.gov provided via SNI and hostname synbioqc.jgi.doe.gov provided via HTTP are different

Then he said he may have misconfigured the server, and made some changes. So I am going to check with my users and have them let me know if they are still experiencing errors - if not, then it was all on our side, and I will let you know what happens. Have a good weekend, and thanks again for all your prompt attention!
Lisa

Jim Robinson

unread,
Mar 25, 2016, 10:34:14 PM3/25/16
to igv-...@googlegroups.com
Ahh, yes, good catch.  I don't know much about configuring https on a server,  but that does look like a configuration error.  Let me know if I can be of further assistance.

Jim
Reply all
Reply to author
Forward
0 new messages