custom data registry XML with presigned S3 URLs

[Andrew C]

Hi,

I'm struggling to figure out how to provide BAM and BAM index URLs correctly to IGV through a custom data server XML file. We host private data on S3 and presign URLs to grant access to BAMs.

I can access BAMS via presigned URLs directly and when I "Load from URL" in IGV (pasting the BAM and index URLs into their respective fields in the dialog box).

I need to populate a custom data XML index with these paths. I read the related threads on 1) specifying the 'index=' parameter with the BAM URL, 2) URL-encoding the index URL to avoid parsing problems.

But I can't get anything to work (using IGV v2.3.94).

• When the BAM index is URLencoded as a parameter in the BAM path, IGV reports 'Java.net.MalformedURLException: no protocol:'.

• When the index is included as a parameter but not URLencoded, IGV reports 'Index file not found.'

Can you show me what a valid XML Resource tag should look like, if I have presigned URLs for the BAM and BAM index to combine in one Resource path?

Example (with dummy URLs):

BAM_URL = https://mybucket.s3.amazonaws.com/path/to/run.id/sample1.md.bam?AWSAccessKeyId=EXAMPLEKEY&Expires=3600000&Signature=XyZaBc123%D

BAI_URL = https://mybucket.s3.amazonaws.com/path/to/run.id/sample1.md.bam.bai?AWSAccessKeyId=EXAMPLEKEY&Expires=3600000&Signature=ZzZ888aBC%3D

XML entry = <Resource name="EXAMPLE" path=?? />

I can't post an example with real URLs online, but could provide more info offline. Thanks.

[Andrew C]

Hi again,

Just looking for clarification - can presigned S3 URLs to a BAM and BAM index file be combined in a single Resource definition of a custom data server XML index?

Or can this only be done through other methods, like a session file or external link to control IGV?

Thanks.

[James Robinson]

It should work, using the "index" attribute, but you are reporting that it doesn't.    If you could provide an example XML file that is not working it would be helpful.   You can email a link to igvteam (at) broadinstitute.org,  that is a private email.   Alternatively posting one with made-up URLs might be enough but having a real example is preferred.

[Shivani Sugandha]

Hi Andrew,

I have the same issue with presigned S3 URLs and IGV. Were you able to resolve this problem?

Thanks.

[Helga Thorvaldsdottir]

Hi - Note that you added to a thread that is 4 years old. Much has changed since then.

If you are having problems, please give us more details. Here are the guidelines from the igv-help group welcome message:

1. The version of IGV you are using, and your computer operating system.
   
2. What you did. If the problem happens when you open a file and/or view certain genomic coordinates, attach the file (or provide a link) and specify the genomic coordinates. Please DO NOT paste the contents of your file into an email or in the group's web interface, as files are whitespace sensitive which gets lost when sent in the body of an email. 
3. Your igv.log file. It will be in <your home directory>/igv.