Uploading .bw URL generated by Box

[Chan, Candy Ho Lam]

Hello,

We are trying to upload bigwig files that are currently stored on a cloud-based system, Box. The URL is a short, direct link that is downloadable, for instance, “https://miami.box.com/shared/static/xxxxxxx.bw”  and there is no problem for the browser to accept it in the Add Custom Track page. However, in track view, there is an error message: “Couldn’t zUncompress 16886 bytes: zlib data error”. 

Searching the online forum, I came across this post https://groups.google.com/a/soe.ucsc.edu/forum/?hl=en&fromgroups#!searchin/genome/box$20url/genome/tvkT92hWTUE/u6TOtPj-BAAJ and the original poster was also using Box to generate the URLs. The responder advised that servers need to have “Accept-Ranges:bytes” enabled, and the engineer at UCSC believes that Box service should support this. Has this been confirmed? Am I receiving this error message in loading the data for my tracks because this feature is disabled on Box, or some other reason? Do you know if any user has successfully used Box to generate URLs for custom tracks?

Thanks in advance. 

Best,

Candy

[Matthew Speir]

Hi Candy,

Thank you for your question about using Box.com for hosting your custom tracks.

You can check if a site accepts byte range requests by using the command "curl -I" and looking for the "Accept-Ranges: bytes" in the output. For example here are the results of this command on file hosted on our public site, http://genome.ucsc.edu:

$ curl -I http://genome.ucsc.edu/goldenPath/help/examples/bigBedExample.bb
HTTP/1.1 200 OK
Date: Thu, 11 Aug 2016 16:26:44 GMT
Server: Apache/2.2.15 (CentOS)
Last-Modified: Mon, 14 Jun 2010 23:42:42 GMT
ETag: "120293-66797-489060b770080"
Accept-Ranges: bytes
Content-Length: 419735
Content-Type: text/plain; charset=UTF-8

I'm unsure of why you're seeing the error “Couldn’t zUncompress 16886 bytes: zlib data error”. Maybe you can send us a direct link to a file that reliably produces this error so that we can reproduce and debug this on our end? You send the URL directly to me if you do not want to share it with our public list.

Another one of our engineers suggests looking into using the Box FTP service. He notes that we should have fixed the FTP tissue the other user mentioned. If you do attempt to use FTP, let us know if it works.

I hope this is helpful. If you have any further questions, please reply to gen...@soe.ucsc.edu. All messages sent to that address are archived on a publicly-accessible Google Groups forum. If your question includes sensitive data, you may send it instead to genom...@soe.ucsc.edu.

Matthew Speir
UCSC Genome Bioinformatics Group

--

[Matthew Speir]

Hi Candy,

We contacted box.com's support and they informed us that they do not
fully support byte-range requests at this time. (One of our engineers
notes that some HTTP requests to files stored on their site allow
byte-ranges, while others do not.) Unfortunately, this means that they
cannot be used to host custom tracks, such as bigWigs or bigBeds.

I hope this is helpful. If you have any further questions, please reply
to gen...@soe.ucsc.edu. All messages sent to that address are archived
on a publicly-accessible Google Groups forum. If your question includes
sensitive data, you may send it instead to genom...@soe.ucsc.edu.

Matthew Speir
UCSC Genome Bioinformatics Group

On 8/10/16 8:12 AM, Chan, Candy Ho Lam wrote:

> Hello,
>
> We are trying to upload bigwig files that are currently stored on a
> cloud-based system, Box. The URL is a short, direct link that is
> downloadable, for instance,
> “https://miami.box.com/shared/static/xxxxxxx.bw” and there is no
> problem for the browser to accept it in the Add Custom Track page.
> However, in track view, there is an error message: “Couldn’t
> zUncompress 16886 bytes: zlib data error”.
>
> Searching the online forum, I came across this post
> https://groups.google.com/a/soe.ucsc.edu/forum/?hl=en&fromgroups#!searchin/genome/box$20url/genome/tvkT92hWTUE/u6TOtPj-BAAJ

> <https://groups.google.com/a/soe.ucsc.edu/forum/?hl=en&fromgroups#%21searchin/genome/box$20url/genome/tvkT92hWTUE/u6TOtPj-BAAJ> and

> the original poster was also using Box to generate the URLs. The
> responder advised that servers need to have “Accept-Ranges:bytes”
> enabled, and the engineer at UCSC believes that Box service should
> support this. Has this been confirmed? Am I receiving this error
> message in loading the data for my tracks because this feature is
> disabled on Box, or some other reason? Do you know if any user has
> successfully used Box to generate URLs for custom tracks?
>
> Thanks in advance.
>
> Best,
> Candy

> --
>