Can I uncompress files in JavaScript?

[demidov...@gmail.com]

Hi,

RCSB.org has compressed PDB.gz data files (e.g. https://files.rcsb.org/download/1CRN.pdb.gz). As I know, browser can uncompress on fly files compressed by server (htaccess deflate gzip).
Can I uncompress files in JavaScript?

Evgeny

[demidov...@gmail.com]

1. Can I check if they use " htaccess deflate gzip" e.g. at https://files.rcsb.org/view/1CRN.pdb?

2. I get "404 Not Found" for https://files.rcsb.org/view/1CRN.pdb.gz. So I can't get pdb.gz? What is the difference between /view/ and /download/?

[Jaume Sánchez]

If you open https://files.rcsb.org/view/1CRN.pdb in Chrome, with DevTools open, in the Network Panel you'll see the attached image, and you can check the headers in the request and the server response.

You can see that the document has gzip enabled, so the server sends the file compressed, and the browser decompresses it, before giving it to the app without having to process it in JS.

You'll see that there's 11.2kB transferred for 49.4kB of resources, you're getting 25% of the original file size.

--
You received this message because you are subscribed to the Google Groups "WebGL Dev List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to webgl-dev-lis...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/webgl-dev-list/a2a5a9c5-98ad-4e48-8b10-fd7ba7465e81n%40googlegroups.com.

[demidov...@gmail.com]

Thank you very much, Jaume. I get for 4CR2.pdb 1.4 MB transferred, 6.8 MB resources, Finish 1.7 Min, Load 2.2 Min. Can I get/show download progress data?

[Jaume Sánchez]

Can I get/show download progress data?

Yes. Kinda. You can get a Reader from the fetch Response, and add the size of the different chunks while it's downloading. Comparing it with the value in the Content-Length header would give you a progress percentage.

Except that in this specific case, since it's gzipped, it won't work: Content-Length is zipped bytes (say, 1.4MB) and the Response.body.Reader returns uncompressed chunks, so your size will add to the final uncompressed size (6.8MB). You end up with a progress of ~500%.

There's no workaround for that, except adding an extra header from the server with the uncompressed size, but since that has to happen on the rcsb.org domain, it's pretty difficult to achieve.

With time the web platform will provide some mechanism to monitor values of Streams, but right now there's no way to do it properly.

The best solution would be to guess an average compression rate for the PDB files, and apply it to the method detailed above.

--

You received this message because you are subscribed to the Google Groups "WebGL Dev List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to webgl-dev-lis...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/webgl-dev-list/2be07c58-32bc-471b-aca9-340b3216dad6n%40googlegroups.com.

[Kai Ninomiya]

The Web platform doesn't provide a way to decompress gzip from JS. You have to either get the browser to decompress on the fly like you mentioned, or use a third party library to decompress. (From a quick Google search, "pako" seems to be the library of choice.)

--

You received this message because you are subscribed to the Google Groups "WebGL Dev List" group.
To unsubscribe from this group and stop receiving emails from it, send an email to webgl-dev-lis...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/webgl-dev-list/f057f547-0e28-4989-af91-47a7fb2ec4a3n%40googlegroups.com.