Partial texture upload in WebGL 2?

[Brian Chirls]

Does anyone know if it will be possible to upload part of a video or image to a GPU texture? I'm pretty sure this is not possible in WebGL 1, and I saw some hints that glTexSubImage2D works differently in GL ES 3 but I can't find anything that's clear on the matter.

I'm working with very large panoramic video images, which can get up to 4096x2048 (x2 for stereo). Even at half that, GPU upload is too slow (see: http://codeflow.org/issues/slow_video_to_texture/) so I want to see if I can only update the part that's currently in view. I experimented with copying part of a large video frame to a 256x256 2d canvas with drawImage and then uploading that, but it's way too slow. Just copying to the 2d canvas takes upwards of 40ms without the call to texImage2D, even though it's just a small subset of the image.

My hope is that I'll be able to read 256^2 chunks of the video frame into one or separate textures very quickly and only use the parts of the frame that I need.

[Kenneth Russell]

There's currently no new capability in ES 3 / WebGL 2 for uploading
part of an HTML video, image, or canvas element to a texture, though
it could be added.

This would have to be plumbed all the way back to the video decoder, I
think, so that at no point would the browser be doing unnecessary
blits of the entire video.

Could you please provide a small test case showing the poor
performance, with assets that aren't copyright-encumbered?

> --
> You received this message because you are subscribed to the Google Groups
> "WebGL Dev List" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to webgl-dev-lis...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Brian Chirls]

Here is one existing example, using a 2048x1024 video: 

http://threejs.org/examples/vr_video.html

In my Chrome Dev tools, I can see that the texImage2D call mostly takes around 6-7ms on my iMac (2.7GHz i5, Intel Iris Pro). It's enough to get about 58fps, but there are a number of occasions where it gets up as high as 30-40ms. FF appears faster, but I can't quite get the exact numbers out of the performance tools.

Here is the same thing, using a 4096x2048 file:

http://output.jsbin.com/qinenacife

Because of the way spherical video gets stretched, the extra resolution makes a big difference in quality. Here the texImage2D calls are much slower, around 30-100ms each. Definitely nowhere near fast enough for the 60fps required for VR.

If it's possible to optimize the video coder, that would be great, but I wouldn't be surprised if simply copying less data to the GPU made a difference.

You received this message because you are subscribed to a topic in the Google Groups "WebGL Dev List" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/webgl-dev-list/Evp-ZPNEpnA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to webgl-dev-lis...@googlegroups.com.

[Won]

How is the pano encoded? What is the target rez? An octahedral projection will have less stretch, and means you could process fewer texels through the entire pipeline (from video decode to texture fetch) for a given quality level. Probably much easier said than done, unfortunately!

[Brian Chirls]

Details on that particular panorama are here:

http://pedrofe.com/rendering-for-oculus-rift-with-arnold/

http://pedrofe.com/oculus-camera/

Yes, I've considered other projections. Unfortunately, a lot of the existing popular tools I've found out there (e.g. http://www.kolor.com/autopano/) only handle this one projection. So it'll take some work to generate the video.

It'll be nice of Google's Jump software is wide open and very configurable, or even programmable. But I'm not counting on it.

I also tried using a still image for the panorama and only using video for the part of the sphere that moves, but getting color to match between a jpg/png and a mp4/webm on multiple browsers is impossible.