OpenGL 2D plot

[Jochen Schröder]

Hi Luke and all,

for the program I'm working on I had to do scatterplots of up to 1e6
points and found that the current way was too slow for me (this might
have changed with Guillaume's work). I took a different approach to
Guillaume and created a 2D OpenGL ViewWidget. The quick and dirty
approach I took was to simply subclass the 3D OpenGL viewwidget and
change it to orthographic projection and change panning and zooming. Are
people interested in this work? If yes, which way would you prefer for
me to integrate this properly, should I create a generic GLviewwidget
class and subclass both the 3d and 2d widget from that? I'm happy to do
that, I was also thinking that in the long run it might be interesting
to make GL an optional plotting engine so that it works just as the
software rendering at the moment.

Cheers
Jochen

[Luke Campagnola]

On Thu, Sep 26, 2013 at 5:59 AM, Jochen Schröder <cyco...@gmail.com> wrote:

for the program I'm working on I had to do scatterplots of up to 1e6 points and found that the current way was too slow for me (this might have changed with Guillaume's work). I took a different approach to Guillaume and created a 2D OpenGL ViewWidget. The quick and dirty approach I took was to simply subclass the 3D OpenGL viewwidget and change it to orthographic projection and change panning and zooming. Are people interested in this work?

Hi Jochen,

I am definitely interested in this approach, but I'm currently taking a more drastic route. The plan is to have vispy (www.vispy.org) replace the internals of pyqtgraph.opengl, once its 2D graphics visuals are written. Still, I might be able to integrate your changes immediately since it is not clear when vispy will be ready. Where can I find your changes?

 

If yes, which way would you prefer for me to integrate this properly, should I create a generic GLviewwidget class and subclass both the 3d and 2d widget from that?

I think my approach would be to keep the GLViewWidget class and add a Camera class which could represent various view modes (ortho, perspective, stereo) and control modes (3d orbit, 2d pan/zoom, 3d flythrough, etc). 

 

I'm happy to do that, I was also thinking that in the long run it might be interesting to make GL an optional plotting engine so that it works just as the software rendering at the moment.

I have thought about this a lot; always happy to have more input / help. See: 

https://github.com/pyqtgraph/pyqtgraph/wiki/Planned-Development#gpu-accelerated-plotting

Luke

[Guillaume Poulin]

Hi Jochen,

My changes might improve things, but with 1 millions points don't expect miracles, maybe ~1Hz not more. It's a little bit better if you zoom and just plot some points, but still not miraculous. The bottleneck here is Qt (A single call take about 0.5s), so there isn't so much that can be done. With this amount of points, the only solutions is probably gpu acceleration. By the way, 1e6 points, it's about number of pixels in your screen, so it doesn't surprise me that we hit some kind of limit over here. Qt is fast, but I doubt it was design with this kind of use in mind.

I agree with Luke that vispy is probably the way to go, but in the mean time you solution can be interesting. Maybe you could fork the project on github (https://github.com/pyqtgraph/pyqtgraph) and put your modification there for others to see. Maybe Luke gonna want to merge it.

Luke, just to be sure I understand your thought, you would like to add an object of class Camera to the GLViewWidget? If it's the case, I agree that's would be a nice way to deal with it.

By the way, pyopengl is still buggy under python 3. Currently, On my system, I'm not able to compile any shader (syntax error, unexpected NEW_IDENTIFIER) and I don't know why. Everything works on python 2.7 thou.

[Luke Campagnola]

On Thu, Sep 26, 2013 at 11:26 AM, Guillaume Poulin <poulin.g...@gmail.com> wrote:

Le jeudi 26 septembre 2013 21:40:52 UTC+8, Luke Campagnola a écrit :

On Thu, Sep 26, 2013 at 5:59 AM, Jochen Schröder <cyco...@gmail.com> wrote:

for the program I'm working on I had to do scatterplots of up to 1e6 points and found that the current way was too slow for me (this might have changed with Guillaume's work). I took a different approach to Guillaume and created a 2D OpenGL ViewWidget. The quick and dirty approach I took was to simply subclass the 3D OpenGL viewwidget and change it to orthographic projection and change panning and zooming. Are people interested in this work?

Hi Jochen,

I am definitely interested in this approach, but I'm currently taking a more drastic route. The plan is to have vispy (www.vispy.org) replace the internals of pyqtgraph.opengl, once its 2D graphics visuals are written. Still, I might be able to integrate your changes immediately since it is not clear when vispy will be ready. Where can I find your changes?

 

If yes, which way would you prefer for me to integrate this properly, should I create a generic GLviewwidget class and subclass both the 3d and 2d widget from that?

I think my approach would be to keep the GLViewWidget class and add a Camera class which could represent various view modes (ortho, perspective, stereo) and control modes (3d orbit, 2d pan/zoom, 3d flythrough, etc). 

Luke, just to be sure I understand your thought, you would like to add an object of class Camera to the GLViewWidget? If it's the case, I agree that's would be a nice way to deal with it.

That's correct; something like  `view.camera = OrthoCamera()`

A related issue is that the widget itself should not be handling the view coordinates / user input at all. These should be separated, much like GraphicsView and ViewBox. But there is quite a bit more work to be done there, and I'm not sure it makes sense to put more effort into the existing pyqtgraph.opengl if it is just to be replaced with vispy..

[Jochen Schröder]

On 27/09/13 01:26, Guillaume Poulin wrote:
> Le jeudi 26 septembre 2013 21:40:52 UTC+8, Luke Campagnola a �crit :

>
> On Thu, Sep 26, 2013 at 5:59 AM, Jochen Schr�der <cyco...@gmail.com
> <javascript:>> wrote:
>
> for the program I'm working on I had to do scatterplots of up to
> 1e6 points and found that the current way was too slow for me
> (this might have changed with Guillaume's work). I took a
> different approach to Guillaume and created a 2D OpenGL
> ViewWidget. The quick and dirty approach I took was to simply
> subclass the 3D OpenGL viewwidget and change it to orthographic
> projection and change panning and zooming. Are people interested
> in this work?
>
>
> Hi Jochen,
> I am definitely interested in this approach, but I'm currently
> taking a more drastic route. The plan is to have vispy

> (www.vispy.org <http://www.vispy.org>) replace the internals of

> pyqtgraph.opengl, once its 2D graphics visuals are written. Still, I
> might be able to integrate your changes immediately since it is not
> clear when vispy will be ready. Where can I find your changes?

Yes I saw that, but it still seems to be quite a bit in the future correct?

>
> If yes, which way would you prefer for me to integrate this
> properly, should I create a generic GLviewwidget class and
> subclass both the 3d and 2d widget from that?
>
>
> I think my approach would be to keep the GLViewWidget class and add
> a Camera class which could represent various view modes (ortho,
> perspective, stereo) and control modes (3d orbit, 2d pan/zoom, 3d
> flythrough, etc).
>

Do you mean add the camera class as an attribute to the GLViewWidget, or
you want GLViewWidget inherit from the camera class?

> I'm happy to do that, I was also thinking that in the long run
> it might be interesting to make GL an optional plotting engine
> so that it works just as the software rendering at the moment.
>
>
> I have thought about this a lot; always happy to have more input /
> help. See:
> https://github.com/pyqtgraph/pyqtgraph/wiki/Planned-Development#gpu-accelerated-plotting

> <https://github.com/pyqtgraph/pyqtgraph/wiki/Planned-Development#gpu-accelerated-plotting>
>

OK I'll have a look.

>
> Luke
>
>
>
> Hi Jochen,
>
> My changes might improve things, but with 1 millions points don't expect
> miracles, maybe ~1Hz not more. It's a little bit better if you zoom and
> just plot some points, but still not miraculous. The bottleneck here is
> Qt (A single call take about 0.5s), so there isn't so much that can be
> done. With this amount of points, the only solutions is probably gpu
> acceleration. By the way, 1e6 points, it's about number of pixels in
> your screen, so it doesn't surprise me that we hit some kind of limit
> over here. Qt is fast, but I doubt it was design with this kind of use
> in mind.
>

Yes I agree, initially I just downsampled the data, however that created
issues when zooming in and out, so I went down the route of using GL. My
biggest issue was actually not really frame rate, but more the delay
when resizing windows which gives the impression of lag. The GL code
version seems to be doing significantly better even on intel graphics
cards.

> I agree with Luke that vispy is probably the way to go, but in the mean
> time you solution can be interesting. Maybe you could fork the project
> on github (https://github.com/pyqtgraph/pyqtgraph) and put your
> modification there for others to see. Maybe Luke gonna want to merge it.

Yes I was going to do that. At the moment the code just lives on my PC
because I really only created things for my use case so far. I'll post
here once I have integrated and tested with pyqtgraph.

>
> Luke, just to be sure I understand your thought, you would like to add
> an object of class Camera to the GLViewWidget? If it's the case, I agree
> that's would be a nice way to deal with it.
>

Ok let me see if I understand correctly,

> By the way, pyopengl is still buggy under python 3. Currently, On my
> system, I'm not able to compile any shader (syntax error, unexpected
> NEW_IDENTIFIER) and I don't know why. Everything works on python 2.7 thou.
>

> --
> -- [ You are subscribed to pyqt...@googlegroups.com. To unsubscribe,
> send email to pyqtgraph+...@googlegroups.com ]
> ---
> You received this message because you are subscribed to the Google
> Groups "pyqtgraph" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to pyqtgraph+...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.

[Jochen Schröder]

Hi Luke,

you can find the OpenGL view widget for 2D plots at
https://github.com/cycomanic/pyqtgraph in the glcamera branch. What I
did is create a 2D and 3D camera class which contain the modelview,
perspective and pan/zoom/orbit methods, a GLViewWidgetBase and a
GLViewWidget and GLViewWidget2D class which inherit from the above (the
Camera classes needed to inherit from QGLWidget to make super work, I
could change that but then need to do the initialisation manually). The
examples directory contains a ScatterPlotSpeedTest_gl.py and the
corresponding template files. Everything should pretty much work as
expected. I changed the window resizing behaviour compared to the 3D
version to match normal 2D plotting, i.e. resizing will squash or
stretch the dimensions. On my laptop with intel graphics I get about 6
fps with 50 000 points and size 10 and that quickly increases to 60 fps
when decreasing the size. Let me know what you think, I can create a
pull request if you want to merge this.

Cheers
Jochen

On 27/09/13 01:54, Luke Campagnola wrote:
> On Thu, Sep 26, 2013 at 11:26 AM, Guillaume Poulin

> <poulin.g...@gmail.com <mailto:poulin.g...@gmail.com>> wrote:
>
> Le jeudi 26 septembre 2013 21:40:52 UTC+8, Luke Campagnola a �crit :
>
> On Thu, Sep 26, 2013 at 5:59 AM, Jochen Schr�der

> <cyco...@gmail.com> wrote:
>
> for the program I'm working on I had to do scatterplots of
> up to 1e6 points and found that the current way was too slow
> for me (this might have changed with Guillaume's work). I
> took a different approach to Guillaume and created a 2D
> OpenGL ViewWidget. The quick and dirty approach I took was
> to simply subclass the 3D OpenGL viewwidget and change it to
> orthographic projection and change panning and zooming. Are
> people interested in this work?
>
>
> Hi Jochen,
> I am definitely interested in this approach, but I'm currently
> taking a more drastic route. The plan is to have vispy

> (www.vispy.org <http://www.vispy.org>) replace the internals of

> pyqtgraph.opengl, once its 2D graphics visuals are written.
> Still, I might be able to integrate your changes immediately
> since it is not clear when vispy will be ready. Where can I find
> your changes?
>
> If yes, which way would you prefer for me to integrate this
> properly, should I create a generic GLviewwidget class and
> subclass both the 3d and 2d widget from that?
>
>
> I think my approach would be to keep the GLViewWidget class and
> add a Camera class which could represent various view modes
> (ortho, perspective, stereo) and control modes (3d orbit, 2d
> pan/zoom, 3d flythrough, etc).
>
>
> Luke, just to be sure I understand your thought, you would like to
> add an object of class Camera to the GLViewWidget? If it's the case,
> I agree that's would be a nice way to deal with it.
>
>
> That's correct; something like `view.camera = OrthoCamera()`
> A related issue is that the widget itself should not be handling the
> view coordinates / user input at all. These should be separated, much
> like GraphicsView and ViewBox. But there is quite a bit more work to be
> done there, and I'm not sure it makes sense to put more effort into the
> existing pyqtgraph.opengl if it is just to be replaced with vispy..
>

[Guillaume Poulin]

Le dimanche 29 septembre 2013 20:48:10 UTC+8, Jochen Schröder a écrit :

Hi Luke,

you can find the OpenGL view widget for 2D plots at
https://github.com/cycomanic/pyqtgraph in the glcamera branch. What I
did is create a 2D and 3D camera class which contain the modelview,
perspective and pan/zoom/orbit methods, a GLViewWidgetBase and a
GLViewWidget and GLViewWidget2D class which inherit from the above (the
Camera classes needed to inherit from QGLWidget to make super work, I
could change that but then need to do the initialisation manually). The
examples directory contains a ScatterPlotSpeedTest_gl.py and the
corresponding template files. Everything should pretty much work as
expected. I changed the window resizing behaviour compared to the 3D
version to match normal 2D plotting, i.e. resizing will squash or
stretch the dimensions. On my laptop with intel graphics I get about 6
fps with 50 000 points and size 10 and that quickly increases to 60 fps
when decreasing the size. Let me know what you think, I can create a
pull request if you want to merge this.

Cheers
Jochen

You mean 500 000 points, right? I tested with 500 000 points on my laptop (i7-3632QM, no GPU), I have ~23 fps.

I didn't watch the detail of the code, but wouldn't it be better to have something like:

class GLCamera(QtOpenGL.QGLWidget):
    ...


class GLCamera2D(GLCamera):
    ...


class GLViewWidget(QtOpenGL.QGLWidget):


    def __init__(self,...):
       ...
       self.camera = GLCamera()


    def projection2D(self):
        self.camera = GLCamera2D()

The way I see it, GLCamera2D it's simply a camera for which the azimuth and elevation are fixed.

To don't break the current interface of GLViewWidget, we could just wrap methods from GLCamera likes it's done in PlotWidget.

[Jochen Schröder]

On 29/09/13 23:55, Guillaume Poulin wrote:
> Le dimanche 29 septembre 2013 20:48:10 UTC+8, Jochen Schr�der a �crit :

Well that's what I did at first, but I found that to keep the current
interface I had to wrap lots of methods which quickly became tedious.
The other question is what do you do if elevation is changed from the
outside. I'll put up the branch with that code as well.

>
> The way I see it, GLCamera2D it's simply a camera for which the azimuth
> and elevation are fixed.

Well it's more fixed fov and elavation I would say (and azimuth a
rotation around z). I used glOrtho instead though.

>
> To don't break the current interface of GLViewWidget, we could just wrap
> methods from GLCamera likes it's done in PlotWidget.
>

[Guillaume Poulin]

Le lundi 30 septembre 2013 05:18:10 UTC+8, Jochen Schröder a écrit :

On 29/09/13 23:55, Guillaume Poulin wrote:

> Le dimanche 29 septembre 2013 20:48:10 UTC+8, Jochen Schr�der a �crit :

Dynamic wrapping of methods it's already part of PyQtGraph, like in PlotWidget:

for m in ['addItem', 'removeItem', 'autoRange', 'clear', 'setXRange', 'setYRange', 'setRange', 'setAspectLocked', 'setMouseEnabled', 'setXLink', 'setYLink', 'enableAutoRange', 'disableAutoRange', 'register', 'unregister', 'viewRect']:
            setattr(self, m, getattr(self.plotItem, m))

Personally, I don't really like this approach, and I think that an inheritance would probably be better (for many reasons) (Luke, is there a reason for PlotWidget to don't inherit from PlotItem?). However, there is one think that I don't like about the inheritance approach in the case of the camera, it's not possible to change easily the type of camera once the object is created. For example, I could create a 3D graph, look at it and after simply being interested about a particular 2D projection. With a camera attribute, you just need to change self.camera, with the inheritance approach it's not so simple. I tried to find a solution that would prevent this wrapping and keep the flexibility of changing the camera, but I didn't find any.

For the elevation problem, as I understand, self.opts is not part of the public interface and then shouldn't be changed from outside. Even if I'm wrong, this behavior can be enforce by making self.opts an @property.

[Luke Campagnola]

On Mon, Sep 30, 2013 at 6:17 AM, Guillaume Poulin <poulin.g...@gmail.com> wrote:

Dynamic wrapping of methods it's already part of PyQtGraph, like in PlotWidget:

for m in ['addItem', 'removeItem', 'autoRange', 'clear', 'setXRange', 'setYRange', 'setRange', 'setAspectLocked', 'setMouseEnabled', 'setXLink', 'setYLink', 'enableAutoRange', 'disableAutoRange', 'register', 'unregister', 'viewRect']:
            setattr(self, m, getattr(self.plotItem, m))

Personally, I don't really like this approach, and I think that an inheritance would probably be better (for many reasons) (Luke, is there a reason for PlotWidget to don't inherit from PlotItem?).

I'm not really happy with it either. The reason is that PlotWidget would have to inherit from two different QObject classes, which is not allowed in PyQt (it's possibly not allowed in Qt at all?).

 

However, there is one think that I don't like about the inheritance approach in the case of the camera, it's not possible to change easily the type of camera once the object is created. For example, I could create a 3D graph, look at it and after simply being interested about a particular 2D projection. With a camera attribute, you just need to change self.camera, with the inheritance approach it's not so simple. I tried to find a solution that would prevent this wrapping and keep the flexibility of changing the camera, but I didn't find any.

I think dynamic wrapping makes sense in this case. This has made me wonder whether having multiple Camera classes is really necessary. We might be able to squeeze all of the functionality we will need into one class. It still makes sense to me for the camera to be a separate object from the view, though.

Another possibility is to rethink the API a bit, maybe use something like `view.camera.pan(x,y,z)` instead.

Luke 

[Guillaume Poulin]

Le mardi 1 octobre 2013 01:53:00 UTC+8, Luke Campagnola a écrit :

I'm not really happy with it either. The reason is that PlotWidget would have to inherit from two different QObject classes, which is not allowed in PyQt (it's possibly not allowed in Qt at all?).

Indeed, a limitation of Qt.

I think dynamic wrapping makes sense in this case. This has made me wonder whether having multiple Camera classes is really necessary. We might be able to squeeze all of the functionality we will need into one class. It still makes sense to me for the camera to be a separate object from the view, though.

Another possibility is to rethink the API a bit, maybe use something like `view.camera.pan(x,y,z)` instead.

Luke 

 An API rethinking could be done, however that would "break" the current API. In that case, the current API should still be valid but raise a deprecation warning at least for the release in which this modification is done. That shouldn't be difficult to implement by doing a dynamic wrapping with a decorator. It can stay there and at the moment that it causes problems, it can be removed.

Guillaume

[Jochen Schröder]

On 29/09/13 23:55, Guillaume Poulin wrote:

> Le dimanche 29 septembre 2013 20:48:10 UTC+8, Jochen Schr�der a �crit :

>
> Hi Luke,
>
> you can find the OpenGL view widget for 2D plots at
> https://github.com/cycomanic/pyqtgraph
> <https://github.com/cycomanic/pyqtgraph> in the glcamera branch. What I
> did is create a 2D and 3D camera class which contain the modelview,
> perspective and pan/zoom/orbit methods, a GLViewWidgetBase and a
> GLViewWidget and GLViewWidget2D class which inherit from the above (the
> Camera classes needed to inherit from QGLWidget to make super work, I
> could change that but then need to do the initialisation manually). The
> examples directory contains a ScatterPlotSpeedTest_gl.py and the
> corresponding template files. Everything should pretty much work as
> expected. I changed the window resizing behaviour compared to the 3D
> version to match normal 2D plotting, i.e. resizing will squash or
> stretch the dimensions. On my laptop with intel graphics I get about 6
> fps with 50 000 points and size 10 and that quickly increases to 60 fps
> when decreasing the size. Let me know what you think, I can create a
> pull request if you want to merge this.
>
> Cheers
> Jochen
>
>
> You mean 500 000 points, right? I tested with 500 000 points on my
> laptop (i7-3632QM, no GPU), I have ~23 fps.
>
> I didn't watch the detail of the code, but wouldn't it be better to have
> something like:
>
> |

> classGLCamera(QtOpenGL.QGLWidget):
> ...
>
>
> classGLCamera2D(GLCamera):
> ...
>
>
> classGLViewWidget(QtOpenGL.QGLWidget):
>
>
> def__init__(self,...):
> ...
> self.camera =GLCamera()
>
>
> defprojection2D(self):
> self.camera =GLCamera2D()
> |
>

> The way I see it, GLCamera2D it's simply a camera for which the azimuth
> and elevation are fixed.
>
> To don't break the current interface of GLViewWidget, we could just wrap
> methods from GLCamera likes it's done in PlotWidget.
>

OK I've implemented this approach you can find it in the
glcamera_attribute branch in my repository. I still had to somewhat
break the GLViewWidget interface. Essentially distance, elevation and
azimuth are now properties of the camera not keys in the opts
dictionary. I think this approach is more more flexible as it allows
defining functions to calculate them.
Now there's still some problems I have with this approach, this still
uses perspective projection, while I think a 2D view really should be
using an orthographic projection. I'm probably going to implement this
as well, but it's a bit more work. I suspect GL is fine with the
projection type being changed underneath it?

Cheers
Jochen

[Luke Campagnola]

On Mon, Oct 7, 2013 at 6:42 PM, Jochen Schröder <cyco...@gmail.com> wrote:

OK I've implemented this approach you can find it in the glcamera_attribute branch in my repository. I still had to somewhat break the GLViewWidget interface. Essentially distance, elevation and azimuth are now properties of the camera not keys in the opts dictionary. I think this approach is more more flexible as it allows defining functions to calculate them.
Now there's still some problems I have with this approach, this still uses perspective projection, while I think a 2D view really should be using an orthographic projection. I'm probably going to implement this as well, but it's a bit more work. I suspect GL is fine with the projection type being changed underneath it?

Absolutely--OpenGL does not care what the transformation matrices look like. It should be simple to substitute glOrtho for glFrustrum, but for an orthographic projection, the field-of-view angle is no longer meaningful; instead you will need x,y ranges. I'd recommend imitating the ViewBox API as much as it makes sense (setRange in particular).

Luke