Drawing items on a plot

[pyqtgraphuser]

I'm currently using pg.ArrowItem to mark certain events on the plot. However, this is quite slow when there are many items. Is there a faster way to draw points, rectangles, circles on a chart. 

[Luke Campagnola]

On Saturday, February 9, 2013 10:00:11 PM UTC-5, pyqtgraphuser wrote:

I'm currently using pg.ArrowItem to mark certain events on the plot. However, this is quite slow when there are many items. Is there a faster way to draw points, rectangles, circles on a chart. 

 The best way is to use a ScatterPlotItem -- this is optimized for drawing a large number of symbols. There is a small set of built-in shapes that you can use with a scatter plot, and you can supply QPainterPath objects to get arbitrary shapes:

import pyqtgraph as pg

plt = pg.plot()

sp = pg.ScatterPlotItem()

plt.addItem(sp)

# Make an arrow path, rotated 45 degrees

tr = pg.QtGui.QTransform()

tr.rotate(-45)

arrow = tr.map(pg.makeArrowPath(headLen=0.25, tailLen=0.25, tipAngle=40, tailWidth=0.07))

# Fill the scatter plot with data

sp.setData(

    x=[.1,.2,.3,.4,.5,.6,.7,.8], 

    y=[.1,.3,.2,.4,.3,.5,.4,.6], 

    symbol=['o', '+', 't', 'd', 's', arrow, arrow, arrow], 

    size=[10, 10, 10, 20, 20, 30, 40, 50])

Note that if you use QPainterPath to make new scatter plot symbols, the shape must fit entirely within the box from (-0.5, -0.5) to (0.5, 0.5).

Luke

 

[Luke M]

I have related question that may belong in a separate topic.

 

I frequently have large timeseries data sets (>10,000 samples) and using other tools (MATLAB for example) I will sometimes plot the line as well as points. I don't have a need for any specific shape, just a point indicating the location of each data point. So far, using the ScatterPlotItem (typically by using the symbol argument to PlotItem's plot method) seems to drastically slow down performance of the plot. My thought is that using QPoint would speed things up but before digging into the source to figure out exactly how to implement that, I wanted to get your input. Is there a good way to speed this up? Maybe I'm overlooking something.

 

I'm also wondering if this is somehow attributable to my system. In your Scatter Plot example, the fourth plot with 10,000 points is fairly slow to respond to zoom/pan commands. Does this not happen on your systems?

 

Thanks

Luke M

[Luke Campagnola]

You are correct; scatter plots do slow things down quite drastically. The 10k point example you mentioned updates at about 1-2Hz on my system. I have tried several different techniques for plotting symbols and the current system is the fastest I have come up with. The major limitation is usually that Qt's API is often difficult to pair with numpy, so a lot of manual work is required to generate the structures that get passed into QPainter calls (compare this to the 3D scatter plot example, which runs about 100x faster because opengl uses the same data structures as numpy.)

If you're interested in experimenting, I would recommend writing entirely for PyQt--see if you can come up with a simple proof of concept that does not require pyqtgraph. If you find something, I'll gladly help turn that into a pyqtgraph item. The current technique uses QPainter.drawPixmapFragments. You might want to try QPainter.drawPoints?

In the long term my plan is to phase out QGraphicsView in favor of an entirely OpenGL-based system. This will take a lot of work, though, since QGraphicsView provides a lot of functionality that will be difficult to replace. I'm working with the developers of a few other graphics libraries to build a common system to fill this need.

Luke

[Luke M]

I see. That makes sense. I'll have to see what I can come up with. Thanks for the input.

 

Luke M

[Naveen Michaud-Agrawal]

In the long term my plan is to phase out QGraphicsView in favor of an entirely OpenGL-based system. This will take a lot of work, though, since QGraphicsView provides a lot of functionality that will be difficult to replace. I'm working with the developers of a few other graphics libraries to build a common system to fill this need.

This sounds pretty interesting. Any information on the web about this common system? Incidentally, have you seen galry (https://github.com/rossant/galry)?

Naveen

[Luke Campagnola]

On Tue, Feb 19, 2013 at 1:12 PM, Naveen Michaud-Agrawal <naveen.mic...@gmail.com> wrote:

In the long term my plan is to phase out QGraphicsView in favor of an entirely OpenGL-based system. This will take a lot of work, though, since QGraphicsView provides a lot of functionality that will be difficult to replace. I'm working with the developers of a few other graphics libraries to build a common system to fill this need.

This sounds pretty interesting. Any information on the web about this common system? Incidentally, have you seen galry (https://github.com/rossant/galry)?

Cyrille (of Galry) is actively participating in the project. There is a placeholder for the project here: http://code.google.com/p/pyvis3d  but we're really still just in the discussion phase.

Luke

 

[pyqtgraphuser]

How would I draw transparent circles of varying sizes on top of a line chart using this approach?

[pyqtgraphuser]

Cyrille (of Galry) is actively participating in the project. There is a placeholder for the project here: http://code.google.com/p/pyvis3d  but we're really still just in the discussion phase.

This is very exciting! 

[Luke Campagnola]

If you want the circle sizes specified in pixels (size is always the same regardless of how the view is zoomed), then like this:

    sp.setData(x, y, symbol='o', size=arrayOfSizes, brush=(255,255,255,100), pxMode=True)

If you want the sizes specified in the same coordinate system as the data, then use pxMode=False.

Luke

 

[pyqtgraphuser]

I get the following error when using the scatterplotitem. 

Traceback (most recent call last):

    size=[10, 10, 10, 20, 20, 30, 40, 50])

  File "/pyqtgraph/graphicsItems/ScatterPlotItem.py", line 293, in setData

    self.addPoints(*args, **kargs)

  File "/pyqtgraph/graphicsItems/ScatterPlotItem.py", line 387, in addPoints

    setMethod(kargs[k], update=False, dataSet=newData, mask=kargs.get('mask', None))

  File "/pyqtgraph/graphicsItems/ScatterPlotItem.py", line 498, in setSize

    if kargs['mask'] is not None:

NameError: global name 'kargs' is not defined

[Luke Campagnola]

On Sun, Feb 24, 2013 at 9:31 AM, pyqtgraphuser <webus...@gmail.com> wrote:

I get the following error when using the scatterplotitem. 

Traceback (most recent call last):

    size=[10, 10, 10, 20, 20, 30, 40, 50])

  File "/pyqtgraph/graphicsItems/ScatterPlotItem.py", line 293, in setData

    self.addPoints(*args, **kargs)

  File "/pyqtgraph/graphicsItems/ScatterPlotItem.py", line 387, in addPoints

    setMethod(kargs[k], update=False, dataSet=newData, mask=kargs.get('mask', None))

  File "/pyqtgraph/graphicsItems/ScatterPlotItem.py", line 498, in setSize

    if kargs['mask'] is not None:

NameError: global name 'kargs' is not defined

Ach, thank you. I'll have this fixed soon.

Luke

 

[Luke Campagnola]

Fixed in 0.9.7. 

Thanks for the report.

Luke 

[pyqtgraphuser]

Hi Luke,

How do I make the arrows and circles be translucent?

[Luke Campagnola]

On Thu, Feb 14, 2013 at 1:00 PM, Luke M <lcj...@gmail.com> wrote:

I frequently have large timeseries data sets (>10,000 samples) and using other tools (MATLAB for example) I will sometimes plot the line as well as points. I don't have a need for any specific shape, just a point indicating the location of each data point. So far, using the ScatterPlotItem (typically by using the symbol argument to PlotItem's plot method) seems to drastically slow down performance of the plot. My thought is that using QPoint would speed things up but before digging into the source to figure out exactly how to implement that, I wanted to get your input. Is there a good way to speed this up? Maybe I'm overlooking something.

I worked out a solution to this using ctypes to directly access the Qt library to call QPainter.drawPoints; see the attached script. It is able to comfortably draw 1e6 points on my machine at about 15 fps. It probably still needs some work (I haven't tested on windows or OSX yet, and there may be some version-specific differences to check for). 

I suspect this method could also be used to speed up the pixmap fragment painting for scatterPlotItem..

Thanks for pushing me to keep thinking about this :)

Luke

[Luke M]

On Friday, May 3, 2013 12:01:03 AM UTC-4, Luke Campagnola wrote:

 

I worked out a solution to this using ctypes to directly access the Qt library to call QPainter.drawPoints; see the attached script. It is able to comfortably draw 1e6 points on my machine at about 15 fps. It probably still needs some work (I haven't tested on windows or OSX yet, and there may be some version-specific differences to check for). 

 

I suspect this method could also be used to speed up the pixmap fragment painting for scatterPlotItem..

 

Thanks for pushing me to keep thinking about this :)

 

 

Luke

 

 

Thanks for looking into it. This sounds good. Unfortunately, I just got a chance to try this out and I get the following error on line 9:

Traceback (most recent call last):

  File "pointsTest.py", line 9, in <module>
    drawPoints = getattr(qtlib, '?drawPoints@QPainter@@QEAAXPEBVQPointF@@H@Z')
  File "C:\Python27\lib\ctypes\__init__.py", line 378, in __getattr__
    func = self.__getitem__(name)
  File "C:\Python27\lib\ctypes\__init__.py", line 383, in __getitem__
    func = self._FuncPtr((name_or_ordinal, self))
AttributeError: function '?drawPoints@QPainter@@QEAAXPEBVQPointF@@H@Z' not found

 

Would this be a PyQt version issue? I'm still on 4.9.1, mostly because I don't have admin rights on my PC to update...

 

Thanks,

Luke M

[Luke Campagnola]

It could very well be. One of the drawbacks to using ctypes is that I need to know the exact function names that the DLL exports, and these names may change between versions and platforms. I used 'dependency walker' (http://www.dependencywalker.com) to find the function name in C:\python27\lib\site-packages\pyqt4\qtgui4.dll

If you can find the name on your system, I'll add it to the list..

Luke

[Luke M]

On Monday, May 6, 2013 1:54:55 PM UTC-4, Luke Campagnola wrote:

On Mon, May 6, 2013 at 1:45 PM, Luke M <lcj...@gmail.com> wrote:

On Friday, May 3, 2013 12:01:03 AM UTC-4, Luke Campagnola wrote:

 

I worked out a solution to this using ctypes to directly access the Qt library to call QPainter.drawPoints; see the attached script. It is able to comfortably draw 1e6 points on my machine at about 15 fps. It probably still needs some work (I haven't tested on windows or OSX yet, and there may be some version-specific differences to check for). 

 

I suspect this method could also be used to speed up the pixmap fragment painting for scatterPlotItem..

 

Thanks for pushing me to keep thinking about this :)

 

 

Luke

 

 

Thanks for looking into it. This sounds good. Unfortunately, I just got a chance to try this out and I get the following error on line 9:

Traceback (most recent call last):

  File "pointsTest.py", line 9, in <module>

    drawPoints = getattr(qtlib, '?draw...@QPainter@@QEAAXPEBVQPointF@@H@Z')

  File "C:\Python27\lib\ctypes\__init__.py", line 378, in __getattr__
    func = self.__getitem__(name)
  File "C:\Python27\lib\ctypes\__init__.py", line 383, in __getitem__
    func = self._FuncPtr((name_or_ordinal, self))

AttributeError: function '?draw...@QPainter@@QEAAXPEBVQPointF@@H@Z' not found

 

Would this be a PyQt version issue? I'm still on 4.9.1, mostly because I don't have admin rights on my PC to update...

It could very well be. One of the drawbacks to using ctypes is that I need to know the exact function names that the DLL exports, and these names may change between versions and platforms. I used 'dependency walker' (http://www.dependencywalker.com) to find the function name in C:\python27\lib\site-packages\pyqt4\qtgui4.dll

If you can find the name on your system, I'll add it to the list..

Luke

 

I found it at '?drawPoints@QPainter@@QAEXPBVQPointF@@H@Z', though it was different on my XP machine with 4.9.1 so there seems to be some other factor.

 

Anyway,  now I'm getting another error:

File "pointsTest.py", line 44, in paint

    drawPoints(ptr, self.data.ctypes, self.data.shape[0])

WindowsError: exception: access violation reading 0x00000001

 

[Luke Campagnola]

That's strange. Are either of the pointers actually equal to 0x1?  (ptr or self.data.ctypes.data)

Luke

 

[Luke M]

On Monday, May 6, 2013 4:03:37 PM UTC-4, Luke Campagnola wrote:

On Mon, May 6, 2013 at 2:29 PM, Luke M <lcj...@gmail.com> wrote:

On Monday, May 6, 2013 1:54:55 PM UTC-4, Luke Campagnola wrote:

On Mon, May 6, 2013 at 1:45 PM, Luke M <lcj...@gmail.com> wrote:

On Friday, May 3, 2013 12:01:03 AM UTC-4, Luke Campagnola wrote:

 

I worked out a solution to this using ctypes to directly access the Qt library to call QPainter.drawPoints; see the attached script. It is able to comfortably draw 1e6 points on my machine at about 15 fps. It probably still needs some work (I haven't tested on windows or OSX yet, and there may be some version-specific differences to check for). 

 

I suspect this method could also be used to speed up the pixmap fragment painting for scatterPlotItem..

 

Thanks for pushing me to keep thinking about this :)

 

 

Luke

 

 

Thanks for looking into it. This sounds good. Unfortunately, I just got a chance to try this out and I get the following error on line 9:

Traceback (most recent call last):

  File "pointsTest.py", line 9, in <module>

    drawPoints = getattr(qtlib, '?draw...@QPainter@@QEAAXPEBVQPointF@@H@Z')

  File "C:\Python27\lib\ctypes\__init__.py", line 378, in __getattr__
    func = self.__getitem__(name)
  File "C:\Python27\lib\ctypes\__init__.py", line 383, in __getitem__
    func = self._FuncPtr((name_or_ordinal, self))

AttributeError: function '?draw...@QPainter@@QEAAXPEBVQPointF@@H@Z' not found

 

Would this be a PyQt version issue? I'm still on 4.9.1, mostly because I don't have admin rights on my PC to update...

It could very well be. One of the drawbacks to using ctypes is that I need to know the exact function names that the DLL exports, and these names may change between versions and platforms. I used 'dependency walker' (http://www.dependencywalker.com) to find the function name in C:\python27\lib\site-packages\pyqt4\qtgui4.dll

If you can find the name on your system, I'll add it to the list..

Luke

 

I found it at '?draw...@QPainter@@QAEXPBVQPointF@@H@Z', though it was different on my XP machine with 4.9.1 so there seems to be some other factor.

 

Anyway,  now I'm getting another error:

File "pointsTest.py", line 44, in paint

    drawPoints(ptr, self.data.ctypes, self.data.shape[0])

WindowsError: exception: access violation reading 0x00000001

That's strange. Are either of the pointers actually equal to 0x1?  (ptr or self.data.ctypes.data)

Luke

 

 

It doesn't look like it. This is a little over my head so if this doesn't answer your question let me know, but printing ptr and self.data.ctypes yields:

ptr: c_void_p(2213320)
data.ctypes: <numpy.core._internal._ctypes object at 0x04780F30>

 

[Luke Campagnola]

The second one should be self.data.ctypes.data to get the memory pointer (although it should remain self.data.ctypes in the original code)

[Luke M]

I'm sorry, it was self.data.ctypes, I just left out the self when I typed it in here... 

[Luke Campagnola]

It's a bit of an illusion:    self   .    data   .   ctypes   .   data     ('data' appears twice)

[Luke M]

Aah, I'm sorry, I completely missed that. So  self.data.ctypes.data prints as "135856160"

[Chris Wright]

For completeness, to make this work on various platforms:

...
from ctypes.util import find_library
if sys.platform == 'win32':
    qtlib = ctypes.windll.qtgui4
    drawPoints = getattr(qtlib, '?drawPoints@QPainter@@QEAAXPEBVQPointF@@H@Z')
else:
    qtlib = ctypes.cdll.LoadLibrary(find_library("QtGui"))
    drawPoints = getattr(qtlib, '_ZN8QPainter10drawPointsEPK7QPointFi')

class PointsItem(QtGui.QGraphicsItem):
....

[Nicholas Tan Jerome]

Hi Luke,

this script is great, if I want to draw rectangle instead of point, how should I modify this?

drawRects = getattr(qtlib, '_ZN12QPaintEngine9drawRectsEPK6QRectFi')

thanks.

- N

[Luke Campagnola]

On Linux, I use the `nm` command to find the function names:

    $ nm -Dg /usr/lib/python2.7/dist-packages/PyQt4/QtGui.so | grep QPainter.*drawRects
                 U _ZN8QPainter9drawRectsEPK5QRecti
                 U _ZN8QPainter9drawRectsEPK6QRectFi

On windows, you can use "dependency walker".

I have also been playing with extracting these names in python:

lib = open(QtGui.__file__, 'rb').read()
if sys.platform == 'win32':
    chars = r'[a-zA-Z0-9\@\?]'
else:
    chars = r'[a-zA-Z0-9_]'
for m in re.finditer('%s*QPainter%s*drawPoints%s*' % (chars,chars,chars), lib):
    print m.group()

--
You received this message because you are subscribed to the Google Groups "pyqtgraph" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pyqtgraph+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/pyqtgraph/201327c8-e094-4631-abeb-e3506760a0ad%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Nicholas Tan Jerome]

Hi Luke,

sorry, I think I did not write properly in my question, actually I already got this part,

but I don't know how to use the drawRect function inside the painter function.

Normally I can just pass drawRect( int x, int y, int width, int height )

but with the ctypes and sip inside, I'm not sure how can I adapt this for drawRect

Thanks.

-N.

[Luke Campagnola]

On Wed, Oct 15, 2014 at 9:51 AM, Nicholas Tan Jerome <nicholas...@gmail.com> wrote:

Hi Luke,

sorry, I think I did not write properly in my question, actually I already got this part,

but I don't know how to use the drawRect function inside the painter function.

Normally I can just pass drawRect( int x, int y, int width, int height )

but with the ctypes and sip inside, I'm not sure how can I adapt this for drawRect

The signature is QPainter.drawRects(QRectF*, int), so the only difference should be that instead of an (N,2) array of x,y floats you would have an (N,4) array of (x,y,w,h) floats. The exact memory packing might be different from that; you can experiment with it or read the Qt source to determine how QRectF is actually assembled.

[Nicholas Tan Jerome]

Hi Luke,

got it. Thanks for the explanation.

a squareTest modified from your pointTest attached :)