Drawing primitives, line thickness and pygame/ pyglet

[Sebastiaan Mathot]

Hi Jon,

As you know, I've been playing around with PsychoPy to create a back-
end for OpenSesame. I encountered a few issues, which you may be able
to shed some light on (although I suspect some of them may be a
property of OpenGL rather than PsychoPy).

THE GOAL

Basically, what I'm doing is creating a canvas class, which offers the
usual drawing primitives (rect(), line(), ellipse(), etc.) and uses
PsychoPy to handle the actual drawing. It kind of works, but the goal
is to make the behavior identical to that of the regular canvas
backend (which uses PyGame), and this isn't the case yet.

THE PROBLEMS

- I use the ShapeStim() to draw lines and rectangles, but the line
thickness is being funny. Using a line thickness of 1 results in
invisible lines, but only when they are horizontal or vertical.

- I use the PatchStim() to draw ellipses, but I have the same problem
here. Basically, I draw a smaller patch inside a bigger patch to get
an unfilled ellipse, but the resulting line thickness is somewhat
erratic. Also, I have a hard time getting the color right, but this is
probably because I have to look into the colorspace method that is
used for the PatchStim. Am I correct in assuming that colornames
(e.g., "red") don't work in this case?

- Finally, the pygame and pyglet winTypes do not appear to be
interchangeable. Specifically, when using pygame and pix units,
coordinates seem to be misinterpreted and all stimuli are so small
that they are basically a dot in the center of the monitor. Using the
same code with pyglet works as expected.

Thanks!

Regards,
Sebastiaan

[Sebastiaan Mathot]

And I encountered another thing. If I try to draw an image using the
SimpleImageStim() it sometimes looks distorted. They are regular PNG
images and I tried it both by passing a PIL image and a filename. If I
scale the same image up x2 or x4 it looks fine (but the original is
distorted). Screenshot:

http://files.cogsci.nl/tmp/simpleimagestim.png

This is undoubtedly a problem with the image not being in the right
format, but I'm not sure how to fix it. It's supposed to work with any
PIL image, right?

[Jonathan Peirce]

On 12/04/2011 10:15, Sebastiaan Mathot wrote:
> Hi Jon,
>
> As you know, I've been playing around with PsychoPy to create a back-
> end for OpenSesame. I encountered a few issues, which you may be able
> to shed some light on (although I suspect some of them may be a
> property of OpenGL rather than PsychoPy).
>
> THE GOAL
>
> Basically, what I'm doing is creating a canvas class, which offers the
> usual drawing primitives (rect(), line(), ellipse(), etc.) and uses
> PsychoPy to handle the actual drawing. It kind of works, but the goal
> is to make the behavior identical to that of the regular canvas
> backend (which uses PyGame), and this isn't the case yet.
>
> THE PROBLEMS
>
> - I use the ShapeStim() to draw lines and rectangles, but the line
> thickness is being funny. Using a line thickness of 1 results in
> invisible lines, but only when they are horizontal or vertical.
>

I don't get invisible lines when I do this on my mac. :-/
For me it looks exactly 1 pixel wide. Try turning interpolation on/off
for the stimulus?

> - I use the PatchStim() to draw ellipses, but I have the same problem
> here. Basically, I draw a smaller patch inside a bigger patch to get
> an unfilled ellipse, but the resulting line thickness is somewhat
> erratic.

I imagine any erratic behaviour is a matter of anti/aliasing? If not,
could you elaborate on 'erratic'? An elliptical patch 2 pixels wider
than another should give a pretty reasonable circle. Take a look at the
script below showing a bunch of ways to draw a circle/ellipse. I've no
idea what pygame uses to draw its built-in circles.

> Also, I have a hard time getting the color right, but this is
> probably because I have to look into the colorspace method that is
> used for the PatchStim. Am I correct in assuming that colornames
> (e.g., "red") don't work in this case?

No, color names should work fine (see below). But yes, it would be worth
reading up on color spaces:
http://www.psychopy.org/general/colours.html

> - Finally, the pygame and pyglet winTypes do not appear to be
> interchangeable. Specifically, when using pygame and pix units,
> coordinates seem to be misinterpreted and all stimuli are so small
> that they are basically a dot in the center of the monitor. Using the
> same code with pyglet works as expected.

this is a bug, that I'll get to very soon.

cheers,
Jon

#-------circles------
from psychopy import visual, event, filters
import numpy as np
win = visual.Window([800,600], units='pix')

#circle as shape
rad = 100#pix
thetas=np.linspace(0,2*np.pi,num=100)
x,y= rad*np.sin(thetas), rad*np.cos(thetas)
xys = list(np.array([x,y]).transpose()) # yuck - this needs improving ;-)
circle2 = visual.ShapeStim(win, vertices=xys, interpolate=False,
lineWidth=1, lineColor='Aquamarine', pos=[0,0])
circle2.draw()

#circle of two patches
outer = visual.PatchStim(win, size=[100,50],mask='circle',
units='pix', color='red', tex=None)
inner = visual.PatchStim(win, size=[98,48],mask='circle',
units='pix', color=0, tex=None)#NB the screen has color=[0,0,0]
outer.draw()
inner.draw()

#circle as mask
aspect=0.75
outer=filters.makeMask(128, #use power of two
shape='circle',radius=[1,aspect])
inner=filters.makeMask(128, #use power of two
shape='circle',radius=[0.9, 0.9*aspect])#thickness is 10% of radius
#masks have -1 for transparent, +1 for opaque
mask=((outer/2+0.5)-(inner/2+0.5))*2-1
print outer.max(), outer.min()
circle3 = visual.PatchStim(win, size=100,mask=mask,interpolate=True,
units='pix', color=1, tex='sin', pos=[-100,200])
circle3.draw()
circle3.setSize(30) #NB line width will scale size of stim
circle3.draw()

win.flip()
event.waitKeys()

[Jonathan Peirce]

This is bound to be to do with an image/pixel format that I haven't yet
encountered.
Could you send me a copy of your image offlist?

But, on the whole, I would recommend you don't use SimpleImage. Patch is
faster in most cases and can handle more than simple images (rotation,
masking etc)

Jon

--
Dr. Jonathan Peirce
Nottingham Visual Neuroscience

http://www.peirce.org.uk/

[Sebastiaan Mathot]

Thanks for the code!

I resolved one of the issues: The funky colors using the PatchStim
were due to not setting "tex = None". However, the line thickness and
image scaling issues remain. As you can see in the screenshot, tilted
lines of 1px are fine, but straight ones are invisible. Similarly,
ellipses with a line thickness of 1 (made by drawing an inner and an
outer patch) show gaps. I guess these issues are actually one and the
same:
http://files.cogsci.nl/tmp/example.png

The image that doesn't scale is here:
http://files.cogsci.nl/software/opensesame/tutorial2/gaze_neutral.png

I read in the doc that, as you said, PatchStim() is recommended over
SimpleImageStim(). However, the power-of-2 restriction makes the
PatchStim() unsuitable in this case, because the OpenSesame canvas is
general purpose and should be able to draw any kind of image. Perhaps
I'll have the canvas autodetect the dimensions and use PatchStim()
when possible, but it seems there's no getting around using
SimpleImageStim() for the moment.

I am running Ubuntu 10.10 (which I should have mentioned before, I
suppose) and I also noted a few issues that may be specific to Linux.
First, the window frequently fails to intialize (about 75% of the
time). When you turn Compiz (i.e., the desktop effects) off it works
fine, but Compiz is on by default on most modern Linux distributions.
Second, when window.close() is called I get the error below
(experiment.window is the PsychoPy window). This only happens when the
application closes as well, not if the window is closed, but the
application (i.e., the OpenSesame GUI) keeps running. It could be
something weird in OpenSesame too (or a combination), I'm not really
sure.

Traceback (most recent call last):
File "./opensesamerun", line 62, in <module>
exp.end()
File "/home/sebastiaan/git/opensesame/libopensesame/experiment.py",
line 272, in end
openexp.experiment.experiment.end(self)
File "/home/sebastiaan/git/opensesame/openexp/experiment.py", line
287, in end
canvas.close_display(self)
File "/home/sebastiaan/git/opensesame/openexp/canvas.py", line 70,
in close_display
exec("openexp._canvas.%s.close_display(experiment)" %
experiment.canvas_backend)
File "<string>", line 1, in <module>
File "/home/sebastiaan/git/opensesame/openexp/_canvas/psycho.py",
line 578, in close_display
experiment.window.close()
File "/usr/lib/pymodules/python2.6/psychopy/visual.py", line 661, in
close
self.setMouseVisible(True)
File "/usr/lib/pymodules/python2.6/psychopy/visual.py", line 1027,
in setMouseVisible
elif
self.winType=='pyglet':self.winHandle.set_mouse_visible(visibility)
File "/usr/lib/pymodules/python2.6/pyglet/window/__init__.py", line
1107, in set_mouse_visible
self.set_mouse_platform_visible()
File "/usr/lib/pymodules/python2.6/pyglet/window/xlib/__init__.py",
line 920, in set_mouse_platform_visible
xlib.XUndefineCursor(self._x_display, self._window)
ctypes.ArgumentError: argument 2: <type 'exceptions.TypeError'>: wrong
type
AL lib: ALc.c:1879: exit(): closing 1 Device
AL lib: ALc.c:1808: alcCloseDevice(): destroying 1 Context(s)
Inconsistency detected by ld.so: dl-close.c: 736: _dl_close: Assertion
`map->l_init_called' failed!

In general though, the PsychoPy backend is working quite nicely
already!

[Sebastiaan Mathot]

I forgot: I should mention that I use the pyglet wintype.

[Jonathan Peirce]

On 13/04/2011 11:20, Sebastiaan Mathot wrote:
> Thanks for the code!
>
> I resolved one of the issues: The funky colors using the PatchStim
> were due to not setting "tex = None". However, the line thickness and
> image scaling issues remain. As you can see in the screenshot, tilted
> lines of 1px are fine, but straight ones are invisible. Similarly,
> ellipses with a line thickness of 1 (made by drawing an inner and an
> outer patch) show gaps. I guess these issues are actually one and the
> same:
> http://files.cogsci.nl/tmp/example.png

OK, I think the issue is a sampling issue - you're trying to draw a very
thin line in between two pixel locations without any antialiasing.
Opengl is doing the best it can to cope with that. Turning interpolation
on for the stimulus will make the line appear, albeit a little blurred,
but I would avoid trying to draw such thin lines unless you put them at
locations centred exactly on a pixel.

> The image that doesn't scale is here:
> http://files.cogsci.nl/software/opensesame/tutorial2/gaze_neutral.png
>
> I read in the doc that, as you said, PatchStim() is recommended over
> SimpleImageStim(). However, the power-of-2 restriction makes the
> PatchStim() unsuitable in this case, because the OpenSesame canvas is
> general purpose and should be able to draw any kind of image. Perhaps
> I'll have the canvas autodetect the dimensions and use PatchStim()
> when possible, but it seems there's no getting around using
> SimpleImageStim() for the moment.

PatchStim sliently resamples an image to be pow2 if it isn't. So you
won't see any difference. If you absolutely want no interpolation done
with your image (never resize, never rotate), then SimpleImage is OK,
but its draw-time is marginally slower because it has to be sent to the
graphics card on each draw.

> I am running Ubuntu 10.10 (which I should have mentioned before, I
> suppose) and I also noted a few issues that may be specific to Linux.
> First, the window frequently fails to intialize (about 75% of the
> time). When you turn Compiz (i.e., the desktop effects) off it works
> fine, but Compiz is on by default on most modern Linux distributions.
> Second, when window.close() is called I get the error below
> (experiment.window is the PsychoPy window). This only happens when the
> application closes as well, not if the window is closed, but the
> application (i.e., the OpenSesame GUI) keeps running. It could be
> something weird in OpenSesame too (or a combination), I'm not really
> sure.

looks like a linux-specific bug in pyglet. But I don't know enough about
the pyglet low-level code (or linux) to be able to fix that.

Jon

http://www.peirce.org.uk/

This message and any attachment are intended solely for the addressee and may contain confidential information. If you have received this message in error, please send it back to me, and immediately delete it. Please do not use, copy or disclose the information contained in this message or in any attachment. Any views or opinions expressed by the author of this email do not necessarily reflect the views of the University of Nottingham.

This message has been checked for viruses but the contents of an attachment
may still contain software viruses which could damage your computer system:
you are advised to perform your own checks. Email communications with the
University of Nottingham may be monitored as permitted by UK legislation.

[Sebastiaan Mathot]

Hi Jon,

The weird thing is that I was not trying to draw in-between two
pixels. At least, I was using pix-units and integers to define the
coordinates. Nevertheless, turning off interpolation solved the issue.
In contrast, in order for the unfilled ellipses to show properly,
interpolation needed to be turned _on_. How weird is that?

I noted on other thing: When trying to draw an unfilled rectangle
using the shapestim using line thickness 1 (interpolate = off), one of
the corners is missing a pixel. This happens regardless of whether I
do it using 3 vertices and a closed shape or 4 vertices. But I guess
that's not really a biggie.

Sorry to flood you with issues. I realize these are probably due to
OpenGL, rather than PsychoPy.

The PsychoPy canvas backend is pretty much finished now!

Regards,
Sebastiaan

On Apr 13, 12:36 pm, Jonathan Peirce

[Sebastiaan Mathot]

Hi Jon,

I played around some more with the SimpleImageStim class to see why it
sometimes gives these weird results:

http://files.cogsci.nl/tmp/example.png

I've found a workaround by modifying the SimpleImageStim class in
visual.py. I'm not sure what the workaround does, to be honest, but it
does work. The problem does not appear to be a bug in pyglet, but in
psychopy. It could be Linux specific though, I don't know. Anyway,
basically what I did is force psychopy to use shaders, like so:

visual.py:1910:
if self._useShaders or True: # <- Hack
self.dataType = GL.GL_FLOAT

The weird thing is that if I politely ask psychopy to use shaders
(setUseShaders), it doesn't allow me. I don't know enough about opengl
to offer a real patch myself (I'm sure there are side-effects to this
hack), but perhaps this information can be of some use to you.

I'm running Ubuntu 10.10, PsychoPy 1.64.00. The problem consistently
occurs with a wide range of images.

Btw, I ran some benchmarks on the OpenSesame PsychoPy backend and it
looks very good! Certainly better than the non-opengl PyGame backend.

Regards,
Sebastiaan

On Apr 14, 11:14 am, Sebastiaan Mathot <s.mat...@cogsci.nl> wrote:
> Hi Jon,
>
> The weird thing is that I was not trying to draw in-between two
> pixels. At least, I was using pix-units and integers to define the
> coordinates. Nevertheless, turning off interpolation solved the issue.
> In contrast, in order for the unfilled ellipses to show properly,
> interpolation needed to be turned _on_. How weird is that?
>
> I noted on other thing: When trying to draw an unfilled rectangle
> using the shapestim using line thickness 1 (interpolate = off), one of
> the corners is missing a pixel. This happens regardless of whether I
> do it using 3 vertices and a closed shape or 4 vertices. But I guess
> that's not really a biggie.
>
> Sorry to flood you with issues. I realize these are probably due to
> OpenGL, rather than PsychoPy.
>
> The PsychoPy canvas backend is pretty much finished now!
>
> Regards,Sebastiaan
>
> On Apr 13, 12:36 pm, Jonathan Peirce
>
>
>
>
>
>
>
> <jonathan.pei...@nottingham.ac.uk> wrote:

> > On 13/04/2011 11:20,SebastiaanMathot wrote:> Thanks for the code!

> > >> On 12/04/2011 13:06,SebastiaanMathot wrote:
>
> > >>> And I encountered another thing. If I try to draw an image using the
> > >>> SimpleImageStim() it sometimes looks distorted. They are regular PNG
> > >>> images and I tried it both by passing a PIL image and a filename. If I
> > >>> scale the same image up x2 or x4 it looks fine (but the original is
> > >>> distorted). Screenshot:
> > >>>http://files.cogsci.nl/tmp/simpleimagestim.png
> > >>> This is undoubtedly a problem with the image not being in the right
> > >>> format, but I'm not sure how to fix it. It's supposed to work with any
> > >>> PIL image, right?

> > >>> On Apr 12, 11:15 am,SebastiaanMathot<ceebassmu...@gmail.com>    wrote:

[Jonathan Peirce]

I think the key is to modify the import of the image during
createTexture, to force its data to be of a particular format before
passing them to OpenGL.

The reason that you aren't getting shadersdespite asking politely, is
that they aren't currently supported under pygame (and under
pre-opengl2.0 graphics cards on pyglet). Because I wasn't really using
the pygame backend I let that drop. But I have it on my list to
reinstate pygame shader support since quite a few users do seem to be
using it.

Jon