Running PsychoPy in a different multiprocessing.Process on OS X

[Pablo Prietz]

Dear community,

I have a main Python script. It's task is to run a custom PsychoPy script in a separate process. But I run into a big problem the second I try to create a window.

The process has forked and you cannot use this CoreFoundation functionality safely. You MUST exec().

Break on __THE_PROCESS_HAS_FORKED_AND_YOU_CANNOT_USE_THIS_COREFOUNDATION_FUNCTIONALITY___YOU_MUST_EXEC__() to debug.

 

I am running the script on OS X El Capitan on a MacBook Pro (Early 2015). A simple test code is added below.

Does somebody have advice on how to fix this problem? Do you run into the same problem on other platforms? Help is very appreciated.

Warm regards,

Pablo Prietz

from multiprocessing import Process

from psychopy import core, visual, event

def test():

win = visual.Window([2560,1440],fullscr=True,allowGUI=True, screen=0, units='pix')

event.waitKeys()

core.quit()

p = Process(target=test)

p.start()

p.join()

[Pablo Prietz]

Small extension: What would also help is a tip how to debug this. How do I set a breakpoint a python program. Does this even help?

[Andrew]

This may be related to my problem I posted yesterday. Have you tried this in another OS? It may be an OSX problem.

Also, I was under the impression that the new process would need to import all the psychopy modules it needs. Does the subprocess inherit modules?

Andrew

[Andrew]

Also, I was under the impression that the new process would need to import all the psychopy modules it needs. Does the subprocess inherit modules?

Huh, apparently it does. Good to know :) 

 

[Andrew]

I just tried your code in Linux Mint, and it ran without errors. There is some problem when psychopy, multiprocessing, and OSX come together that is way beyond my abilities to speculate on.

[Andrew]

Also, I just tried moving the psychopy imports into your test() function and the code ran on OSX. Sounds similar to my problem and also to previous issues people have experienced: http://stackoverflow.com/questions/8106002/using-the-python-multiprocessing-module-for-io-with-pygame-on-mac-os-10-7

[Pablo Prietz]

Thank you very much for your ideas! If I move the imports to the test() function it works for me too. But the problem is that the actual program that I have problems with imports after creating the process. Therefore it should work. I do not see why. I know it is difficult to say anything without the code.

I will try to run my actual program on an other OS on Monday. I will report back then.

[Pablo Prietz]

Maybe I should add, that I am using pyglui in the main thread. I uploaded a crash report on Gist, too: https://gist.github.com/papr/9ed1828d03e84b3a6085

[Pablo Prietz]

Btw, what would happen if we would exec() ourselves? Are we able to do this? Would it break something? I have no knowledge about the fork/exec behaviour.

[Jonathan Peirce]

Fact:
    OpenGL is a state machine (http://whatis.techtarget.com/definition/state-machine) and should not be used in a second process. It can lead to unpredictable rendering errors. Imagine thread A sets the current memory location to a particular spot and starts uploading a texture, but during the upload thread B can send another command to the card changing the memory location. All sorts of nasty.

Guesses (based on the above):
    - For Pablo's problem my guess is that OSX has decided you're trying to use OpenGL from two locations at once and prevents you from doing so. (Caveat: this would make more sense for a second thread than a second process. I would have thought that two separate processes, with no shared memory, *should* be able to use opengl simultaneously)
    - For Andrew's issue I wouldn't be surprised if opencv also loads opengl and creates some contexts (for rapid/convenient sharing of video memory with the graphcis card). So you've got a  similar problem that you've got two threads trying to access opengl and that's bad. Ideally try and dig around in opencv and see if it is indeed using opengl and try to load only the parts that don't.

In both cases the corollary guess is that Linux is more willing to let you mess things up and doesn't stop you. It hasn't actually caused a problem so you survived but bear in mind you're treading some dangerous ground. I don't know any of that, but it sounds reasonably likely! ;-)

best wishes
Jon

--
You received this message because you are subscribed to the Google Groups "psychopy-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to psychopy-user...@googlegroups.com.
To post to this group, send email to psychop...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/psychopy-users/8b774574-bf80-444e-8db5-d72bf4844321%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

-- 
Jonathan Peirce
University of Nottingham

http://www.peirce.org.uk

This message and any attachment are intended solely for the addressee
and may contain confidential information. If you have received this
message in error, please send it back to me, and immediately delete it. 

Please do not use, copy or disclose the information contained in this
message or in any attachment.  Any views or opinions expressed by the
author of this email do not necessarily reflect the views of the
University of Nottingham.

This message has been checked for viruses but the contents of an
attachment may still contain software viruses which could damage your
computer system, you are advised to perform your own checks. Email
communications with the University of Nottingham may be monitored as
permitted by UK legislation.

[Jeremy Gray]

Pablo,

Perhaps you can elaborate your original intention behind this: "I have a main Python script. It's task is to run a custom PsychoPy script in a separate process." What are you trying to achieve? Maybe there is another way to accomplish that. 

--Jeremy

To view this discussion on the web visit https://groups.google.com/d/msgid/psychopy-users/56CAEF97.10508%40gmail.com.

[Andrew]

Thanks, Jon. I'll have a look and see what I can see. I realize that what I'm doing isn't exactly intended use for PsychoPy, so I appreciate your weighing in at all.

[Pablo Prietz]

Hey guys, thank you for your feedback. It helped me to google way more specifically. :) The problem is solved.

Andrew, to what I am doing: My task is to write an interface for PsychoPy scripts to be able to interact with the Pupil Labs eyetracking software. My way to implement is to write a plugin for the Pupil software which spawns a second process running the PsychoPy script and provides a simple API to get current gaze positions, start/stop calibration and recoding procedures, etc.

The actual solution is already implemented in the Pupil software. Since Python's multiprocessing module only forks processes without exec'ting them and OS X's CoreFoundation is not able to handle calls from forked-only processes, CoreFoundation decides to crash the process instead. The solution is to use billiard a fork of the Python 2.7 multiprocessing package. It is able to disable forking with forking_enable(0) and spawning a completely new process (instead of copying it with fork()). Since this causes the script to run from the beginning in the new process you have to make sure that you do not create a loop creating new processes. To ensure this you have to call freeze_support() as one of the first lines in your code.

Below you can see a fixed version of my sample script from above. I hope this helps everyone who has similar problems.

from billiard import Process, forking_enable, freeze_support

forking_enable(0)

from psychopy import core, visual, event

def test():

win = visual.Window([0,0],fullscr=True,allowGUI=True, screen=0, units='pix')

event.waitKeys()

if __name__ == '__main__':

freeze_support()

[Jon]

Glad to hear you got something working and thanks for posting a solution here.

Jon