Just saying hello!; and a note regarding multiprocessing...

46 views
Skip to first unread message

Tom Swirly

unread,
Sep 19, 2016, 2:35:47 PM9/19/16
to pi3d
Hello, pi3d people!  Just wanted to say hello, but also had some info that might be helpful.

One of the issues with pi3d, if I recall correctly, is that openGL needs the main thread.

I'm dealing with "yet another thing that needs the main thread" - and went to using Python's multiprocessing.

We're not quite done yet but it's going very well, and it's extremely powerful and not too difficult to use.  

In particular, it's easy to write proxies that control one object from another process, so you could actually be running pi3d that way as a "normal" Python program from the command line...


Hope all is well!

--
     /t

http://radio.swirly.com - art music radio 24/7 366/1000

Paddy

unread,
Sep 20, 2016, 4:54:33 AM9/20/16
to pi3d, t...@swirly.com
Tom, good to hear from you. I did come across the issue of needing other things running in the main thread when working on the google devart competition with @davidedc https://github.com/davidedc/devart-template/blob/master/project_code/AutoVJ.py specifically I wanted to use the Flask http server. Like you I ended up using multiprocessing which is quite nice as the behaviour and methods etc are similar to threading so it's all quite familiar. I did try to get some python applications to speed up on the 4 core raspberry pi by splitting things over four processes, however piping or queuing the data killed any speed advantage. At some stage I was going to look at using the shared memory or server process to get round this but haven't got round to it... partly because any number crunching can normally be done by numpy which is already miles faster than anything I could write!

Tom Swirly

unread,
Sep 21, 2016, 12:38:07 PM9/21/16
to Paddy, pi3d
Yes, I'm also worried about the cost of the piping - it serializes everything using pickle - so I'm expecting to use shared memory.

My first experiments on shared memory seemed to work right the first time - https://github.com/timedata-org/timedata_visualizer/blob/master/tests/mp_experiments.py#L70-L73

I'm having much more of a trouble with "proxies" because - you guessed it - the proxy server wants to run in the main thread.  :-D  I might just skip that part.

Take care, and ping me if you need help on... whatever.  :-)  By the way, we'll be living in Amsterdam starting December 1...

On Tue, Sep 20, 2016 at 4:54 AM, Paddy <pat...@eldwick.org.uk> wrote:
Tom, good to hear from you. I did come across the issue of needing other things running in the main thread when working on the google devart competition with @davidedc https://github.com/davidedc/devart-template/blob/master/project_code/AutoVJ.py specifically I wanted to use the Flask http server. Like you I ended up using multiprocessing which is quite nice as the behaviour and methods etc are similar to threading so it's all quite familiar. I did try to get some python applications to speed up on the 4 core raspberry pi by splitting things over four processes, however piping or queuing the data killed any speed advantage. At some stage I was going to look at using the shared memory or server process to get round this but haven't got round to it... partly because any number crunching can normally be done by numpy which is already miles faster than anything I could write!

--
You received this message because you are subscribed to the Google Groups "pi3d" group.
To unsubscribe from this group and stop receiving emails from it, send an email to pi3d+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply all
Reply to author
Forward
0 new messages