from multiprocessing import Queue

[gert]

http://stackoverflow.com/questions/1687344/how-do-you-generate-random-unique-identifiers-in-a-multi-process-and-multi-thread

Alex ask how modwsgi works.

import os, threading
from multiprocessing import Queue

def sid(q):
while True:
u = os.urandom(8).decode('hex')
q.put(u)

q = Queue(2)

t = threading.Thread(target=sid, args=(q,))
t.daemon = True
t.start()

def application(environ, response):
o = q.get()
response('200 OK', [('Content-type', 'text/plain'),('Content-
Length', str(len(o)))])
return [o]

Will this work on multiple processes ?

[Graham Dumpleton]

2009/11/8 gert <gert.c...@gmail.com>:

Did you try it?

Graham

[gert]

nope it does not do the multiprocessing thing in mod_wsgi only the
multi threading

from threading import Thread

from multiprocessing import Queue

def sid(q):

u = 0
while True:
u += 1
q.put(u)

q = Queue(2)

t = Thread(target=sid, args=(q,))

t.daemon = True
t.start()

def application(environ, response):

o = str(q.get())

[Graham Dumpleton]

What do you need the unique id in the first place? What is the maximum
length it can be?

2009/11/9 gert <gert.c...@gmail.com>:

[gert]

Session id's or barcodes for concert tickets etc.

But never mind that, it raises a much more interesting question. Can
the built in multiprocessing module in python be able to work in a
mod_wsgi environment ?

On Nov 9, 12:21 am, Graham Dumpleton <graham.dumple...@gmail.com>
wrote:

> What do you need the unique id in the first place? What is the maximum
> length it can be?
>

> 2009/11/9 gert <gert.cuyk...@gmail.com>:

[Graham Dumpleton]

2009/11/9 gert <gert.c...@gmail.com>:

>
> Session id's or barcodes for concert tickets etc.
>
> But never mind that, it raises a much more interesting question. Can
> the built in multiprocessing module in python be able to work in a
> mod_wsgi environment ?

If the multiprocessing module can only communicate between processes
whereby one was forked from another then no, it cannot be used to
communicate between mod_wsgi daemon mode or even Apache server child
processes in embedded mode, as you can't run code in the parent
process to the fork. Even then, not sure if multiprocessing module
would allow you to communicate between peered child processes of the
fork, or only back to the parent process.

Why do you need the processes to communicate with each other?

Graham

[gert]

On Nov 9, 12:40 am, Graham Dumpleton <graham.dumple...@gmail.com>
wrote:
> 2009/11/9 gert <gert.cuyk...@gmail.com>:

>
> > Session id's or barcodes for concert tickets etc.
>
> > But never mind that, it raises a much more interesting question. Can
> > the built in multiprocessing module in python be able to work in a
> > mod_wsgi environment ?
>
> If the multiprocessing module can only communicate between processes
> whereby one was forked from another then no, it cannot be used to
> communicate between mod_wsgi daemon mode or even Apache server child
> processes in embedded mode, as you can't run code in the parent
> process to the fork. Even then, not sure if multiprocessing module
> would allow you to communicate between peered child processes of the
> fork, or only back to the parent process.
>
> Why do you need the processes to communicate with each other?
>

There are lots of interesting stuff processes can tel to each other
like, do not try to execute this because me process 1 is already
calculating it, tell the user I will sent him a email in 5 minutes
when its done so that he stops hitting the damn refresh button.

Yes I know you communicate in other way's but
http://docs.python.org/library/multiprocessing.html is just the
coolest one
So what about a from modwsgi import multiprocessing :)

[Graham Dumpleton]

2009/11/9 gert <gert.c...@gmail.com>:

>
> On Nov 9, 12:40 am, Graham Dumpleton <graham.dumple...@gmail.com>
> wrote:
>> 2009/11/9 gert <gert.cuyk...@gmail.com>:
>>
>> > Session id's or barcodes for concert tickets etc.
>>
>> > But never mind that, it raises a much more interesting question. Can
>> > the built in multiprocessing module in python be able to work in a
>> > mod_wsgi environment ?
>>
>> If the multiprocessing module can only communicate between processes
>> whereby one was forked from another then no, it cannot be used to
>> communicate between mod_wsgi daemon mode or even Apache server child
>> processes in embedded mode, as you can't run code in the parent
>> process to the fork. Even then, not sure if multiprocessing module
>> would allow you to communicate between peered child processes of the
>> fork, or only back to the parent process.
>>
>> Why do you need the processes to communicate with each other?
>>
>
> There are lots of interesting stuff processes can tel to each other
> like, do not try to execute this because me process 1 is already
> calculating it, tell the user I will sent him a email in 5 minutes
> when its done so that he stops hitting the damn refresh button.
>
> Yes I know you communicate in other way's but
> http://docs.python.org/library/multiprocessing.html is just the
> coolest one
> So what about a from modwsgi import multiprocessing :)

Not necessarily the coolest. It does have a low entry point though.

If you want to see something really awesome (scary), have a look at:

http://ose.sourceforge.net
http://ose.sourceforge.net/browse.php?group=python-manual&entry=manual.htm

It provides full request/reply and publish/subscribe messaging
infrastructure. I use it where I work across hundreds of sites and
thousands of machines. In one case the network of processes is
probably over 1500.

I have learnt more about how threading works in Python now so
technically needs to be tweaked a little to be truly conforming for
use in context of multiple sub interpreters, but that was implemented
so could be used in any sub interpreter in Apache processes, plus
allow one to communicate between sub interpreters as well as to other
processes.

Do note that this needs a central process to act as message exchange
if doing interprocess communications. But then, one can create a dummy
daemon process group and preload script to initialise it as such.

I really should go back and make sure it all works under mod_wsgi
since last time I played with it in conjunction with Apache was with
mod_python.

Graham

[gert]

On Nov 9, 1:17 am, Graham Dumpleton <graham.dumple...@gmail.com>

wrote:
> 2009/11/9 gert <gert.cuyk...@gmail.com>:
>
> > On Nov 9, 12:40 am, Graham Dumpleton <graham.dumple...@gmail.com>
> > wrote:
> >> 2009/11/9 gert <gert.cuyk...@gmail.com>:
>
> >> > Session id's or barcodes for concert tickets etc.
>
> >> > But never mind that, it raises a much more interesting question. Can
> >> > the built in multiprocessing module in python be able to work in a
> >> > mod_wsgi environment ?
>
> >> If the multiprocessing module can only communicate between processes
> >> whereby one was forked from another then no, it cannot be used to
> >> communicate between mod_wsgi daemon mode or even Apache server child
> >> processes in embedded mode, as you can't run code in the parent
> >> process to the fork. Even then, not sure if multiprocessing module
> >> would allow you to communicate between peered child processes of the
> >> fork, or only back to the parent process.
>
> >> Why do you need the processes to communicate with each other?
>
> > There are lots of interesting stuff processes can tel to each other
> > like, do not try to execute this because me process 1 is already
> > calculating it, tell the user I will sent him a email in 5 minutes
> > when its done so that he stops hitting the damn refresh button.
>
> > Yes I know you communicate in other way's but

> >http://docs.python.org/library/multiprocessing.htmlis just the

> > coolest one
> > So what about a from modwsgi import multiprocessing :)
>
> Not necessarily the coolest. It does have a low entry point though.
>
> If you want to see something really awesome (scary), have a look at:
>
>  http://ose.sourceforge.net

>  http://ose.sourceforge.net/browse.php?group=python-manual&entry=manua...

>
> It provides full request/reply and publish/subscribe messaging
> infrastructure. I use it where I work across hundreds of sites and
> thousands of machines. In one case the network of processes is
> probably over 1500.
>
> I have learnt more about how threading works in Python now so
> technically needs to be tweaked a little to be truly conforming for
> use in context of multiple sub interpreters, but that was implemented
> so could be used in any sub interpreter in Apache processes, plus
> allow one to communicate between sub interpreters as well as to other
> processes.
>
> Do note that this needs a central process to act as message exchange
> if doing interprocess communications. But then, one can create a dummy
> daemon process group and preload script to initialise it as such.
>
> I really should go back and make sure it all works under mod_wsgi
> since last time I played with it in conjunction with Apache was with
> mod_python.
>

I can live with a dummy daemon Queue :)
As long I do not need to sent a soap message to it...
or any framework crap.

[Graham Dumpleton]

2009/11/9 gert <gert.c...@gmail.com>:

Don't worry, I will send you a really nice simple example later on how
to use it to aggregate mod_wsgi usage statistics across multiple
processes. It isn't that evil.

Graham