[Python-ideas] [Value Returning Threard]

[yoav glazner]

Hi all,

My Idea is to have a value returning Thread.

I'll explain by example.

<pycode>

def foo():

    time.sleep(20)

    return 'bar'

value = thread.startValueReturningThread(foo) #i need a better name for the function...)

#here we do some work

mg = moonGravity()

mg.disable()

#now we need the value that foo returned

print value #this would be blocking untill foo is done!

</pycode>

This feature should provide a way to increase performance when possible with simple syntax.

What do you think?

[Chris Rebert]

Sounds like you haven't heard of Futures -
http://en.wikipedia.org/wiki/Future_(programming)
One of probably many recipes for them in Python:
http://code.activestate.com/recipes/84317/

Cheers,
Chris
--
http://blog.rebertia.com
_______________________________________________
Python-ideas mailing list
Python...@python.org
http://mail.python.org/mailman/listinfo/python-ideas

[George Sakkis]

You're looking for the threadmethod decorator [1]. I'm not sure it's
robust and useful enough to be included in the standard library
though.

George

[1] http://code.activestate.com/recipes/440569/

[MRAB]

I think it would be better if startValueReturningThread() returned what
we could call a 'deferred result' object:

result = thread.startValueReturningThread(foo)

If you requested the actual result before it was available then it would
wait:

print result.value() # or 'result.value'?

It could be implemented using a thread and a queue.

[Antoine Pitrou]

MRAB <google@...> writes:
>
> I think it would be better if startValueReturningThread() returned what
> we could call a 'deferred result' object:

Or perhaps you just want to use Twisted.

Regards

Antoine.

[Aahz]

On Wed, May 06, 2009, yoav glazner wrote:
>
> #now we need the value that foo returned
> print value #this would be blocking untill foo is done!
> </pycode>
>
> This feature should provide a way to increase performance when possible with
> simple syntax.

Please provide more explanation for why the currently available features
are insufficient. Note that what you're asking for would change the
semantics of Python at a very deep level and will almost certainly be
rejected in its present form.
--
Aahz (aa...@pythoncraft.com) <*> http://www.pythoncraft.com/

"It is easier to optimize correct code than to correct optimized code."
--Bill Harlan

[John Graham]

If the OP is asking for futures, couldn't that be implemented in a
library without touching the language? It's a powerful enough feature
to consider putting it in a library. It's going to be one of the
'primitives' of C++0x if one wanted a precedent (although I'm not
suggesting everything C++ does is a good thing. Or even most of what
C++ is a good thing).

[yoav glazner]

 

>Please provide more explanation for why the currently available features
>are insufficient.  Note that what you're asking for would change the
>semantics of Python at a very deep level and will almost certainly be
>rejected in its present form.
--
>Aahz (aa...@pythoncraft.com)           <*>         http://www.pythoncraft.com/

 

I'm no python expert but i had to do a stress test on a libary. (checking thread safety)

here is the code (we soe absraction):

def testMultiThread():

  global resultDict

  resultDict = {}

  def testFuncThread(func):

       global resultDict

       if func():

           resultDict[func] = 'Passed'

       else:

           resultDict[func] = 'Falied'

   thread.start_new(testFuncThread,(Test1,))

   thread.start_new(testFuncThread,(Test2,))

   thread.start_new(testFuncThread,(Test3,))

  

   while (len(resultDict) < 3):

         time.sleep(5)

   return resultDict #here i did something else such as logging and returning True or False

 

Maybe there is a better way for doing this?

If we had something like i suggested (with .value or not)

 

def testMultiThread():

    resultDict = {}

   

      resultDict[Test1]= thread.startValueReturningThread(Test1)

      resultDict[Test1]= thread.startValueReturningThread(Test2)

      resultDict[Test1]= thread.startValueReturningThread(Test3)

 

      return resultDict #or again, pharsing the result...

 

 

[Antoine Pitrou]

John Graham <john.a.graham@...> writes:
>
> If the OP is asking for futures, couldn't that be implemented in a
> library without touching the language? It's a powerful enough feature
> to consider putting it in a library.

I don't know what C++ futures are but you should take a look at Twisted Deferred
objects. The official implementation is in Python but a C implementation has
been lingering on for years, you may be able to give them some help:
http://twistedmatrix.com/trac/ticket/2245

Regards

Antoine.

[Robert Collins]

On Thu, 2009-05-07 at 06:31 +0000, Antoine Pitrou wrote:
> John Graham <john.a.graham@...> writes:
> >
> > If the OP is asking for futures, couldn't that be implemented in a
> > library without touching the language? It's a powerful enough feature
> > to consider putting it in a library.
>
> I don't know what C++ futures are but you should take a look at Twisted Deferred
> objects. The official implementation is in Python but a C implementation has
> been lingering on for years, you may be able to give them some help:
> http://twistedmatrix.com/trac/ticket/2245

Futures block, Defereds call forward.

foo = something_returning_a_future() # does not block
foo.get_value() # blocks

def something_returning_a_future():
def worker():
[some code here that will does expensive work]
return Future(worker)

class Future:
def __init__(callable):
# imagine thread-safe code here to run callable in a thread

def get_value(self):
if not self.result:
self.result = self.queue.pop()
return self.result

This is approximately a Future implementation for python. The difference
- and its quite a big one - to Defereds is that defereds are a
call-forward approach, whereas Futures are proxy objects that are waited
on to get their results.

twisted depends on never blocking as a core part of the design, Futures
and Promises don't work well here.

-Rob

[Aahz]

On Thu, May 07, 2009, yoav glazner wrote:
>
> I'm no python expert but i had to do a stress test on a libary.

> [...]

> Maybe there is a better way for doing this?

For questions about using Python, please post to comp.lang.python

--
Aahz (aa...@pythoncraft.com) <*> http://www.pythoncraft.com/

"It is easier to optimize correct code than to correct optimized code."