Getting a set of lambda functions
Martin Manns
I try to get a set of lambda functions that allows me executing each
function code exactly once. Therefore, I would like to modify the set
function to compare the func_code properties (or the lambda
functions to use this property for comparison).
(The reason is that the real function list is quite large (> 1E5), there
are only few functions with non-equal code and the order of execution
is not important.)
How can I achieve this?
>>> func_strings=['x', 'x+1', 'x+2', 'x']
>>> funclist = [eval('lambda x:' + func) for func in func_strings]
>>> len(funclist)
4
>>> len(set(funclist))
4
>>> funclist[0].func_code == funclist[3].func_code
True
>>> funclist[0] == funclist[3]
False
Thanks in advance
Martin
Ivan Illarionov
Maybe make a set of code objects?
func_code_set = set([f.func_code for f in funclist])
funclist = []
for fc in func_code_set:
f = lambda x: x
f.func_code = fc
funclist.append(f)
-- Ivan
Ivan Illarionov
Maybe make a set of code objects?
Martin Manns
Works for me.
Thanks
Martin
I V
> function code exactly once. Therefore, I would like to modify the set
> function to compare the func_code properties (or the lambda functions to
> use this property for comparison).
With Ivan's approach, you lose access to the actual lambdas, so you need
to create a new function and then modify its code object to actually call
the code; this seems a little clunky to me. You might instead want to
wrap the lambdas in an object that will do the comparison you want:
class Code(object):
def __init__(self, func):
self._func = func
def __cmp__(self, other):
return cmp(self._func.func_code, other._func.func_code)
def __call__(self, *args, **kwargs):
return self._func(*args, **kwargs)
func_set = set(Code(f) for f in funclist)
bearoph...@lycos.com
> You might instead want to
>wrap the lambdas in an object that will do the comparison you want:
This looks very nice, I haven't tried it yet, but if it works well
then it may deserve to be stored in the cookbook, or better, it may
become the built-in behavior of hashing functions.
Bye,
bearophile
bearoph...@lycos.com
from inspect import getargspec
class HashableFunction(object):
"""Class that can be used to wrap functions, to allow their
hashing,
for example to create a set of unique functions.
>>> func_strings = ['x', 'x+1', 'x+2', 'x']
>>> func_list1 = [eval('lambda x:' + func) for func in
func_strings]
>>> func_set = set(HashableFunction(f) for f in func_list1)
>>> len(func_set)
3
>>> set(f(10) for f in func_set) == set([10, 11, 12])
True
>>> func_list2 = [lambda x=1: x, lambda x: x]
>>> len(set(HashableFunction(f) for f in func_list2))
2
>>> class Foo:
... def __eq__(self, other): raise NotImplementedError
>>> func_list3 = [lambda x=Foo(): x, lambda x=Foo(): x]
>>> set(HashableFunction(f) for f in func_list3)
Traceback (most recent call last):
...
NotImplementedError
"""
# Original code by I V <ivl...@gmail.com>, 25 May 2008,
# http://groups.google.com/group/comp.lang.python/browse_thread/thread/a973de8f3562675c
# modified by bearophile
def __init__(self, func):
self._func = func
signature = getargspec(func)
self._hash = hash(self._func.func_code) ^ \
hash(tuple(signature[0]) + tuple(signature[1:3]))
def __eq__(self, other):
return self._func.func_code == other._func.func_code and \
getargspec(self._func) == getargspec(other._func)
def __hash__(self):
return self._hash
def __call__(self, *args, **kwargs):
return self._func(*args, **kwargs)
I haven't put a full hashing of getargspec(self._func) too into the
__init__() because it may contain too much nested unhashable
structures, but the __eq__() is able to tell them apart anyway, with
some collisions.
Bye,
bearophile
Denis Kasak
Isn't this a bug? Shouldn't it be possible to create a set of
different lambda functions via a loop? At first I thought it was just
a quirk of list comprehensions, but the following example also yields
incorrect (or at least unintuitive) results:
>>> spam = []
>>> for i in range(10):
... spam.append(lambda: i)
>>> spam[0]()
9
>>> spam[1]()
9
Manually creating the lambdas and appending them to a list works as
expected, naturally; I don't see a good reason why it wouldn't work
with a loop. Am I missing something?
--
Denis Kasak
Denis Kasak
Isn't this a bug? Shouldn't it be possible to create a set of
Scott David Daniels
...
>>>> spam = []
>>>> for i in range(10):
> ... spam.append(lambda: i)
>>>> spam[0]()
> 9
>>>> spam[1]()
> 9
>
> Manually creating the lambdas and appending them to a list works as
> expected, naturally; I don't see a good reason why it wouldn't work
> with a loop. Am I missing something?
Yes, you are missing something: binding time. your anonymous function
returns the value of "i" in the enclosing scope _at_the_time_of_
_the function's_execution_. If you follow your previous example with
'del i' and then execute any of your spam functions, you'll get an
"NameError: global name 'i' is not defined".
There are a couple of ways to slve your problem:
(1) use default args to do the binding at the function definition time.
for i in range(10):
spam.append(lambda arg=i: arg)
The lambda expression is normally spelled "lambda i=i: i)", but if you
don't know the idiom, I find the "i=i" part confuses people.
(2) Use something like functools.partial. if you are simply binding
different args to the "same" function:
import functools
def thefunction(x):
return x * 2
spam = [functools.partial(thefunction, i) for i in range(10)]
--Scott David Daniels
Scott....@Acm.Org
Denis Kasak
> Denis Kasak wrote:
> ...
>>>>> spam = []
>>>>> for i in range(10):
>> ... spam.append(lambda: i)
>>>>> spam[0]()
>> 9
>>>>> spam[1]()
>> 9
>>
>> Manually creating the lambdas and appending them to a list works as
>> expected, naturally; I don't see a good reason why it wouldn't work
>> with a loop. Am I missing something?
>
> Yes, you are missing something: binding time. your anonymous function
> returns the value of "i" in the enclosing scope _at_the_time_of_
> _the function's_execution_. If you follow your previous example with
> 'del i' and then execute any of your spam functions, you'll get an
> "NameError: global name 'i' is not defined".
Ah, the problem was in the subtle misunderstanding of the semantics of
lambda functions on my part. It's much clearer now. Thanks.
> There are a couple of ways to slve your problem:
> (1) use default args to do the binding at the function definition time.
> for i in range(10):
> spam.append(lambda arg=i: arg)
> The lambda expression is normally spelled "lambda i=i: i)", but if you
> don't know the idiom, I find the "i=i" part confuses people.
Indeed. It hasn't occured to me that lambdas could bind the variables
inside them by name. I guess my lambdas are still a little shaky. :-)
--
Denis Kasak
Martin Manns
bearoph...@lycos.com wrote:
> This may have some bugs left, but it looks a bit better:
[...]
> self._hash = hash(self._func.func_code) ^ \
> hash(tuple(signature[0]) + tuple(signature[1:3]))
> def __eq__(self, other):
> return self._func.func_code == other._func.func_code and \
> getargspec(self._func) == getargspec(other._func)
> def __hash__(self):
> return self._hash
> def __call__(self, *args, **kwargs):
> return self._func(*args, **kwargs)
>
> I haven't put a full hashing of getargspec(self._func) too into the
> __init__() because it may contain too much nested unhashable
> structures, but the __eq__() is able to tell them apart anyway, with
> some collisions.
Looking at a function:
>>> a=lambda x:x
>>> dir(a)
['__call__', '__class__', '__delattr__', '__dict__', '__doc__',
'__get__', '__getattribute__', '__hash__', '__init__', '__module__',
'__name__', '__new__', '__reduce__', '__reduce_ex__', '__repr__',
'__setattr__', '__str__', 'func_closure', 'func_code', 'func_defaults',
'func_dict', 'func_doc', 'func_globals', 'func_name']
Should func_globals and func_name also be taken into account for
__eq__()?
Best Regards
Martin
Terry Reedy
"I V" <ivl...@gmail.com> wrote in message
news:eci_j.4164$mh5...@nlpi067.nbdc.sbc.com...
| On Sun, 25 May 2008 13:43:15 +0200, Martin Manns wrote:
| > I try to get a set of lambda functions that allows me executing each
I think it worth the reminder that Python has lambda *expressions* that
result in function objects that identical to the function objects produced
by def statements (except for the special name '<lambda>'). For the
purpose of this thread,comparing functions by comparing their code objects,
the name attribute is irrelevant. So this discussion has nothing to do
with the syntactic form of function definitions.
| > function code exactly once. Therefore, I would like to modify the set
| > function to compare the func_code properties (or the lambda functions
to
| > use this property for comparison).
|
| With Ivan's approach, you lose access to the actual lambdas,
I presume you mean function object rather than lambda expression
| so you need
| to create a new function and then modify its code object to actually call
| the code; this seems a little clunky to me. You might instead want to
| wrap the lambdas in an object that will do the comparison you want:
|
| class Code(object):
| def __init__(self, func):
| self._func = func
|
| def __cmp__(self, other):
| return cmp(self._func.func_code, other._func.func_code)
|
| def __call__(self, *args, **kwargs):
| return self._func(*args, **kwargs)
If one could subclass from function instead of object, this would be even
simpler (even with Bearophile's addition) since no __call__ would be
needed. But we cannot, partly because there did not seem to be much use
case when this was once discussed. But here is one ;-)
tjr
Terry Reedy
"Martin Manns" <mma...@gmx.net> wrote in message
news:g1cu8n$5g4$1...@aioe.org...
It depends on what *you* want == to mean.
tjr
John Nagle
> Hi,
>
> I try to get a set of lambda functions that allows me executing each
> function code exactly once. Therefore, I would like to modify the set
> function to compare the func_code properties (or the lambda
> functions to use this property for comparison).
>
> (The reason is that the real function list is quite large (> 1E5), there
> are only few functions with non-equal code and the order of execution
> is not important.)
>
> How can I achieve this?
>
>>>> func_strings=['x', 'x+1', 'x+2', 'x']
>>>> funclist = [eval('lambda x:' + func) for func in func_strings]
What are you actually trying to do, anyway? What's the application?
You probably don't want to use "eval" that way. If you want function
objects out, use "compile".
John Nagle
Martin Manns
John Nagle <na...@animats.com> wrote:
> >>>> func_strings=['x', 'x+1', 'x+2', 'x']
> >>>> funclist = [eval('lambda x:' + func) for func in func_strings]
>
> What are you actually trying to do, anyway? What's the
> application?
>
> You probably don't want to use "eval" that way. If you want
> function objects out, use "compile".
I am writing on an application that I call pyspread
(http://pyspread.sourceforge.net).
It provides a grid (mapped to a numpy.array) into which users can type
in strings that contain python expressions.
Each expression is transformed into a function object in a second
numpy.array of the same size, so that each function can be called by
accessing the respective cell of the grid.
eval seems to work quite fine and provides nice access to globals,
system functions, modules, etc. Therefore, one can do general
programming tasks within the grid without worrying about cell
precedence.
compile returns code objects instead of function objects. I can of
course evaluate these code objects. However, when I store the code
objects, it is an extra operation each time I want to call the
function. So what is the benefit?
More specific: Is there any benefit of calling:
eval(compile(mystring, '', 'eval'))
instead of
eval(mystring)
?
What are the drawbacks of my approach that I may be missing?
Any suggestions?
Best Regards
Martin
Ivan Illarionov
Yes, there is.
Your original example can be rewritten
>>> func_strings=['x', 'x+1', 'x+2', 'x']
>>> funclist = [compile('lambda x: %s' % func, '<string>', 'eval') for func in func_strings]
>>> len(funclist)
4
>>> len(set(funclist))
3
>>> eval(funclist[0])(1)
1
Ivan