Web Images Videos Maps News Shopping Gmail more »
Recently Visited Groups | Help | Sign in
Google Groups Home
specifying constants for a function (WAS: generator expressions: performance anomaly?)
There are currently too many topics in this group that display first. To make this topic appear first, remove this option from another topic.
There was an error processing your request. Please try again.
flag
  10 messages - Collapse all  -  Translate all to Translated (View all originals)
The group you are posting to is a Usenet group. Messages posted to this group will make your email address visible to anyone on the Internet.
Your reply message has not been sent.
Your post was successful
 
From:
To:
Cc:
Followup To:
Add Cc | Add Followup-to | Edit Subject
Subject:
Validation:
For verification purposes please type the characters you see in the picture below or the numbers you hear by clicking the accessibility icon. Listen and type the numbers you hear
 
Steven Bethard  
View profile  
 More options Jan 21 2005, 8:57 pm
Newsgroups: comp.lang.python
From: Steven Bethard <steven.beth...@gmail.com>
Date: Fri, 21 Jan 2005 18:57:58 -0700
Local: Fri, Jan 21 2005 8:57 pm
Subject: specifying constants for a function (WAS: generator expressions: performance anomaly?)
I wrote:

 > If you really want locals that don't contribute to arguments, I'd be
 > much happier with something like a decorator, e.g.[1]:
 >
 > @with_consts(i=1, deftime=time.ctime())
 > def foo(x, y=123, *args, **kw):
 >    return x*y, kw.get('which_time')=='now' and time.ctime() or deftime
 >
 > Then you don't have to mix parameter declarations with locals
 > definitions.
 >
 > Steve
 >
 > [1] I have no idea how implementable such a decorator would be.  I'd
 > just like to see function constants declared separate from arguments
 > since they mean such different things.

I played around with this, and I think it's basically implementable:

py> import new
py> class with_consts(object):
...     def __init__(self, **consts):
...         self.consts = consts
...     def __call__(self, func):
...         return new.function(func.func_code,
...                             dict(globals(), **self.consts))
...
py> @with_consts(y=123)
... def f(x):
...     return x*y, str
...
py> f(2)
(246, <type 'str'>)

I just update the function globals with the keywords passed to the
decorator.  The only problem is that updates to globals() aren't
reflected in the function's globals:

py> str = 5
py> f(2)
(246, <type 'str'>)

Steve


    Reply to author    Forward  
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Nick Coghlan  
View profile  
 More options Jan 22 2005, 1:22 am
Newsgroups: comp.lang.python
From: Nick Coghlan <ncogh...@iinet.net.au>
Date: Sat, 22 Jan 2005 16:22:33 +1000
Local: Sat, Jan 22 2005 1:22 am
Subject: Re: specifying constants for a function (WAS: generator expressions: performance anomaly?)

Raymond's constant binding decorator is probably a good model for how to do it:
http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/277940

Cheers,
Nick.

--
Nick Coghlan   |   ncogh...@email.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://boredomandlaziness.skystorm.net


    Reply to author    Forward  
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Bengt Richter  
View profile  
 More options Jan 22 2005, 11:28 pm
Newsgroups: comp.lang.python
From: b...@oz.net (Bengt Richter)
Date: Sun, 23 Jan 2005 04:28:27 GMT
Local: Sat, Jan 22 2005 11:28 pm
Subject: Re: specifying constants for a function (WAS: generator expressions: performance anomaly?)

I thought so too. I modified it to accept **presets as a keyword argument
and generate constants and assignments from its values matching assignment names
when the rhs was __frompresets__, e.g.,

 >>> from makeconstpre import make_constants as pre
 >>> import time
 >>> @pre(verbose=True, deftime=time.ctime(), a=1, b=2, c=3, pi=__import__('math').pi)
 ... def foo():
 ...     deftime = __frompresets__
 ...     b, a = __frompresets__
 ...     c = __frompresets__
 ...     pi = __frompresets__
 ...     return locals()
 ...
 __frompresets__ : deftime --> Sat Jan 22 20:18:09 2005
 __frompresets__ : ('b', 'a') --> (2, 1)
 __frompresets__ : c --> 3
 __frompresets__ : pi --> 3.14159265359
 locals --> <built-in function locals>
 >>> import dis
 >>> dis.dis(foo)
   3           0 LOAD_CONST               1 ('Sat Jan 22 20:18:09 2005')
               3 STORE_FAST               3 (deftime)

   4           6 LOAD_CONST               2 ((2, 1))
               9 UNPACK_SEQUENCE          2
              12 STORE_FAST               2 (b)
              15 STORE_FAST               0 (a)

   5          18 LOAD_CONST               3 (3)
              21 STORE_FAST               1 (c)

   6          24 LOAD_CONST               4 (3.1415926535897931)
              27 STORE_FAST               4 (pi)

   7          30 LOAD_CONST               5 (<built-in function locals>)
              33 CALL_FUNCTION            0
              36 RETURN_VALUE
 >>> foo()
 {'a': 1, 'c': 3, 'pi': 3.1415926535897931, 'b': 2, 'deftime': 'Sat Jan 22 20:18:09 2005'}

Not vary tested as yet, but seems to work. I want to eliminate redundant constants if the
same name is assigned = __frompresets__ more than once. Right now I brute force generate
another constant.

Regards,
Bengt Richter


    Reply to author    Forward  
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Steven Bethard  
View profile  
 More options Jan 23 2005, 3:08 pm
Newsgroups: comp.lang.python
From: Steven Bethard <steven.beth...@gmail.com>
Date: Sun, 23 Jan 2005 13:08:11 -0700
Local: Sun, Jan 23 2005 3:08 pm
Subject: Re: specifying constants for a function (WAS: generator expressions: performance anomaly?)

Yeah, I thought about this recipe, but opcodes always scare me too much. ;)

Steve


    Reply to author    Forward  
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Steven Bethard  
View profile  
 More options Jan 23 2005, 3:14 pm
Newsgroups: comp.lang.python
From: Steven Bethard <steven.beth...@gmail.com>
Date: Sun, 23 Jan 2005 13:14:10 -0700
Local: Sun, Jan 23 2005 3:14 pm
Subject: Re: specifying constants for a function (WAS: generator expressions: performance anomaly?)

Hmm... Having to state
     deftime = __frompresets__
when you've already stated
     deftime=time.ctime()
seems a little redundant to me...

Steve


    Reply to author    Forward  
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Bengt Richter  
View profile  
 More options Jan 24 2005, 2:30 am
Newsgroups: comp.lang.python
From: b...@oz.net (Bengt Richter)
Date: Mon, 24 Jan 2005 07:30:32 GMT
Local: Mon, Jan 24 2005 2:30 am
Subject: Re: specifying constants for a function (WAS: generator expressions: performance anomaly?)

On Sun, 23 Jan 2005 13:14:10 -0700, Steven Bethard <steven.beth...@gmail.com> wrote:
>Bengt Richter wrote:
>> On Sat, 22 Jan 2005 16:22:33 +1000, Nick Coghlan <ncogh...@iinet.net.au> wrote:

[Steven Bethard]
>>>> > If you really want locals that don't contribute to arguments, I'd be
>>>> > much happier with something like a decorator, e.g.[1]:

>>>> > @with_consts(i=1, deftime=time.ctime())
>>>> > def foo(x, y=123, *args, **kw):
>>>> >    return x*y, kw.get('which_time')=='now' and time.ctime() or deftime

>>>> > Then you don't have to mix parameter declarations with locals
>>>> > definitions.

[...]

[Nick Coghlan]

Yeah, but as a quick hack, it made the byte code twiddling all changing byte codes in-place.
I didn't want to verify that byte code wouldn't need relocation fixups if I inserted something,
(unless of course within a suite with branches or loops) but hopefully it's easily so at the
beginning.

BTW, unlike default arguments, varname = __frompresets__ will work anywhere in the code,
(for the current decorator version) and the generated byte code will have a load of
an appropriate constant (the same one if the same name is being assigned). This would
be tricky to get right if done in a hidden just-in-time manner only in branches that
used the values, but hidden initialization of the whole presets set at the beginning
should be relatively easy if there are no relocation problems.
So, e.g., for

 >>> presets = dict(a=1, b=2, deftime=__import__('time').ctime())

in the decorator args, the next version will act as if the decorated
function had the source code

 >>> print '%s = __frompresets__' % ', '.join(sorted(presets))
 a, b, deftime = __frompresets__

for the current version, except that it will be hidden.

Regards,
Bengt Richter


    Reply to author    Forward  
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Steven Bethard  
View profile  
 More options Jan 24 2005, 2:31 am
Newsgroups: comp.lang.python
From: Steven Bethard <steven.beth...@gmail.com>
Date: Mon, 24 Jan 2005 00:31:17 -0700
Local: Mon, Jan 24 2005 2:31 am
Subject: Re: specifying constants for a function (WAS: generator expressions: performance anomaly?)

Bengt Richter wrote:
> So, e.g., for

>  >>> presets = dict(a=1, b=2, deftime=__import__('time').ctime())

> in the decorator args, the next version will act as if the decorated
> function had the source code

>  >>> print '%s = __frompresets__' % ', '.join(sorted(presets))
>  a, b, deftime = __frompresets__

> for the current version, except that it will be hidden.

Cool!  Keep us posted.  I assume you'll put this into the Cookbook?

Steve


    Reply to author    Forward  
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Bengt Richter  
View profile  
 More options Jan 24 2005, 3:35 pm
Newsgroups: comp.lang.python
From: b...@oz.net (Bengt Richter)
Date: Mon, 24 Jan 2005 20:35:09 GMT
Local: Mon, Jan 24 2005 3:35 pm
Subject: Re: specifying constants for a function (WAS: generator expressions: performance anomaly?)

Not ready for prime time, I guess, but I made a module that does presets
and also adjusts parameters and count to do currying without nested function
calling in the curried function.

 >>> from presets import presets, curry
 >>> @presets(verbose=True, a=1, b=2, deftime=__import__('time').ctime())
 ... def foo():
 ...    print (a, b)
 ...    print deftime
 ...

 presets: -- "name(?)" means name may be unused
            a = 1
            b = 2
      deftime = 'Mon Jan 24 12:16:07 2005'

 >>> foo()
 (1, 2)
 Mon Jan 24 12:16:07 2005
 >>> @curry(verbose=True, y=444)
 ... def bar(x=3, y=4):
 ...     return x*y
 ...

 presets: -- "name(?)" means name may be unused
            y = 444

 >>> bar(2)
 888
 >>> bar(2, 3)
 Traceback (most recent call last):
   File "<stdin>", line 1, in ?
 TypeError: bar() takes at most 1 argument (2 given)
 >>> bar()
 1332
 >>> import dis
 >>> dis.dis(bar)
   1           0 LOAD_CONST               1 (444)
               3 STORE_FAST               1 (y)

   3           6 LOAD_FAST                0 (x)
               9 LOAD_FAST                1 (y)
              12 BINARY_MULTIPLY
              13 RETURN_VALUE
 >>> dis.dis(foo)
   1           0 LOAD_CONST               1 ((1, 2, 'Mon Jan 24 12:16:07 2005'))
               3 UNPACK_SEQUENCE          3
               6 STORE_FAST               0 (a)
               9 STORE_FAST               1 (b)
              12 STORE_FAST               2 (deftime)

   3          15 LOAD_FAST                0 (a)
              18 LOAD_FAST                1 (b)
              21 BUILD_TUPLE              2
              24 PRINT_ITEM
              25 PRINT_NEWLINE

   4          26 LOAD_FAST                2 (deftime)
              29 PRINT_ITEM
              30 PRINT_NEWLINE
              31 LOAD_CONST               0 (None)
              34 RETURN_VALUE

Now I have to see whether I can optimize the result with Raymond's make_constants
as two decorators in a row... (pre without keyword args is an alias for Raymond's make_constants,
just handy for the moment)

 >>> from pre import pre
 >>> @pre(verbose=True)
 ... @presets(verbose=True, a=1, b=2, deftime=__import__('time').ctime())
 ... def foo():
 ...    print (a, b)
 ...    print deftime
 ...    print __name__
 ...

 presets: -- "name(?)" means name may be unused
            a = 1
            b = 2
      deftime = 'Mon Jan 24 12:29:21 2005'

 __name__ --> __main__
 >>> foo()
 (1, 2)
 Mon Jan 24 12:29:21 2005
 __main__
 >>> dis.dis(foo)
   1           0 LOAD_CONST               1 ((1, 2, 'Mon Jan 24 12:29:21 2005'))
               3 UNPACK_SEQUENCE          3
               6 STORE_FAST               0 (a)
               9 STORE_FAST               1 (b)
              12 STORE_FAST               2 (deftime)

   3          15 LOAD_FAST                0 (a)
              18 LOAD_FAST                1 (b)
              21 BUILD_TUPLE              2
              24 PRINT_ITEM
              25 PRINT_NEWLINE

   4          26 LOAD_FAST                2 (deftime)
              29 PRINT_ITEM
              30 PRINT_NEWLINE

   5          31 LOAD_CONST               2 ('__main__')
              34 PRINT_ITEM
              35 PRINT_NEWLINE
              36 LOAD_CONST               0 (None)
              39 RETURN_VALUE

Regards,
Bengt Richter


    Reply to author    Forward  
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Bengt Richter  
View profile  
 More options Jan 25 2005, 4:18 am
Newsgroups: comp.lang.python
From: b...@oz.net (Bengt Richter)
Date: Tue, 25 Jan 2005 09:18:38 GMT
Local: Tues, Jan 25 2005 4:18 am
Subject: Re: specifying constants for a function (WAS: generator expressions: performance anomaly?)

Not much reaction, so I guess I won't see it if any, since I am going off line for a while.
BTW, this exercise makes me realize that you could do some other interesting things too,
if you put your mind to it. E.g., it might be interesting to inject properties into
the local namespace of a function, so that e.g., x=1 would would trigger an effect
like someobj.x = 1, where type(someobj).x was a property. It wouldn't be
too hard to make

    @addprop(verbose=True, x=property(getx, setx, delx))
    def foo():
        x = 123
        return x

etc. Maybe could be used to monitor access to specific locals for debug,
or other uses. Haven't even begun to think about that.

Another thing is to turn selected local defs into constants, since executing
the aggregation-of-parts code that accomplished the def may well be unchanging.
This would reduce the execution cost of nested functions. I guess that is
a specific case of a more general sticky-evaluate-rhs-once kind of directive
for locals that are only assigned in one place. special sentinels could
be used in the decorator keyword values to indicate this treatment of the
associated name, e.g.
    from presets import presets
    @presets(x=presets.STICKY, y=123)
    def foo(y):
        x = initx() # done once on the first call, making x references constant after that
        return x*y

Another thing that could be done is to inject pre and post things like atexit stuff.
Also maybe some adaptation things that is just a buzzword so far that I haven't explored.

BTW, is there another op that needs relocation besides JUMP_ABSOLUTE?

I was wondering about doing code mods by messing with an ast, but a decorator would
have to chase down the source and reparse I guess, in order to do that. It would be
interesting to be able to hook into ast stuff with some kind of metacompile hook (hand waving ;-)

Bye for a while.

Regards,
Bengt Richter


    Reply to author    Forward  
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Steven Bethard  
View profile  
 More options Jan 25 2005, 1:58 pm
Newsgroups: comp.lang.python
From: Steven Bethard <steven.beth...@gmail.com>
Date: Tue, 25 Jan 2005 11:58:30 -0700
Local: Tues, Jan 25 2005 1:58 pm
Subject: Re: specifying constants for a function (WAS: generator expressions: performance anomaly?)

Sorry, it's definitely cool -- I was just waiting for you to post the
Cookbook link so I could see how you did it. ;)

Steve


    Reply to author    Forward  
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
End of messages
« Back to Discussions « Newer topic     Older topic »

Create a group - Google Groups - Google Home - Terms of Service - Privacy Policy
©2009 Google