Feature proposal: the new math
Cliff Wells
# test.py
from breve.tags.html import tags
from breve.flatten import flatten
globals ( ).update ( tags )
menu_items = [
{ 'href': '/home', 'data': 'Home', 'class': 'menu-item' },
{ 'href': '/about', 'data': 'About', 'class': 'menu-item' },
]
template = html [
body [
ul [
li ( class_ = '$class' ) [
a ( href='$href', class_ = '${class}-link' ) [ '$data' ]
] * menu_items
]
]
]
print flatten ( template )
<html>
<body>
<ul>
<li class="menu-item">
<a class="menu-item-link" href="/home">Home</a>
</li>
<li class="menu-item">
<a class="menu-item-link" href="/about">About</a>
</li>
</ul>
</body>
</html>
Make sense? It uses Python's string.Template to do the substitution
(specifically safe_substitute). It's not going to be super fast because
it has to generate a bunch of new tags (turns one tag into many - and
that includes multiplying children), then traverse that tree of tags and
apply substitution to the attributes and any children that happen to be
strings, but I don't expect it to be horrifically slow either.
Of course, it has no effect on speed if you don't use it. And it's only
16 lines of code in one class, so it's not really "bloat".
Some people dislike overloading operators, so another potential way to
achieve this same effect (not implemented yet) would be via a new
directive:
ul [
foreach ( menu_items ) [
li ( class_ = '$class' ) [
a ( href='$href', class_ = '${class}-link' ) [ '$data' ]
]
]
]
And yet another way might be to use a custom attribute:
ul [
li ( class_ = '$class', foreach = menu_items ) [
a ( href='$href', class_ = '${class}-link' ) [ '$data' ]
]
]
I have no real preference (although the multiplication method is already
working).
I know this list is quiet, but I'd really be interested in hearing
feedback from people, pro or con, or alternate suggestions for achieving
a similar goal.
Regards,
Cliff
Cliff Wells
*please* do not use it in production code as the API is still subject to
change (or complete removal).
Please test and let me know what you think.
Regards,
Cliff
Cliff Wells
>
> (specifically safe_substitute). It's not going to be super fast
> because it has to generate a bunch of new tags (turns one tag into
> many - and that includes multiplying children), then traverse that
> tree of tags and apply substitution to the attributes and any children
> that happen to be strings, but I don't expect it to be horrifically
> slow either.
Actually, my concerns about speed were completely unfounded:
from breve.tags.html import tags
from breve.flatten import flatten
globals ( ).update ( tags )
from time import time
menu_items = [
{ 'href': '/home', 'data': 'Home', 'class': 'menu-item' }
] * 10000
template1 = html [
body [
ul [
li ( class_ = '$class' ) [
span [
' - ',
a ( href='$href', class_ = '${class}-link' ) [ '$data' ]
],
] * menu_items
]
]
]
template2 = html [
body [
ul [ [
li ( class_ = _m [ 'class' ] ) [
span [
' - ',
a ( href = _m [ 'href' ], class_ = '%(class)s-link' % _m ) [ _m [ 'data' ] ]
],
] for _m in menu_items
] ]
]
]
t = time ( )
flatten ( template1 )
total1 = time ( ) - t
t = time ( )
flatten ( template2 )
total2 = time ( ) - t
print total1, total2
>>> 1.83087301254 1.83428692818
This isn't a scientific test, but it demonstrates that the difference is
negligible, even for a list with 10,000 items.
Tests were run on a 1.7GHz Centrino laptop w/2GB of RAM.
Regards,
Cliff
josh...@gmail.com
> >>> 1.83087301254 1.83428692818
That .004 seconds could add up. I am a little concerned that we will
have serious speed issues if we _don't_ implement this.
Cliff Wells
>
> > >>> 1.83087301254 1.83428692818
>
> That .004 seconds could add up. I am a little concerned that we will
> have serious speed issues if we _don't_ implement this.
That must be *really* new math, since the new way was 0.004s slower ;-)
Cliff
josh...@gmail.com
>
> > That .004 seconds could add up. I am a little concerned that we will
> > have serious speed issues if we _don't_ implement this.
>
> That must be *really* new math, since the new way was 0.004s slower ;-)
>
> Cliff
check it on my system. so I cannot verify.
josh...@gmail.com
On Apr 5, 12:24 pm, Cliff Wells <cl...@twisty-industries.com> wrote:
>
> > > >>> 1.83087301254 1.83428692818
>
> > That .004 seconds could add up. I am a little concerned that we will
> > have serious speed issues if we _don't_ implement this.
>
> That must be *really* new math, since the new way was 0.004s slower ;-)
>
> Cliff
computing equivalent of an Escalade SUV?
>>>
>>> print total1, total2
0.00208711624146 0.625516891479
Yeah, how do you like them apples? /me boggles.
Cliff Wells
Eh, sorry, read the results backwards :P Hey, that's how close they
are!
Cliff
Cliff Wells
On Sat, 2008-04-05 at 12:59 -0700, josh...@gmail.com wrote:
> You know that "dual core" of mine that you seems to think is the
> computing equivalent of an Escalade SUV?
>
> >>>
> >>> print total1, total2
> 0.00208711624146 0.625516891479
>
> Yeah, how do you like them apples? /me boggles.
That's kinda (really) weird... what version of Python? I mean, if it's
correct, that's outstanding, but I'm suspicious. Maybe run it again?
Or better yet, run each test in a loop:
t = time ( )
for i in range ( 1000 ):
flatten ( template1 )
total1 = time ( ) - t
t = time ( )
for i in range ( 1000 ):
flatten ( template2 )
total2 = time ( ) - t
Cliff
Cliff Wells
On Sat, 2008-04-05 at 11:07 -0700, Cliff Wells wrote:
> Actually, my concerns about speed were completely unfounded:
>
> >>> 1.83087301254 1.83428692818
Bah. Talk about measuring the wrong thing...
Here's the correct test:
import sys
from time import time
from breve.tags import render_pattern as pattern
from breve.tags.html import tags
from breve.flatten import flatten
globals ( ).update ( tags )
menu_items = [
{ 'href': '/home', 'data': 'Home', 'class': 'menu-item' }
] * 10000
test = sys.argv [ 1 ]
t = time ( )
if test == '1':
template = html [
body [
ul [
li ( class_ = '$class' ) [
span [
' - ',
a ( href='$href', class_ = '${class}-link' ) [ '$data' ]
],
] * menu_items
]
]
]
if test == '2':
template = html [
body [
ul [ [
li ( class_ = _m [ 'class' ] ) [
span [
' - ',
a ( href = _m [ 'href' ], class_ = '%(class)s-link' % _m ) [ _m [ 'data' ] ]
],
] for _m in menu_items
] ]
]
]
total = time ( ) - t
print "time to run", total
t = time ( )
flatten ( template )
total = time ( ) - t
print "time to flatten", total
[root@portableevil trunk]# python test.py 1
time to run 21.2399451733
time to flatten 2.14032316208
[root@portableevil trunk]# python test.py 2
time to run 1.14334893227
time to flatten 3.54391717911
[root@portableevil trunk]# python test.py 1
time to run 20.7446119785
time to flatten 1.91219210625
[root@portableevil trunk]# python test.py 2
time to run 1.6469810009
time to flatten 2.26762390137
So the new method is *considerably* slower. They just take about the
same amount of time to flatten (as they should, since at the time of
flattening the DOMs are equivalent).
I'll see if I can't do some optimization, but ultimately this might be
the cost of convenience in this case.
I might test with plain Python string interpolation as well since it's
around 10x faster than string.Template.
Sorry for the false reports.
Regards,
Cliff
Cliff Wells
On Sat, 2008-04-05 at 13:52 -0700, Cliff Wells wrote:
>
> On Sat, 2008-04-05 at 11:07 -0700, Cliff Wells wrote:
> > Actually, my concerns about speed were completely unfounded:
>
> >
> > >>> 1.83087301254 1.83428692818
>
> Bah. Talk about measuring the wrong thing...
> [root@portableevil trunk]# python test.py 1
> time to run 20.7446119785
> time to flatten 1.91219210625
>
> [root@portableevil trunk]# python test.py 2
> time to run 1.6469810009
> time to flatten 2.26762390137
>
> So the new method is *considerably* slower. They just take about the
> same amount of time to flatten (as they should, since at the time of
> flattening the DOMs are equivalent).
>
> I'll see if I can't do some optimization, but ultimately this might be
> the cost of convenience in this case.
Or maybe not so much. By eliminating deepcopy and using copy on demand
I avoid traversing the tree twice (once for the deepcopy and once to
modify elements). This gave a nice boost (still 10k elements in list):
[root@portableevil trunk]# python test.py 1
time to run 5.17018699646
time to flatten 2.11776804924
20.7s to 5.2s ain't too bad for a few minutes of work =)
Switching from string.Template shaved off another 0.5s, but I don't feel
the loss of robustness and simplicity is worth it (string.Template won't
throw an error on missing dict keys or stray $'s).
I may try to flatten the tree recursion into iteration to see what
effect that has (besides code bloat =).
Regards,
Cliff
josh...@gmail.com
> I avoid traversing the tree twice (once for the deepcopy and once to
> modify elements). This gave a nice boost (still 10k elements in list):
>
> [root@portableevil trunk]# python test.py 1
> time to run 5.17018699646
> time to flatten 2.11776804924
>
> 20.7s to 5.2s ain't too bad for a few minutes of work =)
>
> Switching from string.Template shaved off another 0.5s, but I don't feel
> the loss of robustness and simplicity is worth it (string.Template won't
> throw an error on missing dict keys or stray $'s).
>
> I may try to flatten the tree recursion into iteration to see what
> effect that has (besides code bloat =).
>
problem with my first test - I don't seem to actually _have_ the
correct stuff in Breve. At least, it's no appearing. Tell me, have
you in fact commited properly to SVN?
http://ashbyte.com/breve/changeset/277 - It looks like it, but I just
checked out from easy_installed from svn:
easy_install -Z http://svn.ashbyte.com/breve/trunk/
And I get this:
Tulkas:~ joshua$ python test1.py 1
Traceback (most recent call last):
File "test1.py", line 3, in <module>
Any advice?
Sincerely,
Joshua
Cliff Wells
On Sat, 2008-04-05 at 15:11 -0700, josh...@gmail.com wrote:
> I think overall .5s is no reason to ignore good syntax. I found the
> problem with my first test - I don't seem to actually _have_ the
> correct stuff in Breve. At least, it's no appearing. Tell me, have
> you in fact commited properly to SVN?
>
> http://ashbyte.com/breve/changeset/277 - It looks like it, but I just
> checked out from easy_installed from svn:
>
> easy_install -Z http://svn.ashbyte.com/breve/trunk/
>
> And I get this:
> Tulkas:~ joshua$ python test1.py 1
> Traceback (most recent call last):
> File "test1.py", line 3, in <module>
> from breve.tags import render_pattern as pattern
> ImportError: cannot import name render_pattern
>
> Any advice?
Yes, try again. That was an abortive attempt at a custom render tag.
Cliff
josh...@gmail.com
> Yes, try again. That was an abortive attempt at a custom render tag.
>
> Cliff
time to flatten 0.584090948105
Tulkas:~ joshua$ python test1.py 2
time to run 0.327229976654
time to flatten 0.586398124695
Tulkas:~ joshua$
Wow, small difference. Python 2.5.1, MacOS X Tho
cbrain
In view of my posting in the "Brevé 2.0 API" thread, I would actually
prefer to write it like this, which will also be a lot faster because
string.Template does not have to be used here and you can actually
just write the following into a Brevé .b file:
html [
body [
macro("menu_item", lambda params: [
li ( class_=param.class_ ) [
a ( href=params.href, class_=params.class_ + '-link' )
[ params.data ]
]),
ul [
[menu_item(args) for args in menu_items]
]
]
]
This seems like a much more natural way to do it, in my opinion.
Nothing special here. No string.Template, no dollar constructions, the
ability to use Python expressions everywhere, etc. No special
multiplication operator either.
--
Sven
cbrain
[menu_item(*args) for args in menu_items]
instead of without the asterisk. The object "menu_item" that is
created in the variable namespace because of the macro() call should
have a __call__() method that calls its given function with all the
named parameters in a Brevé Namespace object, so that you can easily
access the contents of the dictionary with the attribute accessors
instead of having to write the square brackets to access them.
--
Sven
Cliff Wells
On Thu, 2008-04-10 at 14:35 -0700, cbrain wrote:
> Hello Cliff,
>
> In view of my posting in the "Brevé 2.0 API" thread, I would actually
> prefer to write it like this, which will also be a lot faster because
> string.Template does not have to be used here and you can actually
> just write the following into a Brevé .b file:
>
> html [
> body [
> macro("menu_item", lambda params: [
> li ( class_=param.class_ ) [
> a ( href=params.href, class_=params.class_ + '-link' )
> [ params.data ]
> ]),
> ul [
> [menu_item(args) for args in menu_items]
> ]
> ]
> ]
>
> This seems like a much more natural way to do it, in my opinion.
> Nothing special here. No string.Template, no dollar constructions, the
> ability to use Python expressions everywhere, etc. No special
> multiplication operator either.
I like this idea. I'll see about getting it into SVN for testing.
Cliff
Cliff Wells
On Thu, 2008-04-10 at 14:35 -0700, cbrain wrote:
>
> In view of my posting in the "Brevé 2.0 API" thread, I would actually
> prefer to write it like this, which will also be a lot faster because
> string.Template does not have to be used here and you can actually
> just write the following into a Brevé .b file:
>
> html [
> body [
> macro("menu_item", lambda params: [
> li ( class_=param.class_ ) [
> a ( href=params.href, class_=params.class_ + '-link' )
> [ params.data ]
> ]),
> ul [
> [menu_item(args) for args in menu_items]
> ]
> ]
> ]
>
> This seems like a much more natural way to do it, in my opinion.
> Nothing special here. No string.Template, no dollar constructions, the
> ability to use Python expressions everywhere, etc. No special
> multiplication operator either.
Does this seem close enough?
from breve.tags.html import tags
from breve import Namespace
globals ( ).update ( tags )
class Macro ( object ):
def __init__ ( self, name, function ):
self.name = name
self.function = function
def __call__ ( self, kw ):
ns = Namespace ( kw )
return self.function ( ns )
def macro ( name, function ):
# we'd probably do this in real life: register_global ( name, Macro ( name, function ) )
globals ( )[ name ] = Macro ( name, function )
data = [
dict ( href = '/', label = 'Home' ),
dict ( href = '/about', label = 'About' ),
dict ( href = '/help', label = 'Help' )
]
template = html [
body [
macro ( 'mymacro', lambda _ns:
li [
a ( href = _ns.href ) [ _ns.label ]
]
),
ul [
[ mymacro ( _d ) for _d in data ]
]
]
]
print template
Regards,
Cliff
Cliff Wells
> Does this seem close enough?
>
>
> from breve.tags.html import tags
> from breve import Namespace
> globals ( ).update ( tags )
>
>
> class Macro ( object ):
> def __init__ ( self, name, function ):
> self.name = name
> self.function = function
>
> def __call__ ( self, kw ):
> ns = Namespace ( kw )
> return self.function ( ns )
>
> def macro ( name, function ):
> # we'd probably do this in real life: register_global ( name, Macro ( name, function ) )
> globals ( )[ name ] = Macro ( name, function )
+ return ''
>
> data = [
> dict ( href = '/', label = 'Home' ),
> dict ( href = '/about', label = 'About' ),
> dict ( href = '/help', label = 'Help' )
> ]
>
> template = html [
> body [
> macro ( 'mymacro', lambda _ns:
> li [
> a ( href = _ns.href ) [ _ns.label ]
> ]
> ),
> ul [
> [ mymacro ( _d ) for _d in data ]
> ]
> ]
> ]
>
> print template
>
Oops, this outputs "None" in the template at the point of the macro
definition. The above change fixes it.
Cliff
Cliff Wells
However, getting it to work required a bit of hack that I feel a bit
dubious about:
# breve.tags.__init__.py
def caller ( ):
return sys._getframe ( 2 )
class Macro ( object ):
def __init__ ( self, name, function ):
self.name = name
self.function = function
def __call__ ( self, kw ):
ns = Namespace ( kw )
return self.function ( ns )
def macro ( name, function ):
frame = caller ( )
frame.f_globals [ name ] = Macro ( name, function )
return ''
Messing with internals isn't something I like to do (although I believe
this to be a well-supported method). On the plus side, macros can work
both inside a template and standalone:
from breve.tags import macro
from breve.tags.html import tags
globals ( ).update ( tags )
data = [
dict ( href = '/', label = 'Home' ),
dict ( href = '/about', label = 'About' ),
dict ( href = '/help', label = 'Help' )
]
template = html [
body [
macro ( 'mymacro', lambda _ns:
li [
a ( href = _ns.href ) [ _ns.label ]
]
),
ul [
[ mymacro ( _d ) for _d in data ]
]
]
]
print template
<html>
<body>
<ul>
<li><a href="/">Home</a></li>
<li><a href="/about">About</a></li>
<li><a href="/help">Help</a></li>
</ul>
</body>
</html>
Regards,
Cliff
Cliff Wells
from breve.tags.html import tags
from breve.tags import macro, assign
from breve.flatten import flatten
globals ( ).update ( tags )
external_links = [
dict ( href = 'http://www.google.com', label = 'Google' ),
dict ( href = 'http://www.yahoo.com', label = 'Yahoo!' ),
dict ( href = 'http://www.amazon.com', label = 'Amazon' )
]
internal_links = [
dict ( href = '/', label = 'Home' ),
dict ( href = '/about', label = 'About' ),
dict ( href = '/help', label = 'Help' )
]
template = (
macro ( 'mymacro', lambda href, label, **kw:
li [
a ( href = href, class_ = mystyle ) [ label ]
]
),
html [
body [
assign ( 'mystyle', 'internal-link' ),
ul [
[ mymacro ( **_d ) for _d in internal_links ]
],
assign ( 'mystyle', 'external-link' ),
ul [
[ mymacro ( **_d ) for _d in external_links ]
]
]
]
)
And of course, this would render as:
<html>
<body>
<ul>
<li><a class="internal-link" href="/">Home</a></li>
<li><a class="internal-link" href="/about">About</a></li>
<li><a class="internal-link" href="/help">Help</a></li>
</ul>
<ul>
<li><a class="external-link" href="http://www.google.com">Google</a></li>
<li><a class="external-link" href="http://www.amazon.com">Amazon</a></li>
<li><a class="external-link" href="http://www.amazon.com">Yahoo!</a></li>
</ul>
</body>
</html>
The obvious advantages I see are:
1) more efficient (no extra Namespace objects created)
2) more explicit (esp. lambda declaration)
3) slightly less magic - easier to explain
If no one objects I'll be going this route instead of the previous way.
Also, note the new "assign" feature. Getting macros to work had some side-effects =)
Regards,
Cliff