Maximum recursion depth issue in rendering templates
20 views
Skip to first unread message
Richard Liao
Sep 25, 2008, 1:27:29 PM9/25/08
to Genshi
I have met a trac crash when I was trying to create new ticket or
access admin pages.
The trac crashes hardly with error: "segmentation fault" in my FreeBSD
box.
After some digging, I found the problem lays in genshi's transform
module.
The following is a test script to reproduce that hard crashe by
setting recursion limit exceeds the platform's capabilities:
{{{
import sys
from genshi.input import HTML
from genshi.builder import tag
from genshi.filters.transform import Transformer
sys.setrecursionlimit(1000 * 20)
stream = HTML('<html><head><title>Some Title</title></head>'
'<body>Some <em>body</em> text.</body></html>')
for i in xrange(1000 * 10):
stream = stream | Transformer('body').prepend(tag.h1('Document
Title'))
print stream.render()
}}}
If comment out setrecursionlimit line to use system default recursion
limit, it raises exception: "RuntimeError: maximum recursion depth
exceeded".
{{{
File ".../genshi/filters/transform.py", line 686, in _unmark
for mark, event in stream:
File ".../genshi/filters/transform.py", line 1129, in __call__
for mark, event in stream:
File ".../genshi/filters/transform.py", line 713, in __call__
for mark, event in stream:
File ".../genshi/filters/transform.py", line 682, in _mark
for event in stream:
File ".../genshi/core.py", line 267, in _ensure
event = stream.next()
File ".../genshi/filters/transform.py", line 686, in _unmark
for mark, event in stream:
File ".../genshi/filters/transform.py", line 1129, in __call__
for mark, event in stream:
File ".../genshi/filters/transform.py", line 713, in __call__
for mark, event in stream:
File ".../genshi/filters/transform.py", line 682, in _mark
for event in stream:
File ".../genshi/core.py", line 267, in _ensure
event = stream.next()
...
}}}
The problem is that the transform module is in very deep and unlimited
recursion.
I am not sure why trac crashes siliently with no exception raised
after I searched trac source codes but can't find any lines about
setrecursionlimit.
I think it's a problem because we can'nt control how many Transformers
will be applied to a template since Transformer always be used by 3rd
party plugins or something else to manipulate template. System
default recursion limit 1000 can be easily exceeded by just add
several complicated transforms.
Regards,
PS, the original post in trac-dev:
http://groups.google.com/group/trac-dev/browse_thread/thread/d62feffee3e324d3/68121cec191d78fe
access admin pages.
The trac crashes hardly with error: "segmentation fault" in my FreeBSD
box.
After some digging, I found the problem lays in genshi's transform
module.
The following is a test script to reproduce that hard crashe by
setting recursion limit exceeds the platform's capabilities:
{{{
import sys
from genshi.input import HTML
from genshi.builder import tag
from genshi.filters.transform import Transformer
sys.setrecursionlimit(1000 * 20)
stream = HTML('<html><head><title>Some Title</title></head>'
'<body>Some <em>body</em> text.</body></html>')
for i in xrange(1000 * 10):
stream = stream | Transformer('body').prepend(tag.h1('Document
Title'))
print stream.render()
}}}
If comment out setrecursionlimit line to use system default recursion
limit, it raises exception: "RuntimeError: maximum recursion depth
exceeded".
{{{
File ".../genshi/filters/transform.py", line 686, in _unmark
for mark, event in stream:
File ".../genshi/filters/transform.py", line 1129, in __call__
for mark, event in stream:
File ".../genshi/filters/transform.py", line 713, in __call__
for mark, event in stream:
File ".../genshi/filters/transform.py", line 682, in _mark
for event in stream:
File ".../genshi/core.py", line 267, in _ensure
event = stream.next()
File ".../genshi/filters/transform.py", line 686, in _unmark
for mark, event in stream:
File ".../genshi/filters/transform.py", line 1129, in __call__
for mark, event in stream:
File ".../genshi/filters/transform.py", line 713, in __call__
for mark, event in stream:
File ".../genshi/filters/transform.py", line 682, in _mark
for event in stream:
File ".../genshi/core.py", line 267, in _ensure
event = stream.next()
...
}}}
The problem is that the transform module is in very deep and unlimited
recursion.
I am not sure why trac crashes siliently with no exception raised
after I searched trac source codes but can't find any lines about
setrecursionlimit.
I think it's a problem because we can'nt control how many Transformers
will be applied to a template since Transformer always be used by 3rd
party plugins or something else to manipulate template. System
default recursion limit 1000 can be easily exceeded by just add
several complicated transforms.
Regards,
PS, the original post in trac-dev:
http://groups.google.com/group/trac-dev/browse_thread/thread/d62feffee3e324d3/68121cec191d78fe
Alec Thomas
Sep 26, 2008, 3:21:33 AM9/26/08
to gen...@googlegroups.com
The only solution to this that I can think of is to buffer the full
stream in between each filter, which of course has its own downsides.
It might be possible to automate this by tracking the number of
filters in a Transformer, and buffering when it hits some limit.
stream in between each filter, which of course has its own downsides.
It might be possible to automate this by tracking the number of
filters in a Transformer, and buffering when it hits some limit.
2008/9/26 Richard Liao <richard...@gmail.com>:
--
And the next thing you know, you're at the zoo, shaving a yak,
all so you can wax your car.
Reply all
Reply to author
Forward
0 new messages