non-nested search

25 views
Skip to first unread message

تامر

unread,
Oct 28, 2009, 9:35:15 AM10/28/09
to beautifulsoup
When a search is run (findAll) on a match, beautiful soup will
continue and search the matched item's children. Is it possible to
prevent this behavior?

I couldn't figure it out, so I wrote the following code which works
(although it is slow):

while True:
divTag = soup.find('div', id=re.compile("post_message"))
if divTag != None:
threadContentList.append( deepcopy(divTag) )
divTag.decompose()
else:
break

Originally, I thought the recursive flag is what I needed to use.
However if recursive is set to false, the function will not search
beyond the first tag. If it is set to true, the function will search
all of the text:

soup = BeautifulSoup(<table><tr><td><table><tr><td></td></tr></table></
td></tr></table>)

# These first two examples seem to show us that "recursive" provides
us with the correct results
soup.findall("table, recursive=True)
>>[<table><tr><td><table><tr><td></td></tr></table></td></tr></table>, <table><tr><td></td></tr></table>, ]

soup.findall("table", recursive=False)
>> [<table><tr><td><table><tr><td></td></tr></table></td></tr></table>, ]

# However, searching something that is not the first element exposes
the issue
soup.findall("tr", recursive=True)
>>[<tr><td><table><tr><td></td></tr></table></td></tr>, <tr><td></td></tr>, ]

soup.findall("tr", recursive=False)
>>[ ]

Any thoughts?
Tim

Aaron DeVore

unread,
Oct 28, 2009, 5:31:03 PM10/28/09
to beauti...@googlegroups.com
Beautiful Soup doesn't have anything that works for you, but you can
walk the tree yourself. You might be able to modify the version of
recursiveChildGenerator on Git Hub. Start a result list. Replace the
body of the inner loop (while current is not stopNode) to append when
a node matches what you need. If it matched, set current to
current._getRecursiveChild().next. Else, set current to current.next.
I think that *might* work.

-Aaron

2009/10/28 تامر <timm...@gmail.com>:

Aaron DeVore

unread,
Oct 28, 2009, 5:33:00 PM10/28/09
to beauti...@googlegroups.com
The Git Hub version line number is at
http://github.com/adevore/old-beautiful-soup/blob/master/BeautifulSoup.py#L869

2009/10/28 Aaron DeVore <aaron....@gmail.com>:
Reply all
Reply to author
Forward
0 new messages