newbe with a question

[Vincent]

I have html with this tins in it
</td>
<td valign="top">
<table border="1" cellpadding="1" cellspacing="0" align="right">
<tbody><tr class="tableheaders">
<td>Owner Name(s)</td>
</tr>

<tr>

<td>PILCHER DONALD L </td>
</tr>

</tbody></table>
</td>
I am trying to get PILCHER DONALD L
This works but there has got to be a better way I just have not
figured it out.
x = soup('table', text = re.compile("Owner Name"))
print 'what I want is', x
[0].parent.parent.parent.tr.nextSibling.nextSibling.next.next.next

Help?
Thanks
Vincent

[Aaron DeVore]

How about this:

soup = (...)
label = soup.find(text="Owner Name(s)")
name = label.findNext('td')

Does that work?

-Aaron DeVore

> --
>
> You received this message because you are subscribed to the Google Groups "beautifulsoup" group.
> To post to this group, send email to beauti...@googlegroups.com.
> To unsubscribe from this group, send email to beautifulsou...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/beautifulsoup?hl=en.
>
>
>

[Aaron DeVore]

Correction:

soup = BeautifulSoup(...) # Forgot the BeautifulSoup part

label = soup.find(text="Owner Name(s)")

# Needs Tag.string to get to the actual name string
name = label.findNext('td').string


If you're doing a bunch of them, you can even go for a list comprehension.

names = [unicode(label.findNext('td').string) for label in
soup.findAll(text="Owner Name(s)")]

[Vincent]

Thanks, that worked and maybe I learned something.
Vincent

On Nov 30, 11:12 am, Aaron DeVore <aaron.dev...@gmail.com> wrote:
> Correction:
>
> soup = BeautifulSoup(...) # Forgot the BeautifulSoup part
> label = soup.find(text="Owner Name(s)")
>
> # Needs Tag.string to get to the actual name string
> name = label.findNext('td').string
>
> If you're doing a bunch of them, you can even go for a list comprehension.
>
> names = [unicode(label.findNext('td').string) for label in
> soup.findAll(text="Owner Name(s)")]
>

> On Mon, Nov 30, 2009 at 9:57 AM, Aaron DeVore <aaron.dev...@gmail.com> wrote:
> > How about this:
>
> > soup = (...)
> > label = soup.find(text="Owner Name(s)")
> > name = label.findNext('td')
>
> > Does that work?
>
> > -Aaron DeVore
>

[bo newbie in Finland]

Hi gurus,

I have a similar question.
The html looks like this (the "ul" tag is repeated within the "a" tag
for a second marriage with a later year):
...
<ul>
<li><em>Married:</em>
1658</li></ul>
...

I'm interested in the year only so I tried your code from above:

...
label = soup.find(text="Married:")
m_year = label.findNext('ul').string

#If you're doing a bunch of them, you can even go for a list
comprehension.

m2_years = [unicode(label.findNext('li').string) for label in
soup.findAll(text="Married:")]
print m_year
print m2_years

The result:
None
[u'None', u'None']

The following gives me both ul tags:
x2 = soup.td.findAll("ul")
x5 = str(x2[0]) # convert x2 to a
string so strip command work
x6 = x5.strip( '<ul>liemMarid:/ ' ) # works normally fine with
strings but not here?
x6 = x6.strip() # remove spaces and
possible line feed
x6 = x6.strip( '<li>/ ') # remove li tag?
x6 = str(x6) # 2:nd time to be
sure
x6 = x6.strip( 'em>') # remove em tag?
x6 = x6.strip( '</em>') # removes what?
print "x6:", x6

x6: Married:</em>
1658

I have been struggling almost all weekend with this little problem.
What did I do wrong?
Thanks for your help!

[Aaron DeVore]

This is a bit tricky to figure out, but doable. What you're trying to
do is get the next node after the label.

<li><em>Marriage:</em>This text is the next node</li>

The structure of the Beautiful Soup tree is like this:

li
<em>
Married:
This text is the next node

So let's say you get the first label:

>>> label = soup.findAll(text="Married:")[0]
>>> label
u'Married:'

The text you need is the next node

>>> label.next
u'\n 1658'

Make sure you don't have whitespace by using strip()

>>> label.next.strip()
u'1658'

There, you have it!

If you want to convert it to an int, use this:

>>> int(label.next)

int(...) automatically strips off whitespace.

To put this in a list comprehension, use unicode(...) to avoid keeping
the NavigableString around:

[unicode(label.next) for label in
soup.findAll(text="Married:")]

Or, for conversion to ints

[int(label.next) for label in
soup.findAll(text="Married:")]

You also might be able to use a regular expression to search for all
substrings that have exactly 4 digits.

Cheers!
Aaron DeVore





Alternatively, you could just use a regular expression (import re)
that searches the entire document. Just match any substrings have
exactly 4 digit characters

[bo newbie in Finland]

Hi Aaron!

Thanks a lot for a quick reply, you saved my night sleep and possibly
few days of work more before giving up ;-)
Yes, getting the label Married: was not so difficult but the data was.
I've been trying just about everything I found from documentation and
examples but no useful result.

Can you also explain my ugly example using strip and why that did not
work as a workaround?
I thought the strip string possibly got confused by the End Of Line in
the middle or that I've found a bug :-o

If I have a clean single line:

#OK:
x5 = '<ul><li><em>Married:</em> 1658</li></ul>'
x6 = x5.strip( '<ul>liemMaried:/ ' )
print x6
1658

But next is bad

#Problem with End Of Line char?:
x5 = '<ul>
<li><em>Married:</em>
1658</ul>'
x6 = x5.strip( '<ul>liemMaried:/ ' )
print x6

Well, the last example was not possible to copy paste from html into
Python so it's "fake" but my.

x2 = soup.td.findAll("ul")
x5 = str(x2[0])

x6 = x5.strip( '<ul>liemMarid:/ ' )

print x2
[<ul>
<li><em>Married:</em>
1658</li></ul>, <ul>
<li><em>Married:</em>
1672</li></ul>]

print x6

<li><em>Married:</em>
1658

I find this result weird and difficult to understand.

Again, thanks a thousands and all the best from the cold north where
we today celebrate the Finns independence.
Bo

[Aaron DeVore]

On Sun, Dec 6, 2009 at 1:31 PM, bo newbie in Finland <fre...@yahoo.com> wrote:
> Hi Aaron!
>
> Thanks a lot for a quick reply, you saved my night sleep and possibly
> few days of work more before giving up ;-)
> Yes, getting the label Married: was not so difficult but the data was.
> I've been trying just about everything I found from documentation and
> examples but no useful result.

Sleep = good. Asking = good. An excellent combination. And now that
we've had this exchange, there's some documentation for archive
searchers.

>
> Can you also explain my ugly example using strip and why that did not
> work as a workaround?
> I thought the strip string possibly got confused by the End Of Line in
> the middle or that I've found a bug :-o
>
> If I have a clean single line:
>
> #OK:
> x5 = '<ul><li><em>Married:</em>  1658</li></ul>'
> x6 = x5.strip( '<ul>liemMaried:/ ' )
> print x6
> 1658
>
> But next is bad
>
> #Problem with End Of Line char?:
> x5 = '<ul>
> <li><em>Married:</em>
>  1658</ul>'
> x6 = x5.strip( '<ul>liemMaried:/ ' )
> print x6

You didn't put \n into the string of characters that strip should
remove. It removed from both ends until it hit a newline. So, yes, it
was a problem with the end of line character.

> Well, the last example was not possible to copy paste from html into
> Python so it's "fake" but my.
>
> x2 = soup.td.findAll("ul")
> x5 = str(x2[0])
> x6 = x5.strip( '<ul>liemMarid:/ ' )
> print x2
> [<ul>
> <li><em>Married:</em>
>  1658</li></ul>, <ul>
> <li><em>Married:</em>
>  1672</li></ul>]
>
> print x6
> <li><em>Married:</em>
>  1658
> I find this result weird and difficult to understand.

It's exactly the same as I explained above. Beautiful Soup didn't
strip out the newline.

> Again, thanks a thousands and all the best from the cold north where
> we today celebrate the Finns independence.
> Bo

You're welcome. And happy whatever-you-call-it. :)

-Aaron DeVore

[bo newbie in Finland]

Hi Aaron!

This is a little embarrassing, I was pretty sure I've also tried to
include the EOL character but I've used slash /n instead of backslash
\n.
So no bug, only the syn-tax barking at me again.
Learning a new language, like Finnish or Python, is hard for an old
man with a hard head.
However, once you learn some then it feels better to learn more.

I have an other little problem more related to Python than
BeautifulSoup.
I'm trying to save data to a list from within a loop
It's printing nicely on the screen all the parents, spouses and
children to "my ancestor" but I've not yet figured out how to save the
data.
...
index = n = 0
for i in x:
for index in range(len(x2)):
n = n + 1
name = i.contents[0]
id = i.get("href").split("=")[-1]
id = int(id[1:])
print "%d: [%s] %s" % (n, id, name)
# save_data[n] = n, id[n], name[n]
...

I've got help also with this loop from a friend but my last line
obviously don't work, but maybe explains what I try to do.

Thanks', we just call it the independence day and is traditionally
celebrated with two candles in the windows for average people and a
big ball at the president's castle for the elite and the more or less
famous celebrities. Most women in the country are watching the ball at
TV and comparing ladies dresses and hairstyles.

Thank's for your kind help Aaron,

Bo

On Dec 6, 11:59 pm, Aaron DeVore <aaron.dev...@gmail.com> wrote:

[Aaron DeVore]

Bo,

I'm not precisely sure what you're trying to do, so a few questions.

- The indentation is a bit weird. Am I getting confused by that?

- What exactly do you mean by parents, spouses (siblings?), and children?

- Can you give me a better idea of what nodes should be outputted?

-Aaron DeVore

> Hi Aaron!
>
> This is a little embarrassing, I was pretty sure I've also tried to
> include the EOL character but I've used slash /n instead of backslash
> \n.

Ah, the joys of typos!

[bo newbie in Finland]

Hi Aaron!

Well, I'm mixing my genealogy hobby with my programming hobby where I
obviously lack the vocabulary.
With parents and so on I mean father, mother, husband/wife, ...

Not sure how this is defined in the programming language :-)

Hmmm, a copy paste error also. Shame on me!
I hope the indentation is correct now or at least it works very nice
printing all interesting data (people with id# in a tag) found inside
the td tag but I now see that I also forgot to include the definition
of x:
...
x = soup.td.findAll("a")

index = n = 0
for i in x:

n = n + 1
name = i.contents[0]
id = i.get("href").split("=")[-1]
id = int(id[1:])
print "%d: [%s] %s" % (n, id, name)

# save_data[n] = n, id[n], name[n]

So what I try to do with the last line (not working) is making a list
that I later can modify and save to a file:
save_data[1] = 1, fathers_id, fathers_firstname lastname
save_data[2] = 2, mothers_id, mothers_firstname lastname
save_data[3] = 3, spouse1s_id, spouse1s_firstname lastname
...

I hope this was better or at least clearer.

Regards,
Bo

[Aaron DeVore]

You could have a list of tuples.

result = []

...processing...
for i in x:
...processing...
result.append((n, id, name)) # the extra (...) creates a tuple

Then you can loop over the list later on using tuple unpacking

for n, id, name in result:
...do stuff...

That's the only way I can think of storing for later use.

-Aaron DeVore

[bo newbie in Finland]

Hi Aaron,

Thanks again for your kind help!

This is exactly what I tried to do but never got past the syntax
terror.
Some times I wish the programs were more forgiving sort of, and
instead of telling me that I don't know (I already know that I don't
know), could give me a little hint like:
"Is this what you're trying to do:"
And then give me a suggestion or two or a pointer to the
documentation.
Yes, based on the error message I should know how to proceed.

Maybe the error handling would then be larger than the actual program
so it's not worth the hassle?
Well, maybe one day.

Bo

On Dec 9, 1:26 am, Aaron DeVore <aaron.dev...@gmail.com> wrote:
> You could have a list of tuples.
>
> result = []
>
> ...processing...
> for i in x:
>     ...processing...
>     result.append((n, id, name)) # the extra (...) creates a tuple
>
> Then you can loop over the list later on using tuple unpacking
>
> for n, id, name in result:
>     ...do stuff...
>
> That's the only way I can think of storing for later use.
>
> -Aaron DeVore
>