Parsing a table with attributes in hebrew

[Akira Kir]

Hello.
I'm trying to parse website where <table> have no attributes except of 'summary' which written in hebrew.
My first try was

#!/usr/bin/env python2
# -*- coding: utf-8 -*-

from bs4 import BeautifulSoup as bs
import re

with open('Leumi.aspx','r') as f:
    data = f.read()


def parse(data):
    summary = "טבלת פירוט חיובים ועסקות"
    soup = bs(data, from_encoding='utf-8')
    data_tabs = soup.find('table',{'summary':summary})

In this case im getting error:

Traceback (most recent call last):
  File "LeumiCardParser.py", line 36, in <module>
    parse(data)
  File "LeumiCardParser.py", line 14, in parse
    data_tabs = soup.find('table',{'summary':summary})
  File "/usr/lib/python2.7/site-packages/beautifulsoup4-4.0.5-py2.7.egg/bs4/element.py", line 1108, in find
    l = self.find_all(name, attrs, recursive, text, 1, **kwargs)
  File "/usr/lib/python2.7/site-packages/beautifulsoup4-4.0.5-py2.7.egg/bs4/element.py", line 1128, in find_all
    return self._find_all(name, attrs, text, limit, generator, **kwargs)
  File "/usr/lib/python2.7/site-packages/beautifulsoup4-4.0.5-py2.7.egg/bs4/element.py", line 434, in _find_all
    found = strainer.search(i)
  File "/usr/lib/python2.7/site-packages/beautifulsoup4-4.0.5-py2.7.egg/bs4/element.py", line 1243, in search
    found = self.search_tag(markup)
  File "/usr/lib/python2.7/site-packages/beautifulsoup4-4.0.5-py2.7.egg/bs4/element.py", line 1215, in search_tag
    if not self._matches(attr_value, match_against):
  File "/usr/lib/python2.7/site-packages/beautifulsoup4-4.0.5-py2.7.egg/bs4/element.py", line 1300, in _matches
    match_against = markup.__class__(match_against)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xd7 in position 0: ordinal not in range(128)
 

second try was to add summary.encode('utf-8'), the error changed... but only a bit:

Traceback (most recent call last):
  File "LeumiCardParser.py", line 36, in <module>
    parse(data)
  File "LeumiCardParser.py", line 14, in parse
    data_tabs = soup.find('table',{'summary':summary.encode('utf-8')})
UnicodeDecodeError: 'ascii' codec can't decode byte 0xd7 in position 0: ordinal not in range(128)
 

There is 11 tables in the html, i need to fetch only one of them. all tables have the same problem - they have only "summary" attribute with value written in hebrew.

Thanks in advance :)

[Leonard Richardson]

This is a bug which will be fixed in the next version.

https://bugs.launchpad.net/beautifulsoup/+bug/1003974

Leonard

> --
> You received this message because you are subscribed to the Google Groups
> "beautifulsoup" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/beautifulsoup/-/TD2fqtHvY1AJ.
> To post to this group, send email to beauti...@googlegroups.com.
> To unsubscribe from this group, send email to
> beautifulsou...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/beautifulsoup?hl=en.

[Thomas Kluyver]

On Wednesday, May 23, 2012 2:19:32 PM UTC+1, Akira Kir wrote:

    summary = "טבלת פירוט חיובים ועסקות"

I don't know if it fixes your problem, you probably want to make this a unicode literal - prefix the string with a u:

summary = u"טבלת פירוט חיובים ועסקות"

Thomas