Tag Attributes Can't Be UNICODE?

[Semigod King]

I encountered an issue with the following simple code:

from BeautifulSoup import BeautifulSoup

s = u"<input type=submit value=\u4e0b id=sb>"
soup = BeautifulSoup(s, fromEncoding="gbk")


I got exception like this:

Traceback (most recent call last):
File "D:\t.py", line 5, in <module>
soup = BeautifulSoup(s, fromEncoding="gbk")
File "C:\Python26\lib\site-packages\BeautifulSoup.py", line 1499, in
__init__
BeautifulStoneSoup.__init__(self, *args, **kwargs)
File "C:\Python26\lib\site-packages\BeautifulSoup.py", line 1230, in
__init__
self._feed(isHTML=isHTML)
File "C:\Python26\lib\site-packages\BeautifulSoup.py", line 1263, in
_feed
self.builder.feed(markup)
File "C:\Python26\lib\HTMLParser.py", line 108, in feed
self.goahead(0)
File "C:\Python26\lib\HTMLParser.py", line 148, in goahead
k = self.parse_starttag(i)
File "C:\Python26\lib\HTMLParser.py", line 263, in parse_starttag
% (rawdata[k:endpos][:20],))
File "C:\Python26\lib\HTMLParser.py", line 115, in error
raise HTMLParseError(message, self.getpos())
HTMLParser.HTMLParseError: junk characters in start tag: u'\u4e0b
id=sb>', at li
ne 1, column 1


Does anybody know how to solve this issue? Thank you.

[Semigod King]

I solved this issue by adding the following lines of code at the beginning of my code. It fixes a bug of HTMLParser module.

import HTMLParser
from BeautifulSoup import BeautifulSoup, NavigableString

# Bug fix of HTMLParser so that BeautifulSoup could parse page returned from
# Baidu.
HTMLParser.attrfind = re.compile(
r'\s*([a-zA-Z_][-.:a-zA-Z_0-9]*)(\s*=\s*'
r'(\'[^\']*\'|"[^"]*"|[^\s>^\[\]{}\|\'\"]*))?')