[barry.hart@predictix.com: Bug when processing <select> tag without of a <form> tag]

[C. Titus Brown]

----- Forwarded message from Barry Hart <barry...@predictix.com> -----

From: Barry Hart <barry...@predictix.com>
To: ti...@idyll.org
Cc: jeremy...@predictix.com
Subject: Bug when processing <select> tag without of a <form> tag
Date: Tue, 5 Aug 2008 15:47:23 -0400 (EDT)
X-Mailer: Microsoft Office Outlook 11

Hi,

I was in the testing tutorial you gave a PyCon. You shared some great
information, and I really appreciated the chance to attend. I work with
Jeremy Jones at Predictix, who I think you may know. We are starting to
use Twill for server monitoring and load testing.

We noticed a problem recently where Twill was reporting a "nested SELECTs"
parse error on a page which looked fine to us. On further investigation,
this was the HTML causing the problem. It is a 'quick links' drop-down
which we include at the top of every page:

<select py:if="not tg.identity.anonymous" id="quick_link_menu">
<option value="">Quick links</option>
<option value="/promotion/list/create_new">Create a new
promotion</option>
<option
value="/promotion/list/?type=planned&amp;dates.begin=&amp;dates.end=">Find
a planned promotion</option>
</select>

Because events on this select tag control are processed on the browser
side by JavaScript, it is not enclosed in a <form> tag. This HTML
uncovered an apparent bug in twill/other_packages/_mechanize_dist -- I
know this is not your code but I wanted to contact you first to see if you
have suggestions on how to submit a fix for this -- 'mechanize' seems like
a weird package because it doesn't have its own mailing list, it suggests
posting to a more generic-sounding wwwsearc...@lists.sourceforge.net
mailing list.

Anyway, the problem itself seems pretty straightforward -- the following
function in ClientForm.py seems to have a bug. Because it exits
immediately if self._current_form is self._global_form, it leaves
self._select set if there is a <select> tag outside of a <form> tag.

def end_select(self):
debug("")
if self._current_form is self._global_form:
return
if self._select is None:
raise ParseError("end of SELECT before start")

if self._option is not None:
self._end_option()

self._select = None

Thus, any subsequent <select> tag will cause a parse error in
start_select():

def start_select(self, attrs):
debug("%s", attrs)
if self._select is not None:
raise ParseError("nested SELECTs")
if self._textarea is not None:
raise ParseError("SELECT inside TEXTAREA")
d = {}
for key, val in attrs:
d[key] = val

self._select = d
self._add_label(d)

self._append_select_control({"__select": d})

A possible fix is to to clear self._select here:
if self._current_form is self._global_form:
self._select = None # NEW CODE
return

or you can monkeypatch around it, as I'm doing:

# Patch the form parser class to ensure end select tags are processed
correctly
from twill.other_packages._mechanize_dist import ClientForm
form_parser_class =
b._browser._factory.soup_factory._forms_factory.form_parser_class
old_end_select = form_parser_class.end_select
def end_select(self):
result = old_end_select(self)
self._select = None
return result
form_parser_class.end_select = end_select

Either of these fixes works but with one apparent side effect - now all
the form indices are shifted by 1, e.g. form 1 becomes form 2.

Do you think it makes sense to address this in Twill or should I contact
the author of 'mechanize' on the email list I mentioned above?

Thanks,
Barry Hart

----- End forwarded message -----

--
C. Titus Brown, c...@msu.edu