HTML-sanitizing helper
Mike Orr
webhelpers.html.converters.sanitize(), defined in
webhelpers.html.render. I'm not sure I'm satisfied with it though.
It strips all tags but leaves their content.
This would handle:
I <i>really</i> like <script language="javascript"></script> steak!
=>
I really like steak!
On the other hand it lets this through:
I <i>really</i> like <script language="javascript">NEFARIOUS
CODE</script> steak!
=>
I really like NEFARIOUS CODE steak
I'm not sure if that can be exploited. It also doesn't resolve HTML
entities. Should it? Because the Javascript may have entities or raw
<'s meant for comparisions.
The HTML parser (Python's HTMLParser) lets raw <'s surrounded by
whitespace through:
A < B
=>
A < B
But raises a fit if it looks like an unfinished tag:
A <B
=>
HTMLParser.HTMLParseError: EOF in middle of construct, at line 1, column 3
This means we can't make a converter that handles all pathological
output without significant work.
I could strip the tag *and* the content, which would remove the
embedded Javascript but make users wonder where their <i> content
went, potentially leading to unreadable text.
PHP's strip_tags just strips the tags but leaves the content, so maybe
that's enough?
http://us2.php.net/manual/en/function.strip-tags.php
The manpage has this caveat:
Because strip_tags() does not actually validate the HTML, partial,
or broken tags can result
in the removal of more text/data than expected.
I've got several various patches implemented in WebHelpers tip. I'll
probably release the beta in a few days, although I would like to give
it a proper manual before final. But I'm still learning how to set
that up with Sphinx.
--
Mike Orr <slugg...@gmail.com>
Mark T.
I had a need for HTML "sanitizing" in the past. When I went searching
for existing code to handle the task, I found Genshi provided quite a
bit of functionality. However, the terminology they use (and I
believe Rails and other libraries) is a little different. Loosely:
HTML sanitizing: remove some or all HTML elements from a string,
using whitelists of allowed HTML elements and HTML attributes
Text serialization: produce plain text from markup streams
The Genshi docs have a pretty good example of filtering a stream to
both sanitize the contents of some HTML tags and then render it to
plaintext:
http://genshi.edgewall.org/wiki/Documentation/0.5.x/streams.html#serialization
Since the Webhelpers library is quite widely used, I thought it may be
more consistent to use similar terminology with other libraries, and
possibly provide the same two types of distinct functionality.
For the curious, in my own code, I performed sanitizing and
serialization a little differently than the Genshi doc examples. It
may not be efficient, but it worked for my purposes:
from genshi.input import HTML
from genshi.filters import HTMLSanitizer
safe_tags = frozenset(['a','b','br','em',... ])
safe_attrs = frozenset(['align','alt',... ])
safe_schemes = frozenset(['ftp', 'http', 'https', 'mailto', None])
def sanitize_html(content_str):
markup = HTML(content_str) | HTMLSanitizer(safe_tags,
safe_attrs, safe_schemes)
return markup.render('html')
def serialize_to_plaintext(content_str):
markup = HTML(content_str)
return markup.render('text')
Best,
Mark
Mike Orr
just leave it out of WebHelpers for now in favor of this recipe?
I started to put support for safe tags, but took it out because it
makes the code significantly more complicated and still doesn't handle
the pathological or unbalanced cases. But Genshi doesn't handle
unbalanced cases at all; it just raises an error if the input is not
well-formed XML. But we can't expect users to input well-formed XML,
or even to know what that means. My intention was to support the
limited markup some newspaper sites allow (<i>, <a>, and a few
others). But given that this is not all that urgent, I was reluctant
to make the basic code more complex for it.
--Mike
Mike Orr <slugg...@gmail.com>