Modifying the response body

[JD Rake]

Dear Bottle Group,

I am trying to write a plug-in to modify the response body for a view that serves .html static files. The objective is to modify the html source file before it is served.

My current implementation is listed at the bottom. I'm requesting help with the following issues:

1. When I replace the <!-- Editor Menu --> block in the html for the response body, I get a ResourceWarning exception. It appears that _file_iter_range in bottle.py isn't closing the file at the end of my generator--the replace method.

.pyenv/versions/3.6.8/lib/python3.6/wsgiref/handlers.py:179: ResourceWarning: unclosed file <_io.BufferedReader name=

2. Are there performance shortcomings to this sort of approach? I've used a generator, which I believe maintains the chunked loading of the static file.

3. Are there obvious security shortcomings? I've made the replace method replace in the header of the html file. The rest of the html file is properly escaped as well.

Thank you for your help,

JD

class OwnerMenuPlugin(object):
    """A plugin for adding options for owners of a project."""

    api = 2

    re_menu_html = regex.compile(b'<!-- Editor Menu -->')

    def replace(self, template, body):
        in_header = True
        for item in body:
            if in_header:
                if b'</head>' in item:
                    in_header = False
                item = self.re_menu_html.sub(template, item)

            yield item
        raise StopIteration

    def apply(self, callback, context):
        def wrapper(*args, **kwargs):
            response = callback(*args, **kwargs)
            if 'path' in kwargs and '.html' in kwargs['path']:
                # Only process html files served
                # First get the template
                template = jinja2_template('menu.html').encode('utf-8')  # string

                # process the links in the body, producing a generator
                response.body = self.replace(template=template,
                                             body=response.body)

            return response
        return wrapper

```

[Marcel Hellkamp]

Hi JD,

1) You have to call `close()` on the body iterator if it is defined, as per WSGI spec.

2) Yes, it will be horrible slow. A simple str.replace instead of a regular expression would be better, but still... why!?

3) It depends. If you do not control the body, which would be the only reason of writing this plugin in the first place, than it may contain all kinds of nasty scripts that are then executed in the context of your own website.

Why are you doing that in the first place?

--
--
You are member of the "bottlepy" group at google groups.
See http://groups.google.de/group/bottlepy for mailing list options.
See http://bottlepy.org/ for news and documentation.

---
You received this message because you are subscribed to the Google Groups "bottlepy" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bottlepy+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bottlepy/610f2f4a-42d9-44fb-b217-071fe61d2892%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[JD Rake]

Thanks Marcel,

I've written a document processing engine, like markdown. The generated html files could be processed on the fly, but these are large enough that it would be worth saving the html file, which can be served by itself. I'd nevertheless like to modify these html files to include additional html elements, like custom menus, based on authenticated views.

Another approach would be to save the html file with some embedded jinja2 variables that could be rendered, if needed.

JD

To unsubscribe from this group and stop receiving emails from it, send an email to bott...@googlegroups.com.

[Marcel Hellkamp]

On 6/4/19 5:01 PM, JD Rake wrote:
> I've written a document processing engine, like markdown. The
> generated html files could be processed on the fly, but these are
> large enough that it would be worth saving the html file, which can be
> served by itself. I'd nevertheless like to modify these html files to
> include additional html elements, like custom menus, based on
> authenticated views.

That makes sense. But how about, instead of caching the entire document,
only cache the important bits (title, body, metadata) and pass these to
a template to render the user-specific page?

> Another approach would be to save the html file with some embedded
> jinja2 variables that could be rendered, if needed.

That would be even worse. While rendering templates is fast, parsing
them is way more expensive than just a str.replace. This is why you
usually only have a hand-full of templates that are parsed only once,
but rendered repeatedly.