Minify (compress) response HTML

[Kernc]

Hi,

Looking to this list for guidance...

The web2py issue #369 (http://code.google.com/p/web2py/issues/detail?
id=369) discusses a possible method of minifying response output HTML
(compressing it as in removing all extraneous whitespace). The issue
was closed with WontFix because "minify feature is now built-into
web2py". I looked through the book, 4th Ed., and the only minifcation
I found (http://web2py.com/books/default/search/29?search=minify) was
that of CSS/JavaScript (response.optimize_css).

In a later issue (http://code.google.com/p/web2py/issues/detail?
id=624), again opened by yours truly, Massimo said, "Will fix the
book. This is already in stable." Yet the only minification I see in
the source tree is for CSS/JS (http://code.google.com/p/web2py/source/
browse/#hg/gluon/contrib/minify).

I'd like to know where's my response.render({...}, minify=True) or how
else am I to use supposedly-existent HTML minification feature.

There is popular demand: http://groups.google.com/group/web2py/browse_thread/thread/1cd9856537e9a6db/392f2c8fc6c6ef4b

Thank you!

[Massimo Di Pierro]

I apologize. My answers were not always informative not correct.

I understood the issue was about minification of css and js which is
included.

As far as the minification of html is concerned. I do not think it
belongs to response.render because there is nothing html specific in
there. If you want to propose the inclusion in contrib of a function
that performs minimification of html, I would not oppose to it.
It could be called with:

from gluon.contrib.htmlminify import minify
retrun minify(response.render(...))

Yet I am not sure this will provide any improvement over gzip http
responses. In fact the gzipping would be performed by the web server
and would result in smaller data and faster processing. It is
different for JS because there the minimization can take advantage
variable name rewriting.

What do other people think?

> There is popular demand:http://groups.google.com/group/web2py/browse_thread/thread/1cd9856537...
>
> Thank you!

[Anthony]

Are you just trying to decrease the network payload? If so, maybe just set the server to gzip the content. Once gzipped, minifying will probably only provide modest additional benefit, and may not be worth the extra processing.

Anthony

[Anthony]

On Tuesday, January 31, 2012 3:40:00 PM UTC-5, Massimo Di Pierro wrote:

from gluon.contrib.htmlminify import minify
retrun minify(response.render(...))

Yet I am not sure this will provide any improvement over gzip http
responses. In fact the gzipping would be performed by the web server
and would result in smaller data and faster processing. It is
different for JS because there the minimization can take advantage
variable name rewriting.

What do other people think?

I agree -- I'm not sure minifying HTML will be worthwhile after gzipping. But if we do it, your proposed approach seems like the way to go (though it might be best if we can find an already tested solution rather than try to build our own and possibly get it wrong).

Anthony 

[Kernc]

Sorry for late reply, didn't get any notification by email due to my
list preferences. It could send me a follow-up on the thread I
posted... Oh, well...

Minification before gzipping still does make a difference:
this is my index.html:

$ wc -c minified.html
4827 minified.html
$ wc -c nonminified.html
74910 nonminified.html
$ gzip minified.html && wc -c minified.html.gz
2134 minified.html.gz
$ gzip nonminified.html && wc -c nonminified.html.gz
8247 nonminified.html.gz

Nearly 4-fold difference of minified&gzipped over non-
minified&gzipped. Not to mention minified HTML is faster and easier
for a browser to parse. And the minified-rendered views take up less
RAM when cache.ram'd (which is what one should do on nearly all
pages)....

I would like the feature for its obfuscating nature. It hides the
implementation more, what is a loop, what is static html, how deep an
indentation goes...

It must be worth something, Google does it (view-source:https://
www.google.com/).

The implementation quoted in comment #2 (http://code.google.com/p/
web2py/issues/detail?id=369#c2) solves the issue quite well for me. I
haven't found any new bugs. It even minifies inline JS. It works. :-)
I renounce all rights to this code, which likely "belong" more to the
original inventor of this: http://packages.python.org/web2py_utils/output.html#compress-output
(As far as I am concerned, you can just use it.)

I agree a separate htmlminify module is an acceptable solution. :-)

[Massimo Di Pierro]

OK, let's add a minifier in contrib.

[Michele Comitini]

Massimo,

About gzipping. It is not true that the webserver does the gzipping
at least not every combination of frontend/backend.

1) Rocket does gzipping?
2) Apache + wsgi does gzipping?
3) <server> + fcgi does gzipping?

What I know for sure is <server> + SCGI does gzipping: I wrote it so
that uses wsgitools wsgi gzip application.
I also think UWSGI does the gzip.

The server usually gzips only static files. Anyway I think that
gzipping can be implemented in python with little overhead since it
uses the underlying C implementation.

mic

2012/2/3 Massimo Di Pierro <massimo....@gmail.com>:

[Kernc]

On Feb 3, 6:07 am, Massimo Di Pierro <massimo.dipie...@gmail.com>
wrote:

> OK, let's add a minifier in contrib.

Thanks! :-)

[Mika Sjöman]

So how do I turn minify on? I can not find any info and I really need my pages to be as small as possible because my users like speed.

Cheers

[Kernc]

On Sun, May 19, 2013 at 9:07 AM, Mika Sjöman <mikas...@gmail.com> wrote:

So how do I turn minify on?

Here's one direct example: http://code.google.com/p/web2py/issues/detail?id=668

If you need to minify many controller responses, you can use the following decorator:

from gluon.contrib.minify import htmlmin
def minify(func):
    def _f():
        out = func()
        return htmlmin.minify(response.render(out) if isinstance(out, dict) else out)
    return _f

used as:

@minify
def some_controller():
    ...
    return dict(...)

[Elcimar]

Hi there. 

Here I had to edit the module to exclude <script... </script> tags from minification, 

otherwise the javascript code I have in the bottom of layout.html doesn't work (!?).

[Kernc]

Can you elaborate what JavaScript code is that?

I had problems with e.g. recaptcha module, which starts the script with:

<script><!--
here be the javascript
--></script>

and when minimized, this became:

<script><!-- var whatever, errors on this first line because of the leading '<!--' on the same line as script source --></script>

which didn't work, so I preserved <!-- and that fixed it.
Other than that, even JavaScript should be safe to minimize, with exception that the current simple code breaks whitespace in strings, which is then, perhaps contrary to programmer's expectation, not preserved.

So what JavaScript is breaking for you?

[Elcimar]

Hmm... Go to listacaiu.com and view source code of the main page..

[Kernc]

On Thu, Jul 25, 2013 at 1:15 AM, Elcimar <elc...@gmail.com> wrote:

Hmm... Go to listacaiu.com and view source code of the main page..

Oh, yes, of course! Single-line comments need to be skipped too.
Will provide a patch ASAP.
Thanks. :-)

[Francisco Costa]

would be nice to have this https://pypi.python.org/pypi/htmlmin/0.1.5 incorporated in web2py