What do you think about unify templates feature?

113 views
Skip to first unread message

J. Pablo Martín Cobos

unread,
Jan 17, 2019, 4:02:06 AM1/17/19
to django-d...@googlegroups.com, django...@googlegroups.com
Hi,

From one year ago, I am using an own command for Django templates that unify them. With an example it is easy to see. If I am to render for example a template call news.html like it:

1. news.html

    {% extends "base.html" %}

    {% block title %}
        {% include "inc.news.title.html" %}
    {% endblock %}

    {% block content %}
        {% for news_item in news %}
            <h2>{{ news_item.title }}</h2>
            <p>{{ news_item.subtitle }}</p>
        {% endfor %}
    {% endblock %}

2. base.html

    <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
    <html xmlns="http://www.w3.org/1999/xhtml" lang="{{ LANGUAGE_CODE|default:"en-us" }}" {% if LANGUAGE_BIDI %}dir="rtl"{% endif %}>
        <head>
            <title>{% block title %}{% endblock %}</title>
        </head>
        <body>
            {% block content %}{% endblock %}
        </body>
    </html>

3. inc.news.title.html
    News

With this command I preproces every template of a settings variable and I get something like this:

news.unify.html

    <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
    <html xmlns="http://www.w3.org/1999/xhtml" lang="{{ LANGUAGE_CODE|default:"en-us" }}" {% if LANGUAGE_BIDI %}dir="rtl"{% endif %}>
        <head>
            <title>News</title>
        </head>
        <body>
            {% for news_item in news %}
                <h2>{{ news_item.title }}</h2>
                <p>{{ news_item.subtitle }}</p>
            {% endfor %}
        </body>
    </html>

So I have a two improves:
  1. It is more fast. And in a real project a view can render easyly 50 templates
  2. I use news.html to develop and news.unify.html to production. So I don't lose legilibility.

What do you think about "unify templates feature"? Do you know if exists a similar public project in github/gitlab/bitbucket etc?


Best,

--
Pablo Martín Cobos
Computer engineer
Python/Django developer

Maciek Olko

unread,
Jan 17, 2019, 12:42:11 PM1/17/19
to django-d...@googlegroups.com, django...@googlegroups.com
Did you try to measure the difference in time of rendering standard and your way?

Regards,
Maciej

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CALNyWLGNcuK8DTnU9w9fyGFhFfT3dAz7vfj3B%2BnDHWTfneLNFw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

shiva kumar

unread,
Jan 17, 2019, 11:03:31 PM1/17/19
to django-d...@googlegroups.com
Could you please explain about it. I hadn't understood ur idea.

--

J. Pablo Martín Cobos

unread,
Jan 18, 2019, 4:31:39 AM1/18/19
to django-d...@googlegroups.com, django...@googlegroups.com
I answer between lines

El jue., 17 ene. 2019 a las 18:42, Maciek Olko (<macie...@gmail.com>) escribió:
Did you try to measure the difference in time of rendering standard and your way?


Yes. I have several examples:

AppTemplateNum templates rendered before unifyNum templates rendered after unifyTime render template before unify (ms)Time render template after unify (ms)Improve
Django adminadmin/index.html3175 - 11071 - 7922%
Django adminadmin/change_list.html26Command error180-300Command error
Django adminadmin/change_form.html16Command error350-400Command error
Django constanceadmin/constance/change_list.html71350-400150-25075%
Django susu/login.html5315- 2010 - 1254%

My command currently does not work with several complex templates like admin/change_list.html or admin/change_form.html But we can improve it, currently it works with almost all.

If there is a nice improve with views that render few templates... imagine with views with a lot of templates.

I have measure these times with django-debug-toolbar-template-timings

With this email only I want to know if you see it can be a important feature to add to Django framework... or it is a silly thing.

Best,

 

For more options, visit https://groups.google.com/d/optout.

J. Pablo Martín Cobos

unread,
Jan 18, 2019, 4:40:29 AM1/18/19
to django-d...@googlegroups.com, django...@googlegroups.com
I answer between lines


El vie., 18 ene. 2019 a las 5:03, shiva kumar (<kannamshi...@gmail.com>) escribió:
Could you please explain about it. I hadn't understood ur idea.

  1. Currently: with you render a view, this render a template. But usually, this template extends of another and include others. So Django has to get these templates to create an HTML. This process get time.
  2. My proposal: Preprocess templates for some (or all) them. Creating a template without extends or includes. So, in my previous email you can see, Django render less templates, so Django win time.
We can get a backend a  little faster with very little effort.

Best!

 

For more options, visit https://groups.google.com/d/optout.


--
Juan Pablo Martín Cobos
Ingeniero informático
Desarrollador Python/Django

Josh Smeaton

unread,
Jan 18, 2019, 7:01:37 AM1/18/19
to Django developers (Contributions to Django itself)
Are you using https://docs.djangoproject.com/en/2.1/ref/templates/api/#django.template.loaders.cached.Loader in your tests? And you should find a way to test rendering from the command line, which will allow you to get much better timing information.

Pavlos Georgiadis

unread,
Jan 18, 2019, 7:44:33 AM1/18/19
to django-d...@googlegroups.com

Hi,

I guess the reason you wrote to this list about your unify command, is because you don't mind contributing it to the django project.

So maybe you could show the code of this command, so people can understand what improvements it can offer.

I am not sure I understood anything so far. Are you just pre-rendering {% include %} tags, so the template engine doesn't have to do that in runtime?


--

Pavlos Georgiadis


For more options, visit https://groups.google.com/d/optout.
-- 
--
Pavlos Georgiadis

Jani Tiainen

unread,
Jan 18, 2019, 8:11:18 AM1/18/19
to 'Anoosha Masood Keen' via Django developers (Contributions to Django itself)
Hi.

Does this require changes to django internals or can it work as an external package?

Also you should release it as a package so people can start using it, also preferably releasing it with BSD license like Django itself is. Otherwise there are very little chances that your solution would make to django it self.

Improvements to rendering are always welcome, but they must come without too much maintenance burden.

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CALNyWLGNcuK8DTnU9w9fyGFhFfT3dAz7vfj3B%2BnDHWTfneLNFw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


--
Jani Tiainen

- Well planned is half done, and a half done has been sufficient before...

J. Pablo Martín Cobos

unread,
Jan 18, 2019, 10:33:04 AM1/18/19
to django-d...@googlegroups.com, django...@googlegroups.com
Hi another time,

I am tring to reply every people in this email:

Josh

I was not using django.template.loaders.cached.Loader, but the result are very similar i.e:

AppTemplateNum templates rendered before unifyNum templates rendered after unify
AVG Time render template before unify (ms)AVG Time render template after unify (ms)
Django adminadmin/index.html31
8070
Django constanceadmin/constance/change_list.html71
330180
Django susu/login.html53
105

Pavlos:
  1. Yes, I wrote this email to contribute another time. Actually I am already Django contributor[1][2]
  2. I didn't share my code, because it is not a nice code. For me it is a teoric question. I don't want anybody have a bad opinion of this proposal for the implementation. But I am sharing [3] it without problems, but please I know this solution is incomplete for many reason.
  3. About your question: "I am not sure I understood anything so far. Are you just pre-rendering {% include %} tags, so the template engine doesn't have to do that in runtime?" Yes :-)
  4. About it: "I think what you did is addressed more neatly with the django.template.loaders.cached.Loader, as Josh Smeaton mentioned." and "Experimenting like this, even if you end up rediscovering something that exists" It is something different than django cache loader, of course I know django cache loader, I fixed a problem with django cached loader [4] five years ago :-). With django cache loader, we get django get a template quickly. But If your view finally render 3 templates, django cached loader will have to get 3 templates (from cache). With my proposal your view render only 1 template (This is not 100% true, 1 + templates about templatetags except extends and include) 
Jani:

Currently you only need move the new template result to the old path of the template. But it is only because I want. If you change this line [5] for this other you don't need any change:

destination_name = template.origin.name


8 years ago I requested a feature related with templates, now I have a new request :-) 8 years ago I tried resolved my proposal, now I only want propose something :-D

I think in this change for Django :-)


REF's



Best,



For more options, visit https://groups.google.com/d/optout.


--
Juan Pablo Martín Cobos
Ingeniero informático
Desarrollador Python/Django

J. Pablo Martín Cobos

unread,
Jan 18, 2019, 12:23:50 PM1/18/19
to django-d...@googlegroups.com, django...@googlegroups.com
Sorry, I reply beetween lines

[6]

Jani Tiainen

unread,
Jan 18, 2019, 3:25:55 PM1/18/19
to 'Anoosha Masood Keen' via Django developers (Contributions to Django itself), django...@googlegroups.com
Hi,

Lets try this again.

Your system could be nice if it really helps rendering speed.

But does it require changes in Django core itself, or is it completely standalone package that doesn't require changing Django itself to operate? If it requires changes in Django itself, it would be useful to describe what changes are needed and why.

Note that you can even no build custom rendering engines since Django supports those now (with default implementations for Django Templating Language and Jinja2) your system actually might fall in this category.

Even code is bad you really need to start get some momentum for your system. Otherwise it's really hard to provide any feedback for example there might be some edgecases you might not have thought of.


To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CALNyWLGJ5ZqHT%2B11YC6sHmRskR7s2u%3DAYmhpKfhd5_eautM-SA%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

J. Pablo Martín Cobos

unread,
Jan 18, 2019, 5:18:09 PM1/18/19
to django-d...@googlegroups.com, django...@googlegroups.com
I reply beetween lines,

El vie., 18 ene. 2019 a las 21:25, Jani Tiainen (<red...@gmail.com>) escribió:
Hi,

Lets try this again.

Your system could be nice if it really helps rendering speed.

Yes we get several miliseconds per request.
 

But does it require changes in Django core itself, or is it completely standalone package that doesn't require changing Django itself to operate? If it requires changes in Django itself, it would be useful to describe what changes are needed and why.


You don't need any change in Django. This command generates a new template. I paste another time my first example: 

1. news.html

    {% extends "base.html" %}

    {% block title %}
        {% include "inc.news.title.html" %}
    {% endblock %}

    {% block content %}
        {% for news_item in news %}
            <h2>{{ news_item.title }}</h2>
            <p>{{ news_item.subtitle }}</p>
        {% endfor %}
    {% endblock %}

2. base.html

    <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
    <html xmlns="http://www.w3.org/1999/xhtml" lang="{{ LANGUAGE_CODE|default:"en-us" }}" {% if LANGUAGE_BIDI %}dir="rtl"{% endif %}>
        <head>
            <title>{% block title %}{% endblock %}</title>
        </head>
        <body>
            {% block content %}{% endblock %}
        </body>
    </html>

3. inc.news.title.html
    News

With this command I preprocess every template of a settings variable and I get something like this:

news.unify.html (or if you want the command can overwrite news.html)

    <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
    <html xmlns="http://www.w3.org/1999/xhtml" lang="{{ LANGUAGE_CODE|default:"en-us" }}" {% if LANGUAGE_BIDI %}dir="rtl"{% endif %}>
        <head>
            <title>News</title>
        </head>
        <body>
            {% for news_item in news %}
                <h2>{{ news_item.title }}</h2>
                <p>{{ news_item.subtitle }}</p>
            {% endfor %}
        </body>
    </html>

It is only a preprocess to get time for each request.

Note that you can even no build custom rendering engines since Django supports those now (with default implementations for Django Templating Language and Jinja2) your system actually might fall in this category.


No, It is not a new render engine. It would be a pre render engine. It is a new feature.
 
Even code is bad you really need to start get some momentum for your system. Otherwise it's really hard to provide any feedback for example there might be some edgecases you might not have thought of.

I want to know if this is a nice feature for Django community, then I will create a ticket... and if nobody resolve it I will propose a change, but I know ifor experience it is very hard collaborate with code in Django, and more difficult with a new feature.

To understand it, the best is to use the command with a simple template such as admin/index.html.

Best,


For more options, visit https://groups.google.com/d/optout.

Jani Tiainen

unread,
Jan 19, 2019, 1:08:13 AM1/19/19
to 'Anoosha Masood Keen' via Django developers (Contributions to Django itself), django...@googlegroups.com
Hi,

You said that this doesn't require any change in Django at all.

So this doesn't need to be in Django at all, it can survive as its own and that way it should be.

So make your enhancement as reusable app and release it to public. Get people to use it. Fix the bugs that appears. Write a good documentation. Give the support.



To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CALNyWLGRhOCZ-1LU_5fRWJV_oB7DGBQAvSRU68cCp4r1mMdH5Q%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

J. Pablo Martín Cobos

unread,
Jan 19, 2019, 3:22:43 AM1/19/19
to django-d...@googlegroups.com



   

El sáb., 19 ene. 2019 7:08, Jani Tiainen <red...@gmail.com> escribió:
Hi,

You said that this doesn't require any change in Django at all.

So this doesn't need to be in Django at all, it can survive as its own and that way it should be.

So make your enhancement as reusable app and release it to public. Get people to use it. Fix the bugs that appears. Write a good documentation. Give the support.


Yes, it is an option, or another option is convince for this feature is in Django.

This doesn't need to be in Django, but I think, it is convenient, like any improve.

Thanks!



Jani Tiainen

unread,
Jan 19, 2019, 5:00:55 AM1/19/19
to django-d...@googlegroups.com
Hi.

Unfortunately Django is already big. Every new feature requires careful consideration since it adds maintenance burden to already loaded maintainers.

So best and pretty much only way to make your feature appear in Django itself is to prove that your solution is widely adopted in everyday use and it's "winning" technology.

That would practically mean that code is well written, constantly maintained and documented.

You can keep showing speed test results but they really won't get you anywhere. Let the people decide is your feature good or not. Get more contributors to your feature to resolve edge cases that doesn't work yet.


Adam Johnson

unread,
Jan 19, 2019, 6:31:20 AM1/19/19
to django-d...@googlegroups.com
I agree, I'd like to see a third party package too.

I'm all up for seeing improvements in template rendering speed but working code in a third party package is way more valuable than more discussion. Many of Django's major features in the past (e.g. SecurityMiddleware, Migrations) were third party packages for a long time before being merged in due to community consensus that they were a good idea.

Nice to see that it is an improvement over the cached loader, but it does seem like it would complicate things for users quite a bit as it stands - having to run another command on deployments, maybe modifying view code to load a different template name in production, and creating more files that increase the size of a deployment or use more memory.

Also I think there might be a number of edge cases that would make "unification" very hard - IIRC you can have an {% if %} around a {% block %} or at least around {{ block.super }}.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAHn91oc7570VgsY%3DFcH%2BGGiqsi_AUDbZ9n%3DA0cf0PrWfh9-G%2Bg%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.


--
Adam

J. Pablo Martín Cobos

unread,
Jan 19, 2019, 1:22:16 PM1/19/19
to django-d...@googlegroups.com
I reply between lines,

El sáb., 19 ene. 2019 a las 11:00, Jani Tiainen (<red...@gmail.com>) escribió:
Hi.

Unfortunately Django is already big. Every new feature requires careful consideration since it adds maintenance burden to already loaded maintainers.

Yes, I know it.
 

So best and pretty much only way to make your feature appear in Django itself is to prove that your solution is widely adopted in everyday use and it's "winning" technology.

That would practically mean that code is well written, constantly maintained and documented.

You can keep showing speed test results but they really won't get you anywhere. Let the people decide is your feature good or not. Get more contributors to your feature to resolve edge cases that doesn't work yet.

My experience: I proposed a new feature (much more complicated to be accepted) 8 years ago [1] (when Django was already a big project). I did a reusable Django app with this feature and I did 3 pull requests (8 years ago, 7 years ago and 5 years ago), but these never were accepted. The feature was implemented 4 years ago.  Currently, I think if I only had create that ticket had been the same.

I don't need nobody use my code, I don't need maintain code... I only need a good idea, and somebody (I or another person) implements it. You have your opinion to contribute, and I had the same opinion, but my experience is other with Django. You can see in my github profile [2] I have colaborate with other projects without problems.

Anyway if I have any time to do a code that works in every cases and I have any time to do a reusable app, and I will do it. In short time, I will create a ticket in Django.

Thanks for your opinion :-)

Best,


REF's




For more options, visit https://groups.google.com/d/optout.

J. Pablo Martín Cobos

unread,
Jan 19, 2019, 1:35:00 PM1/19/19
to django-d...@googlegroups.com
I reply between lines,

El sáb., 19 ene. 2019 a las 12:31, Adam Johnson (<m...@adamj.eu>) escribió:
I agree, I'd like to see a third party package too.


If I have any time I will do it. And I will say something in this thread.
 

I'm all up for seeing improvements in template rendering speed but working code in a third party package is way more valuable than more discussion. Many of Django's major features in the past (e.g. SecurityMiddleware, Migrations) were third party packages for a long time before being merged in due to community consensus that they were a good idea.

Yes, With this email I see it is a nice feature. If I have any time I will work in this feature. I don't need more discussion :-)
 

Nice to see that it is an improvement over the cached loader, but it does seem like it would complicate things for users quite a bit as it stands - having to run another command on deployments, maybe modifying view code to load a different template name in production, and creating more files that increase the size of a deployment or use more memory.

It will be like collecstatic command. Also, it would be a optional feature. You want to win several miliseconds per request, run this command. If you don't want win these miliseconds... you don't have do anything. 
 

Also I think there might be a number of edge cases that would make "unification" very hard - IIRC you can have an {% if %} around a {% block %} or at least around {{ block.super }}.


It is not a problem. I complicate my previous example, it now works:

1. news.html

    {% extends "base.html" %}

    {% block title %}
        {% include "inc.news.title.html" %}
    {% endblock %}

    {% block content %}
        {% if news %}
            {% for news_item in news %}
                <h2>{{ news_item.title }}</h2>
                <p>{{ news_item.subtitle }}</p>
            {% endfor %}
        {% else %}
            {{ block.super }}
        {% endif %}
    {% endblock %}

2. base.html

    <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
    <html xmlns="http://www.w3.org/1999/xhtml" lang="{{ LANGUAGE_CODE|default:"en-us" }}" {% if LANGUAGE_BIDI %}dir="rtl"{% endif %}>
        <head>
            <title>{% block title %}{% endblock %}</title>
        </head>
        <body>
            {% block content %}
                Sorry no results
            {% endblock %}
        </body>
    </html>

3. inc.news.title.html
    News

With this command I preproces every template of a settings variable and I get something like this:

news.unify.html

    <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
    <html xmlns="http://www.w3.org/1999/xhtml" lang="{{ LANGUAGE_CODE|default:"en-us" }}" {% if LANGUAGE_BIDI %}dir="rtl"{% endif %}>
        <head>
            <title>News</title>
        </head>
        <body>
            {% if news %}
                {% for news_item in news %}
                    <h2>{{ news_item.title }}</h2>
                    <p>{{ news_item.subtitle }}</p>
                {% endfor %}
            {% else %}
                Sorry no results
            {% endif %}
        </body>
    </html>

Best,

 

For more options, visit https://groups.google.com/d/optout.

J. Pablo Martín Cobos

unread,
Jan 19, 2019, 1:41:56 PM1/19/19
to django-d...@googlegroups.com
Bonus track: If you have a template with complete HTML (without blocks). You can add to this command another features like compress HTML or use line styles (using pyliner[1]) and win more time yet :-) Because this feature does not consume run time, but this feature waste preprocessing time, like compilemessage command.

Best,
Reply all
Reply to author
Forward
0 new messages