Feature: Template Components

[Emil Stenström]

Hi,

This is the first feature proposal as part of my general drive for
getting Django to work better for javascript heavy sites.

Template Components
-------------------

React.js popularized the notion that in front-end development, code
organization should be based on interface components, not split up into
HTML, Javascript and CSS. Here's the original presentation and the
rationale behind organizing around components:
https://www.youtube.com/watch?v=x7cQ3mrcKaY&t=2m7s

In Django, adding a Javascript calendar to you site requires changes to
four different locations in your project:

- /app/templatetags/calendar_tags.py <- A custom inclusion template tag
- /app/templates/calendar.html <- Some HTML in the template dir
- /static/css/style.css <- Add some CSS to style.css
- /static/js/script.js <- Add some JS to scipt.js

There is no connection within Django between HTML and the CSS + JS.
Django does not help you, or even suggest a structure for you, like it
does with all Python code.

On the other hand, we have Form Assets:
https://docs.djangoproject.com/en/1.8/topics/forms/media/

Example:

from django import forms
class CalendarWidget(forms.TextInput):
class Media:
css = {'all': ('calendar.css',)}
js = ('calendar.js',)

>>> w = CalendarWidget()
>>> print(w.media)
<link href="calendar.css" media="all" rel="stylesheet" />
<script src="script.js"></script>

... which define a kind of component, but one that is tied to a form
field. My suggestion is a new package: django.template.component, that
builds on the Media class from Form Assets, and allows you to define
your components like this:

from django.template import component
class Calendar(component.Component):
def context(self, date):
return {"date": date}
class Media:
template = "app/components/calendar/calendar.html"
css = {'all': ('app/components/calendar/calendar.css',)}
js = ('app/components/calendar/calendar.js',)

component.register(name="calendar", component=Calendar)

... and later in your template:

{% load components %}
{% block extra_media %}{% component_dependencies %}{% endblock %}
{% block main %}
{% component "calendar" date=custom_date %}
{% endblock %}

---

Advantages:
- You to keep the python, html, css, and javascript in one location, and
explicitly define the dependencies between them.
- All <link> and <script> tags would be handled by the
component_dependencies template tag (the Media class behind the scences).
- No need for inclusion template tags
- Better customization of the admin, where you could simply register
your own components to replace existing ones
- Reuse of components from the admin outside of it. Just use the name
from the admin and send the correct parameters. If you want to change
the CSS, just import the class and override the properties you want to
change.

---

I think this would be a great addition to Django, and some of the work
is already done in the Media class that form assets is using. This would
of course be optional, but I think lots of people would see the benefits
of using this structure instead of their existing one.

Would anyone we willing to work with me on this? Do you think it makes
sense to put this in Django? I don't see any need for this code to
change often, and the API is fairly similar to an existing one.

Thoughts? Ideas?

Regards,

Emil Stenström
Twitter: @EmilStenstrom

[Riccardo Magliocchetti]

Hi,

Il 30/05/2015 18:52, Emil Stenström ha scritto:
> Hi,
>
> This is the first feature proposal as part of my general drive for getting
> Django to work better for javascript heavy sites.

This is a bold premise :)

> Template Components
> -------------------
>
> React.js popularized the notion that in front-end development, code organization
> should be based on interface components, not split up into HTML, Javascript and
> CSS. Here's the original presentation and the rationale behind organizing around
> components:
> https://www.youtube.com/watch?v=x7cQ3mrcKaY&t=2m7s

This is a nice react.js introduction:
https://facebook.github.io/react/docs/thinking-in-react.html

[snip]

> ... which define a kind of component, but one that is tied to a form field. My
> suggestion is a new package: django.template.component, that builds on the Media
> class from Form Assets, and allows you to define your components like this:
>
> from django.template import component
> class Calendar(component.Component):
> def context(self, date):
> return {"date": date}
> class Media:
> template = "app/components/calendar/calendar.html"
> css = {'all': ('app/components/calendar/calendar.css',)}
> js = ('app/components/calendar/calendar.js',)
>
> component.register(name="calendar", component=Calendar)
>
> ... and later in your template:
>
> {% load components %}
> {% block extra_media %}{% component_dependencies %}{% endblock %}
> {% block main %}
> {% component "calendar" date=custom_date %}
> {% endblock %}

But your proposal keeps html and js separated. I think you are solving a problem
for the one that just want to use a component but you are not solving the
problem for the one that is writing components. At least not in the react.js
way, especially from such a bold premise :)

thanks

--
Riccardo Magliocchetti
@rmistaken

http://menodizero.it

[Emil Stenström]

Hi,

On Sunday, 31 May 2015 10:27:24 UTC+2, riccardo.magliocchetti wrote:

Hi,

Il 30/05/2015 18:52, Emil Stenström ha scritto:
> Hi,
>
> This is the first feature proposal as part of my general drive for getting
> Django to work better for javascript heavy sites.

This is a bold premise :)

Yes, I know it is bold. If you look at my three proposed features together, and think of a Django with these features included... I would definitely think that building javascript applications would be much easier. That's how I think of this.

[snip]
> ... which define a kind of component, but one that is tied to a form field. My
> suggestion is a new package: django.template.component, that builds on the Media
> class from Form Assets, and allows you to define your components like this:
>
> from django.template import component
> class Calendar(component.Component):
>      def context(self, date):
>          return {"date": date}
>      class Media:
>          template = "app/components/calendar/calendar.html"
>          css = {'all': ('app/components/calendar/calendar.css',)}
>          js = ('app/components/calendar/calendar.js',)
>
> component.register(name="calendar", component=Calendar)
>
> ... and later in your template:
>
> {% load components %}
> {% block extra_media %}{% component_dependencies %}{% endblock %}
> {% block main %}
>      {% component "calendar" date=custom_date %}
> {% endblock %}

But your proposal keeps html and js separated. I think you are solving a problem
for the one that just want to use a component but you are not solving the
problem for the one that is writing components. At least not in the react.js
way, especially from such a bold premise :)

 
I agree that I don't quite do it the same way as React. But that's not the point here either, to somehow bundle Reacts ideas inside Django. My point is that keeping the four parts that make a component closely together in the project source, would make for a better structure. They only idea that comes from React is thinking in components rather than nested templates and template tags, which are Django's current way of solving this.

So this is really a rather small suggestion, but that I think would make the component mindset fit better in with Django. Also, a lot of the code is already written in the Media class.

Thanks for taking the time to comment!

/E

[Riccardo Magliocchetti]

Hi Emil,

Il 31/05/2015 11:00, Emil Stenström ha scritto:
> Hi,
>
> On Sunday, 31 May 2015 10:27:24 UTC+2, riccardo.magliocchetti wrote:
>
> Hi,
>
> Il 30/05/2015 18:52, Emil Stenström ha scritto:
> But your proposal keeps html and js separated. I think you are solving a
> problem
> for the one that just want to use a component but you are not solving the
> problem for the one that is writing components. At least not in the react.js
> way, especially from such a bold premise :)
>
> I agree that I don't quite do it the same way as React. But that's not the point
> here either, to somehow bundle Reacts ideas inside Django. My point is that
> keeping the four parts that make a component closely together in the project
> source, would make for a better structure. They only idea that comes from React
> is thinking in components rather than nested templates and template tags, which
> are Django's current way of solving this.

I see, but who are going to do a big js app without using a framework like
angular / react.js / ember / whatever? I'm not saying your Component proposal is
without merit but i can't see how it can fit where the js app is done with a
framework.

[Emil Stenström]

The idea is simply to keep interface components together in the Django project tree. That wouldn't change your options in regards to what javascript framework to use, just give you some help with organizing your code.

Say you decide to use React as just JS framework. Since React puts the HTML inside your javascript your Django component would simply be:

class ReactCalendar(component.Component):

    def context(self, date):
        return {"date": date}
    class Media:

        template = None

        css = {'all': ('app/components/calendar/calendar.css',)}
        js = ('app/components/calendar/calendar.js',)

Since React doesn't handle CSS the component model would give you a way of tying thing together anyway. And with the component template tag you would be able to decide where to include your React component.

[Joe Tennies]

I actually think this is a great idea. In my mind it parallels Drupal's "block" idea. (This is actually what I keep hoping DjangoCMS is.)

That stated, it is more of a construct. I think a great idea is to make an extension module.

I don't know how long you've been in this community, but Django is now quite stable and people rely on their APIs. Your bold ideas really need to be tested by fire before they go into the Django proper. There's been lots of discussion about a year ago to remove/reduce a lot of the contrib.

Being in Django is where things go to die. Once an API is released to the public, it takes a fair amount of work to remove/change it. You don't even want to know how long it took to get schema migrations into Django. South had been the defacto way for quite a few years. That stated, Andrew had to make some changes to South due to design decisions that were made earlier in South. Before that, there was other projects like django-evolution (which just got an update 2 months ago).

So, I followed the Drupal group for a while. The thing the Django community really needs is a couple good opinionated groups of how to put together a good Django site. Drupal has Lullabot (who have quite a few core devs on staff). Django is not going to be the people telling people how to use Django. You seem like a great person to start this for Django. Note that you'll have to have a thick skin and create some pretty great sites in your own right to prove out your ideas to others. You'll also need to get your ideas out via things like blog posts, tutorials, and podcasts.

I would like the Caktus, DjangoCMS, FeinCMS, etc people to do the same. This would help people to see some different ideas on how to use and extend Django.

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/a05f2dc1-3515-4b23-96f5-479d2722b82c%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--

Joe Tennies
ten...@gmail.com

[Unai Zalakain]

Finally someone expressed my own feelings about it perfectly :D

>> <https://groups.google.com/d/msgid/django-developers/a05f2dc1-3515-4b23-96f5-479d2722b82c%40googlegroups.com?utm_medium=email&utm_source=footer>
>> .

>>
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
>
>--
>Joe Tennies
>ten...@gmail.com
>

>--
>You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
>To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
>To post to this group, send email to django-d...@googlegroups.com.
>Visit this group at http://groups.google.com/group/django-developers.

>To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CACiOJ6uz5f42vSQ_5A%2BU_GyUHSPuNrO6vWahGpzdbSH9cusFiQ%40mail.gmail.com.

>For more options, visit https://groups.google.com/d/optout.

--

unai

[Sam Solomon]

So a former co-worker with some help/guidance from me developed a component system on top of Django that sounds sorta like what you are all talking about. It's very complicated and I'm still not sure whether it was ultimately a good idea to use or not, but it does make some things very simple (and we have no plans to move away from it at this point after ~2 years of use). I'll post some snippets from our internal docs on it now and if people are interested, I can explain more and possibly open source it:

Background: What?

Specialized class based views and sub-views, that facilitate:

• Split page template into components, each of which can be POSTed to and/or updated via ajax.
• View code and templates for full page view and ajax update is the same code. (DRY, though more data over the wire than some of the newer javascript frameworks)
• Updating multiple parts of the page with a single request (example: POSTing to one component can cause dependent components to also be updated in the same request)
• Built-in compatibility to handle no-js in most situations
• Deferred loading of components, for performance

Background: Why?

• Complex site with lots of logged in user interactions where content should be easy to change dynamically
• Slow connections overseas (minimize round trips and limit full page loads without sacrificing dev time too much)
• Some may not even have JS? (Though we don't really cater to them that much anymore, this was a consideration originally and still mostly supported by default outside of where we use d3 and things like that for rendering certain things.)

[Emil Stenström]

On Monday, 1 June 2015 03:32:26 UTC+2, Rotund wrote:

I actually think this is a great idea. In my mind it parallels Drupal's "block" idea. (This is actually what I keep hoping DjangoCMS is.)

That stated, it is more of a construct. I think a great idea is to make an extension module.

Thanks! And I definitely think this should be developed as a separate module.

 

I don't know how long you've been in this community, but Django is now quite stable and people rely on their APIs. Your bold ideas really need to be tested by fire before they go into the Django proper. There's been lots of discussion about a year ago to remove/reduce a lot of the contrib.

Being in Django is where things go to die. Once an API is released to the public, it takes a fair amount of work to remove/change it. You don't even want to know how long it took to get schema migrations into Django. South had been the defacto way for quite a few years. That stated, Andrew had to make some changes to South due to design decisions that were made earlier in South. Before that, there was other projects like django-evolution (which just got an update 2 months ago).

I've been in the community a long while, so I know how things usually work. My hope for this is that there would be a strong need in the community to solve these kinds of problems. This doesn't mean that the process will be any different, or take less time.
 

So, I followed the Drupal group for a while. The thing the Django community really needs is a couple good opinionated groups of how to put together a good Django site. Drupal has Lullabot (who have quite a few core devs on staff). Django is not going to be the people telling people how to use Django. You seem like a great person to start this for Django. Note that you'll have to have a thick skin and create some pretty great sites in your own right to prove out your ideas to others. You'll also need to get your ideas out via things like blog posts, tutorials, and podcasts.

I would like the Caktus, DjangoCMS, FeinCMS, etc people to do the same. This would help people to see some different ideas on how to use and extend Django.

 

This is a neat idea and I would really want to read those blog posts. Maybe that's the way to go.

I don't see my set of ideas to be THAT opinionated. None of them force you to change your existing sites. I see them as tools, that someone that needs heavier javascript features will definitely need.

[Emil Stenström]

On Monday, 1 June 2015 19:12:36 UTC+2, Sam Solomon wrote:

So a former co-worker with some help/guidance from me developed a component system on top of Django that sounds sorta like what you are all talking about. It's very complicated and I'm still not sure whether it was ultimately a good idea to use or not, but it does make some things very simple (and we have no plans to move away from it at this point after ~2 years of use). I'll post some snippets from our internal docs on it now and if people are interested, I can explain more and possibly open source it <snip>

Interesting! I would definitely want to see some code examples here. Could you show me how me more about how it works?

[Sam Solomon]

If someone actually wants to use it I can set aside some time within the next week or two to actually open source the whole thing (docs + source on github/pypi).

Here is an example of some of the basic functionality though:

What the following code does:

Creates a page for taking attendance of something that has 2 components, a listing component that lists existing attendees and a entry component that adds new people to the list.

We have javascript that disables the default form action and on form submit submits via ajax. If there are errors, the form is displayed again with any errors showing (over ajax), if there aren't errors, it resets the form and also updates the listing component over ajax in a format like this (note, though we haven't extended ourselves, it was made so that you could send payloads other than "new_html" and make a custom handler for "framework_example_attendance_entry" that could do something other than simply dump new_html into the given div):

{

    "actions": {

        "framework_example_attendance_entry": {

            "component_key": "framework_example_attendance_entry",

            "new_html": "<form method=\"POST\"

                                action=\"/framework_example/\">

                            <div style=\"display:none\">

                                <input type=\"hidden\"

                                    name=\"csrfmiddlewaretoken\"

                                    value=\"....\">

                            </div>

                            <p><label for=\"id_name\">Name:</label>

                            <input id=\"id_name\" type=\"text\"

                                name=\"name\" maxlength="75" />

                            <input type=\"hidden\" name=\"page_key\"

                                value=\"framework_example_attendance_entry\"

                                id=\"id_page_key\" />

                            <input type=\"hidden\" name=\"param_key\"

                                id=\"id_param_key\" />

                            <input type=\"hidden\" name=\"version\"

                                value=\"1\" id=\"id_version\" /></p>

                            <input type=\"submit\">

                        </form>",

            "version": 1,

            "js_failure_mode": "no_warn_no_update"

        },

        "framework_example_listing": {

            "component_key": "framework_example_listing",

            "new_html": "<ul>

                            <li>George - Aug. 12, 2014, 1:46 p.m.</li>

                            <li>Smith - Aug. 12, 2014, 1:46 p.m.</li>

                            <li>Joseph - Aug. 12, 2014, 1:23 p.m. </li>

                            <li>Steve - Aug. 12, 2014, 1:22 p.m.</li>

                            <li>John - Aug. 12, 2014, 1:22 p.m.</li>

                            <li>Andy - Aug. 12, 2014, 12:09 p.m.</li>

                            <li>Sam - Aug. 12, 2014, 11:13 a.m.</li>

                        </ul>",

            "version": 1,

            "js_failure_mode": "no_warn_no_update"

        }

    }

}

views.py:

class AttendanceMixin(object):

    # We encourage using Mixins that contain @obj_cache decorated methods

    # that will (or in some cases may) be used by multiple components.

    @obj_cache

    def attendance_list(self):

        return list(AttendanceRecord.objects.order_by('-id'))

    @obj_cache

    def num_attendees(self):

        # This is free IFF we already need a list of attendees in the

        # request, otherwise  we should be using

        # `AttendanceRecord.objects.count()`

        # Also note that it is fine to chain @obj_cache decorated methods.

        return len(self.attendance_list)

class AttendanceEntryComponent(AttendanceMixin, Component):

    template_name = "framework_example/entry_component.html"

    def init(self):

        self.ctx.this_form_url = self.this_url()

        if not self.is_post():

            self.ctx.attendance_form = AttendanceRecordForm(

                **self.form_init())

    def handler(self, request):

        self.ctx.attendance_form = AttendanceRecordForm(request.POST,

                                                        **self.form_init())

        if self.ctx.attendance_form.is_valid():

            self.ctx.attendance_form.save()

            self.ctx.attendance_form = AttendanceRecordForm(

                **self.form_init())

            self.add_dependent_component(AttendanceListingComponent)

            return True

    def final(self):

        # The @obj_cache decorator turns methods into properties so note

        # that this isn't `self.num_attendees()`

        self.ctx.num_attendees = self.num_attendees

class AttendanceListingComponent(AttendanceMixin, Component):

    template_name = "framework_example/listing_component.html"

    def init(self):

        self.ctx.attendance_list = self.attendance_list

class AttendancePage(Page):

    template_name = "framework_example/attendance_page.html"

    def set_components(self):

        self.add_component(AttendanceListingComponent)

urls.py:

    component_url(r'^attendance_listing/$',

                  ComponentClass=AttendanceListingComponent,

                  name="framework_example_listing"),

    component_url(r'^$',

                  ComponentClass=AttendanceEntryComponent,

                  name="framework_example_attendance_entry",

                  PageClass=AttendancePage),

templates/framework_example/entry_component.html:

Add a new attendee (Currently {{ num_attendees }} attending)

<form method="POST" action="{{ this_form_url }}">

    {% csrf_token %}

    {{ attendance_form.as_p }}

    <input type="submit">

</form>

templates/framework_example/listing_component.html:

<ul>

    {% for attendance_record in attendance_list %}

        <li>

            {{ attendance_record.name }} - {{ attendance_record.arrival_time }}

        </li>

    {% endfor %}

</ul>

templates/framework_example/attendance_page.html

{% extends "base.html" %}

{% block content %}

    {% comment %}

        The surrounding div id follows a specific formula so it can be

        automatically updated on ajax requests: `cmp_<component_name>_id`

    {% endcomment %}

    <div id="cmp_framework_example_listing_id">

        {% comment %}

            Similarly, you can grab the rendered html from the component by

            asking for `component_name` from the dictionary of rendered html

            `components`:

        {% endcomment %}

        {{ components.framework_example_listing }}

    </div>

    <div id="cmp_framework_example_attendance_entry_id">

        {{ components.framework_example_attendance_entry }}

    </div>

{% endblock content %}

--
You received this message because you are subscribed to a topic in the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/django-developers/FmBM8VdxJ08/unsubscribe.
To unsubscribe from this group and all its topics, send an email to django-develop...@googlegroups.com.

To post to this group, send email to django-d...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/5e562720-9484-4e4f-9507-129d55a4cfce%40googlegroups.com.

[Yoong Kang Lim]

This seems like a huge change. If you were to include this feature in Django, would it be straightforward for users to migrate from previous versions?

[Sam Solomon]

Speaking only for myself and the component framework we built here, I don't think I would expect it to ever be rolled into Django itself.

However, the framework we made does work without any modifications to Django and works just fine alongside existing function and classed based views. This means no changes to existing codebases would be necessary (unless you wanted to buy into the concept completely and switch existing stuff over rather than just use it for new features, in which case it would require a lot of work to split up your views/templates into components (though if you already split up the code in your views in a logical way and use template includes rather than really long template files and stuff like that, it may not be that bad)).

[Emil Stenström]

On Tuesday, 9 June 2015 13:21:27 UTC+2, Yoong Kang Lim wrote:

This seems like a huge change. If you were to include this feature in Django, would it be straightforward for users to migrate from previous versions?

What I'm suggesting is not a huge change. I'm just saying that Django should have something called a "Component" that template authors could decide to use, or completely ignore (that's the backwards compatible bit). My suggestion is based on the Media class from FormAssets, with a few helpers on top of that. It's neither huge or hard, and can be completely built as a third party app (which I have planned).

[Curtis Maloney]

This sounds a bit like combining django-sniplates with django-amn, and going a bit further...

Fragments of templates, list of JS/CSS dependencies, and a way to collect it all together and ensure your page has everything  you need...

Sounds interesting to me... I'd be happy to collaborate on it with you... once we see where it goes, and compare it with the existing implementation, people will have more food for thought :)

--

Curtis

--
You received this message because you are subscribed to the Google Groups "Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.

To post to this group, send email to django-d...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/5569EAC5.1020804%40kth.se.

[Emil Stenström]

On Wednesday, 10 June 2015 02:55:46 UTC+2, Curtis Maloney wrote:

This sounds a bit like combining django-sniplates with django-amn, and going a bit further...

Fragments of templates, list of JS/CSS dependencies, and a way to collect it all together and ensure your page has everything  you need...

Sounds interesting to me... I'd be happy to collaborate on it with you... once we see where it goes, and compare it with the existing implementation, people will have more food for thought :)

Sound like fun! I have a repository setup at https://github.com/EmilStenstrom/django-components and I've given you commit access. I've also started a chat channel connected to that repo so we can talk: https://gitter.im/EmilStenstrom/django-components

Looking forward to this!