[GSOC] Template Engine Compilation and Runtime Refactoring Progress

[Armin Ronacher]

Hi,

This is the first weekly status update of many to come. Just a quick
overview of this project and myself.

My nickname on freenode is 'mitsuhiko' and this is the preferred form
to reach me in general. Alternatively my twitter account with the same
name is a good way of reaching me even when I'm not at my computer, so
is mail.

As the template system in Django is already a pretty isolated system
by itself it shouldn't be too hard to keep the work nicely merge-able.
I have not yet decided if I will keep this branch on github or
somewhere else. The Jinja2/Django maintenance aspect of this will be
decided at a late point, probably even just after GSOC so it can be
ignored for the duration of GSOC.

Regards,
Armin

[Armin Ronacher]

Hi,

The repository with my changes will be here:
https://github.com/mitsuhiko/django-template-compilation

As this proposal implements a 'newtemplates' module in Django instead
of modifying the existing one until it's proven to be compatible in
every single regard, I will have it as a separate repository. The idea
is that it can then easily be tested with other changes in Django
itself.

Regards,
Armin

[Armin Ronacher]

Hi,

The progress went slower than expected because I had to analyze some
more runtime differences between Django and Jinja2 which I missed at
first. The main problem are changes in the safe-string behavior but I
think I found a possible solution to counter that. I had to diverge
from my previous plan to keep the nodes unchanged from Jinja2 so at
that point it's pretty sure that Jinja2 will not survive this GSOC
without breaking backwards compat on it's extension interface either
which now simplify further plannings.

First pieces of currently meaningless code are checked in, so is an
OBSERVATIONS file which has some notes on internals. I will update
this file with more internal notes as the project progresses.

Regards,
Armin

[Armin Ronacher]

Hi,

Weekly progress report: I started on documenting some of the behavior
that the current django template engine has and how it affects the
compilation. The nodes are mostly implemented and some of them can be
interpreted already. I want to finish the interpreter first because
that way we can easily figure out if it's compatible with the existing
interpreter and if it is, start on compiling it down to AST nodes and
from that into actual Python bytecode.

Regards,
Armin

[Armin Ronacher]

Hi,

I'm in between two conferences right now and university and a bunch of
other stuff needs attention as well so progress is slowed down quite a
bit right now. Will however catch up next week with what I cannot do
this one.


Regards,
Armin

[Anatoly Vostryakov]

Hi, Armin!

I've thought today. May be it will possible to add support of
parameters for functions or object methods in django templates.
I mean, something like this in django template:

{{ obj.method(parameter1, parameter2) }}

How I remember it is possible in jinja2 someway and it is what I
missing in django templates. I wrote it to you because you're working
with django templates now and author of jinja2.

Regards,
Anatoly Vostryakov

On Jun 17, 4:07 am, Armin Ronacher <armin.ronac...@active-4.com>
wrote:

[Alex Gaynor]

--
You received this message because you are subscribed to the Google Groups "Django developers" group.
To post to this group, send email to django-d...@googlegroups.com.
To unsubscribe from this group, send email to django-develop...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.

No, the purpose of this GSOC is not to change the semantics of the template engine.

Alex

--
"I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
"The people's good is the highest law." -- Cicero

[Armin Ronacher]

Hi,

On Jun 17, 2:18 pm, Anatoly Vostryakov <a.vostrja...@gmail.com> wrote:
>   I've thought today. May be it will possible to add support of
> parameters for functions or object methods in django templates.
> I mean, something like this in django template:

The compiler is able to handle this case, but the syntax of the Django
templating language does not. I will not change any features in the
Django template engine during this proposal and actually reused most
of the parser toolchain until fall. If someone wants to step up and
continue improving the features on the language after GSOC I am happy
to assist.


Regards,
Armin

[Anatoly Vostryakov]

Hi!

> The compiler is able to handle this case, but the syntax of the Django
> templating language does not.  I will not change any features in the
> Django template engine during this proposal and actually reused most
> of the parser toolchain until fall.  If someone wants to step up and
> continue improving the features on the language after GSOC I am happy
> to assist.

Ok. Thanks for answer. I'll think to add support of parameters to
functions in django template engine after GSOC. From my point of view
it is
backward compatible feature. All old django templates will continue to
work.

Regards,
Anatoly

On Jun 18, 7:52 pm, Armin Ronacher <armin.ronac...@active-4.com>
wrote:

[Aymeric Augustin]

On 19 juin 2011, at 12:39, Anatoly Vostryakov wrote:
> Ok. Thanks for answer. I'll think to add support of parameters to
> functions in django template engine after GSOC. From my point of view
> it is
> backward compatible feature. All old django templates will continue to
> work.

Hi Anatoly,

Before working on a patch, you'll probably want to check the mailing list archives and bug tracker, as this feature has been discussed and rejected in the past.

The last paragraph of https://docs.djangoproject.com/en/dev/topics/templates/#accessing-method-calls explains why.

Best regards,

--
Aymeric Augustin.

[Armin Ronacher]

Hi,

Progress update. This week I was starting the work on the compiler
and I have at least for Django's use case most compilation things
ironed out. However it turns out that compiling AST nodes made from
scratch is a painful process as the interpreter will randomly segfault
or fail compiling with useless error messages. For debugging the AST
I wrote a bunch of small helpers but the current HEAD revision
exhibits some weird behavior I have not found the cause for yet.
Generally however good progress for a week full of EuroPython :)

Regards,
Armin

[Armin Ronacher]

Hi,

Status of this week: I spend the last week mostly trying to figure out how to debug a bunch of annoying issues and due to the slow progress I decided on doing an AST -> Python code translation for debugging purposes. The identifier tracking method I naively started using initially (which is assuming that there are no branches and recording identifiers for each scope while the transformer is creating code) turns out to be by far not good enough as it misses identifiers that are referenced after blocks. I will need to heavily improve on that.

Regards,
Armin

[Armin Ronacher]

Hi,

Current status: The freezing of free variables into a context
dictionary for includes and extension is currently not working as
expected. The reason is obvious but the solution is not entirely
clear to me yet. I need to track what variables were assigned to
until a specific statement, not only what variables will be assigned
or referenced for a block. I don't like to do that while transforming
the AST, but walking that another time sounds like a horrible idea was
well. A bit torn, I might just use the current identifier tracking
code and also record the id of the node that triggers the assignment.
Then I could reconstruct the state before each statement I think.


Regards,
Armin

[Kevin Tran]

Has the repository been deleted? https://github.com/mitsuhiko/django-template-compilation is giving me a 404.

[Dougal Matthews]

On Thursday, 28 July 2011 at 09:43, Kevin Tran wrote:
> Has the repository been deleted? https://github.com/mitsuhiko/django-template-compilation is giving me a 404.

Try here: https://github.com/mitsuhiko/templatetk