Template variable resolution on objects that are no mappings but implement __getitem__

[funky...@riseup.net]

ciao tutti,

i've already opened a ticket [1] that elaborates why i consider the
behaviour of the template engines variable lookup as buggy, i'll copy
the description below.

i also opened a pull request [2] that demonstrates my progress on a
solution so far.

thanks for any feedback.

[1]: https://code.djangoproject.com/ticket/28782
[2]: https://github.com/django/django/pull/9341

bug description w/o markup:

there are classes in the Pythoniverse that implement a __getitem__
method for the sake of syntactical sweetness, but they are no mappings.
(in my case it's a wrapper around an xml document where __getitem__
returns the result of xpath evaluations. there are other use-cases
around, but i don't remember any atm.).
this causes a lookup in a template rendering context on an attribute of
such instances to return an unexpected value, because foo[bar] doesn't
necessarily raise an exception and returns some value, while foo.bar was
intended. this is caused by the very implicit implementation of
django.template.base.Variable._resolve_lookup.
my approach to solve this issue is to refactor that method to an
explicit behaviour mainly based on type checks, so a member lookup is
only performed on instances that base on collections.abc.Mapping which
any implementation of a mapping should. my rationale is that the
documentations mentions a dictionary (though ​mapping would be more
precise term) lookup, not a lookup via __getitem__, and
dictionary/mapping objects should base on the mentioned abstract class
(the why is elaborated in ​PEP 3119).
beside solving my problem, i would argue that in doubt "explicit is
better than implicit" excels "it's better to ask forgiveness than to ask
permission" and the explicit approach is more comprehensible.

[Adam Johnson]

there are classes in the Pythoniverse that implement a __getitem__ method for the sake of syntactical sweetness, but they are no mappings.

My understanding of what ducktyping means is that by implementing __getitem__ such objects are declaring they are mappings.

I'm -1 on this change because the scope is very large (potentially breaking millions of lines of templates in the real world) and it's also a fairly subtle difference, which makes it hard to explain to end users.

Some of your other suggestions for speeding up template rendering would be worth looking into though, any speed gains would be welcome.

As to solving your initial problem, Tim's suggestion on the ticket of wrapping such objects with a class that implements the correct lookup behaviour in templates, is a sensible approach.

--
You received this message because you are subscribed to the Google Groups "Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscribe@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/4c3db0c806233f390acdeb91af836d98%40riseup.net.
For more options, visit https://groups.google.com/d/optout.

--

Adam

[funky...@riseup.net]

ciao Adam,

>> there are classes in the Pythoniverse that implement a __getitem__
>> method for the sake of syntactical sweetness, but they are no
>> mappings.
>
> My understanding of what ducktyping means is that by implementing
> __getitem__ such objects are declaring they are mappings.

nope, that would mean that sequences are to be considered mappings as
well. please see the (more or less enlightning) rationale by Guido in
the mentioned PEP. it describes a paradigm that is essential in the
dawning Age of Typing in Python. which is a good one in terms of code
quality.

> I'm -1 on this change because the scope is very large (potentially
> breaking millions of lines of templates in the real world) and it's
> also a fairly subtle difference, which makes it hard to explain to end
> users.

as i said, the implementation strictly follows what is described in the
documentation. so it may only be to be explained that the implementation
of an object that they are using is wrong in regards to the mentioned
PEP 3119.
furthermore there's still a fallback with a deprecation warning in
place.

> Some of your other suggestions for speeding up template rendering
> would be worth looking into though, any speed gains would be welcome.

afaik, i have done nothing to increase speed. just readability and
congruence with the state of Python. can you point out what you mean
specifically?

> As to solving your initial problem, Tim's suggestion on the ticket of
> wrapping such objects with a class that implements the correct lookup
> behaviour in templates, is a sensible approach.

yet, my understanding is still that my problem is caused by a bug in
Django.

best regards, Frank

[Adam Johnson]

so it may only be to be explained that the implementation
of an object that they are using is wrong in regards to the mentioned
PEP 3119.

PEP 3119 does say that ABC's are "not the end of ducktyping": https://www.python.org/dev/peps/pep-3119/#abcs-vs-duck-typing . As I understand it, objects/programs can't be "wrong" with respect to the PEP, they might just find using ABC's useful.

 

afaik, i have done nothing to increase speed. just readability and
congruence with the state of Python. can you point out what you mean
specifically?

You wrote two other comments in your initial message on the ticket about suggestions for template rendering, re: SimpleNamespace and ChainMap.

--
You received this message because you are subscribed to the Google Groups "Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscribe@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/a05c31889c61d55295dc3ba5eacf064d%40riseup.net.

For more options, visit https://groups.google.com/d/optout.

--

Adam

[Frank Sachsenheim]

ciao Adam,

though i'd still argue that this behaviour should be changed and it can
be achieved on a reasonable path, i figured i'm better off with Jinja2,
which implements a more concise syntax:
http://jinja.pocoo.org/docs/2.10/templates/#variables

however, the docs of the Django templating system could be more precise
here:
https://docs.djangoproject.com/en/1.11/ref/templates/language/#variables

> You wrote two other comments in your initial message on the ticket
> about suggestions for template rendering, re: SimpleNamespace and
> ChainMap.

regarding the SimpleNamespace: for my gusto it would make view functions
more slick and partly enable useful autocompletion in PyCharm. but in
terms of speed, it would require a type check and one type conversion.

if only SimpleNamespace objects would be allowed as context objects that
might actually be faster, the bytecode of a mapping access has one
instruction more.
on the other hand, one would need a wrapper around it that modeled a
(full; and no pun intended) mapping interface around for a use with the
following:

i gave that ChainMap thing a try, but failed by lacking a deeper
understanding what's going on with some methods on some BaseContext
subclasses. there's hardly inline documentation and some code parts are
still weird to me. i don't have the capacity to dig into this, but it
really looks promising, i'd bet about 50% code reduction if done
properly and more consistence. i can provide a patch with my changes so
far for an idea.

btw, i'm a little confused as there's a fancy trac instance where we're
not having this conversation. ?:-/

best regards, Frank