Templates: __html__ method support

[Jonas H]

Proposal: Support the __html__ method as an alternative/addition to the __str__ for turning objects into strings in the template layer.

If this has been discussed before, please point me to it; I couldn't find anything with the search function.

Some custom classes may have, in addition to a __str__ representation, a natural representation that is better suited for HTML output. Example:

class Money:

    def __init__(self, amount, currency):

        self.amount = amount

        self.currency = currency

    def __str__(self):

        return '%s %s' % (self.currency, self.amount)

    def __html__(self):

        # Always show amount and currency on same line

        return '%s\xa0%s' % (self.currency, self.amount)

`conditional_escape` and friends already consider the __html__ method, and this works out well:

>>> str(Money(1, '$'))

'$ 1'

>>> conditional_escape(Money(1, '$'))

'$\xa01'

In templates however it doesn't work that way because variables are always turned into strings before stuffing them into `conditional_escape` (see https://github.com/django/django/blob/98706bb35e7de0e445cc336f669919047bf46b75/django/template/base.py#L977). My suggestion is to change the behaviour of that function so that it works as follows:

- Given I write {{ foo }}

- Does foo have a __html__ method? If yes, return `foo.__html__()`

- Otherwise, return `conditional_escape(str(foo))`

Do think that's a good idea?

Jonas

[George-Cristian Bîrzan]

On Tuesday, September 26, 2017 at 3:34:29 PM UTC+3, Jonas H wrote:

Proposal: Support the __html__ method as an alternative/addition to the __str__ for turning objects into strings in the template layer.

Dunder methods' names shouldn't be invented, so a better name is needed.

[Collin Anderson]

I think this is a good idea. Django has used __html__ internally for the last 4 years so it's not something new [0]. I'm surprised this doesn't work out-of-the box.

[0] https://code.djangoproject.com/ticket/7261

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscribe@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/16f056c0-0ed8-424f-ae9e-9ab7a7bf66cb%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Brice Parent]

I like the idea a lot. But wouldn't it be better to have it as a separate library? It seems to be something that could be quite common, and have many use cases both inside and outside Django (we could sometimes benefit from having an html exception message for example).

And I guess that would mean we should also have an html() function which would call .__html__() if it exists, and fallback to .__str__() if not.

Django's template would call this html() function in its templates, while models and other objects could just declare this __html__() method.

And if it gets adopted by more than just Django, it could be integrated into the standard Python library and benefit to many more projects.

Only question: Where should it stop? Should there also be a __json__, a __yaml__ and an __xml__ methods? Those are also quite common representations we could want from a class, even more nowadays that a big tendency is to develop microservices which communicate through APIs, and frontends being more and more delegated to javascript libraries.

Le 26/09/17 à 14:34, Jonas H a écrit :

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.

To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.

Visit this group at https://groups.google.com/group/django-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/ea088de2-e538-4808-a7fd-8726929e2b91%40googlegroups.com.

[Jonas Haag]

Collin Anderson:

I think this is a good idea. Django has used __html__ internally for the last 4 years so it's not something new [0]. I'm surprised this doesn't work out-of-the box.

I too expected this to work already, and I was surprised to find it doesn’t. Hence this thread. (It also surprises me that this hasn’t been discussed on this list before, to be honest.)

Brice Parent:

I like the idea a lot. But wouldn't it be better to have it as a separate library? It seems to be something that could be quite common, and have many use cases both inside and outside Django (we could sometimes benefit from having an html exception message for example).

As you have correctly observed this can’t live in a separate library entirely as it requires integration into the template language to be useful:

[Tom Forbes]

The problem with something more complex like yaml or json is it's not easy to combine the output. If those methods return a string, as in actual json, it's not easy to do anything with them (like combine them into an array or another object). YAML is also whitespace sensitive. If they return a dict or some python object that can be combined and serialised as a whole, then that's kind of confusing and could be more generic.

XML and HTML don't suffer as much from this I think.

To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscribe@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.

Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/ea088de2-e538-4808-a7fd-8726929e2b91%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.

To unsubscribe from this group and stop receiving emails from it, send an email to django-developers+unsubscribe@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.

Visit this group at https://groups.google.com/group/django-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/b3d37420-052d-74a2-5a39-e10d4c180f17%40brice.xyz.

[Brice Parent]

Le 26/09/17 à 22:33, Tom Forbes a écrit :

The problem with something more complex like yaml or json is it's not easy to combine the output. If those methods return a string, as in actual json, it's not easy to do anything with them (like combine them into an array or another object). YAML is also whitespace sensitive. If they return a dict or some python object that can be combined and serialised as a whole, then that's kind of confusing and could be more generic.

XML and HTML don't suffer as much from this I think.

Having __yaml__ return a string representation of a complete yaml document doesn't feel wrong at all to me, and the inability to embed the return of one __yaml__ call into another one doesn't strike me as a problem. Their purposes is just to be used as is, like the return of an API call for example.
But I agree, that not having the same behaviour (__html__() would just return a string with some tags, not a full html document with <html><head>... tags) depending on the type of representation is inconsistent and might feel weird and hard to understand.

To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.

Visit this group at https://groups.google.com/group/django-developers.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAFNZOJP45PchAC_1kqMOek0TRwfTc0qavfZ2Qf_-TzTpZto_eA%40mail.gmail.com.

[Aymeric Augustin]

Hello,

This could be a regression because https://github.com/django/django/commit/3483682749577b4b5a8141a766489d5b460e30e9 looks like it implemented that behavior; unfortunately but didn't include a test for the template engine, only for the escaping APIs.

(If it's indeed a regression and someone feels like bisecting it, I'm taking bets on the multiple template engines patch I landed in 1.8...)

Looking at https://code.djangoproject.com/ticket/23831 and https://code.djangoproject.com/ticket/7261, the intent was to provide full interoperability between Django and HTML escaping implementations that support on __html__; there was no exception for the template engine.

If someone wants to write a patch with a test, the least I can do is review it :-)

Best regards,

-- 

Aymeric.

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.

[Brice Parent]

>> Brice Parent:
>>
>> I like the idea a lot. But wouldn't it be better to have it as a
>> separate library? It seems to be something that could be quite
>> common, and have many use cases both inside and outside Django (we
>> could sometimes benefit from having an html exception message for
>> example).
>>
>
> As you have correctly observed this can’t live in a separate library
> entirely as it requires integration into the template language to be
> useful:

It can, but that would mean add a new dependency to Django (until this
module is accepted into the standard library, if it ever gets there).

- Brice

[Aymeric Augustin]

On 26 Sep 2017, at 22:18, Brice Parent <con...@brice.xyz> wrote:

Only question: Where should it stop? Should there also be a __json__, a __yaml__ and an __xml__ methods?

Hello,

__html__ is a fairly well established convention in the Python community, pioneered by Armin Ronacher around 2006 (give or take a couple years).

He first suggested Django supported it just over ten years ago: https://groups.google.com/d/msg/django-developers/IGsLpBwiKbc/4WeewXq2d1oJ

I'm not aware of any other conventional dunder method with such widespread adoption.

Best regards,

-- 

Aymeric.

[Jonas Haag]

Aymeric Augustin:

This could be a regression

Without having had a look at any of the links you’ve posted, this does NOT work in Jinja2 the way I suggested:

>>> jinja2.Template('{{foo}}').render({'foo': Money('1', '$')})

'$ 1'

So whatever the changes for better __html__ interoperability were, they seem unrelated to what I’m suggesting.

[Aymeric Augustin]

On 26 Sep 2017, at 22:56, Jonas Haag <jo...@lophus.org> wrote:

Without having had a look at any of the links you’ve posted, this does NOT work in Jinja2 the way I suggested:

>>> jinja2.Template('{{foo}}').render({'foo': Money('1', '$')})

'$ 1'

So whatever the changes for better __html__ interoperability were, they seem unrelated to what I’m suggesting.

That's because Jinja2 priorizes speed over security and disables autoescaping by default. To trigger __html__, you need to enable it.

Slightly adapted example:

>>> class Money:

...     def __init__(self, amount, currency):

...         self.amount = amount

...         self.currency = currency

...     def __str__(self):

...         return 'str %s %s' % (self.currency, self.amount)

...     def __html__(self):

...         # Always show amount and currency on same line

...         return 'html %s %s' % (self.currency, self.amount)

...

>>> import jinja2

>>> jinja2.Template('{{foo}}', autoescape=True).render({'foo': Money('1', '$')})

'html $ 1'

>>> jinja2.Template('{{foo}}').render({'foo': Money('1', '$')})

'str $ 1'

-- 

Aymeric.

[Aymeric Augustin]

And the test case for Django templates is:

>>> class Money:

...     def __init__(self, amount, currency):

...         self.amount = amount

...         self.currency = currency

...     def __str__(self):

...         return 'str %s %s' % (self.currency, self.amount)

...     def __html__(self):

...         # Always show amount and currency on same line

...         return 'html %s %s' % (self.currency, self.amount)

...

>>> from django.template import Context, Engine

>>> Engine().from_string('{{foo}}').render(Context({'foo': Money('1', '$')}))

'str $ 1'

This should return 'html $ 1'.

-- 

Aymeric.

[Jonas Haag]

Aymeric Augustin:

On 26 Sep 2017, at 22:56, Jonas Haag <jo...@lophus.org> wrote:

Without having had a look at any of the links you’ve posted, this does NOT work in Jinja2 the way I suggested:

>>> jinja2.Template('{{foo}}').render({'foo': Money('1', '$')})

'$ 1'

So whatever the changes for better __html__ interoperability were, they seem unrelated to what I’m suggesting.

That's because Jinja2 priorizes speed over security and disables autoescaping by default. To trigger __html__, you need to enable it.

I see, thanks for pointing this out!

[Jonas Haag]

Am 26.09.2017 um 22:50 schrieb Aymeric Augustin <aymeric....@polytechnique.org>:

Hello,

This could be a regression because https://github.com/django/django/commit/3483682749577b4b5a8141a766489d5b460e30e9 looks like it implemented that behavior;

The patch doesn’t implement that behaviour, as can be seen here: https://github.com/django/django/commit/3483682749577b4b5a8141a766489d5b460e30e9#diff-8cbe1fb6d589cb6e35b956704d7a1285L881 It still force_text’s everything.

Looks like this has indeed never been implemented in Django. Maybe as on oversight from the original suggestion by Armin, or maybe on purpose (why though?)?

[Patryk Zawadzki]

W dniu wtorek, 26 września 2017 14:34:29 UTC+2 użytkownik Jonas H napisał:

Proposal: Support the __html__ method as an alternative/addition to the __str__ for turning objects into strings in the template layer.

If this has been discussed before, please point me to it; I couldn't find anything with the search function.

Some custom classes may have, in addition to a __str__ representation, a natural representation that is better suited for HTML output.

Another proposal:

Introduce a single-dispatch html function (https://docs.python.org/3/library/functools.html#functools.singledispatch).

Have the default implementation call param.__html__() if it exists and fall back to param.__str__().

Document using html.register(...) as the preferred extension method.