[Django] #25297: The makemessages command does not detect the context of {% trans %} templatetags with filter

[Django]

#25297: The makemessages command does not detect the context of {% trans %}
templatetags with filter
--------------------------------------+--------------------
Reporter: apocquet | Owner: nobody
Type: Uncategorized | Status: new
Component: Internationalization | Version: 1.8
Severity: Normal | Keywords:
Triage Stage: Unreviewed | Has patch: 0
Easy pickings: 0 | UI/UX: 0
--------------------------------------+--------------------
The following:
{{{#!html+django
{% trans 'inactive'|upper context 'plural' %}
}}}

Is (incorrectly) extracted as:
{{{#!po
msgid "inactive"
msgstr ""
}}}

Whereas the following:
{{{#!html+django
{% trans 'inactive' context 'plural' %}
}}}

Is (correctly) extracted as:
{{{#!po
msgctxt "plural"
msgid "inactive"
msgstr ""
}}}

Translating the former has no effect on the template rendering, as it
lacks the context.
Translating the latter makes the translation to be correctly rendered,
both without the filter, and with the filter if added '''after'''
{{{makemessages}}} was called.

I think this is a bug with the {{{makemessages}}} command.

Environnement:
Linux Mint Debian Edition
CPython 3.4.2
Django 1.8.3
GNU gettext 0.19.3

Command used:
{{{./manage.py makemessages -v 1 -l fr}}}
{{{./manage.py compilemessages}}}

--
Ticket URL: <https://code.djangoproject.com/ticket/25297>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

[Django]

#25297: The makemessages command does not detect the context of {% trans %}

templatetags with a filter
-------------------------------------+-------------------------------------

Reporter: apocquet | Owner: nobody
Type: Uncategorized | Status: new

Component: | Version: 1.8
Internationalization |
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0

Easy pickings: 0 | UI/UX: 0

-------------------------------------+-------------------------------------
Changes (by apocquet):

* needs_better_patch: => 0
* needs_docs: => 0
* needs_tests: => 0

--
Ticket URL: <https://code.djangoproject.com/ticket/25297#comment:1>

[Django]

#25297: makemessages doesn't detect the context of {% trans %} strings with a
filter
--------------------------------------+------------------------------------
Reporter: apocquet | Owner: nobody
Type: Bug | Status: new
Component: Internationalization | Version: 1.8
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted

Has patch: 0 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0

Easy pickings: 0 | UI/UX: 0

--------------------------------------+------------------------------------
Changes (by timgraham):

* type: Uncategorized => Bug
* stage: Unreviewed => Accepted

Comment:

If this cannot be made to work for some reason, documenting the
restriction may be helpful.

--
Ticket URL: <https://code.djangoproject.com/ticket/25297#comment:2>

[Django]

#25297: makemessages doesn't detect the context of {% trans %} strings with a
filter
--------------------------------------+------------------------------------

Reporter: apocquet | Owner: nobody
Type: Bug | Status: new
Component: Internationalization | Version: 1.8

Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0

Easy pickings: 0 | UI/UX: 0

--------------------------------------+------------------------------------

Comment (by apocquet):

I think the problem may come from the `inline_re` definition in
django.utils.translation.trans_real . It it used by the `templatize`
function, which it itself called from the commandline handler.

Current:
{{{#!python
inline_re =
re.compile(r"""^\s*trans\s+((?:"[^"]*?")|(?:'[^']*?'))(\s+.*context\s+((?:"[^"]*?")|(?:'[^']*?')))?\s*""")
}}}

Possible correction:
{{{#!python
inline_re =
re.compile(r"""^\s*trans\s+((?:"[^"]*?")|(?:'[^']*?'))(?:\s*\|\s*[^\s:]+(?::(?:[^\s'":]+|(?:"[^"]*?")|(?:'[^']*?')))?)*(\s+.*context\s+((?:"[^"]*?")|(?:'[^']*?')))?\s*""")
}}}

The `(?:\s*\|\s*[^\s]+(?::(?:\s+|(?:"[^"]*?")|(?:'[^']*?'))?))*` regex
should catch and throw away any filter. I may be wrong, I don't know the
precise syntax of a filter name.
The first part is quite simple and matches a `|` followed by a filter
name. The second part (starting with `?::`) is more complex because it
must matches filter arguments, including string literals.

Test
[http://pythex.org/?regex=%5E%5Cs*trans%5Cs%2B((%3F%3A%22%5B%5E%22%5D*%3F%22)%7C(%3F%3A%27%5B%5E%27%5D*%3F%27))(%3F%3A%5Cs*%5C%7C%5Cs*%5B%5E%5Cs%3A%5D%2B(%3F%3A%3A(%3F%3A%5B%5E%5Cs%27%22%3A%5D%2B%7C(%3F%3A%22%5B%5E%22%5D*%3F%22)%7C(%3F%3A%27%5B%5E%27%5D*%3F%27)))%3F)*(%5Cs%2B.*context%5Cs%2B((%3F%3A%22%5B%5E%22%5D*%3F%22)%7C(%3F%3A%27%5B%5E%27%5D*%3F%27)))%3F%5Cs*&test_string=%20trans%20%27inactive%27%20%7C%20upper%3A%27foo%20bar%27%20context%20%27plural%27%20&ignorecase=0&multiline=0&dotall=0&verbose=0
here].

--
Ticket URL: <https://code.djangoproject.com/ticket/25297#comment:3>

[Django]

#25297: makemessages doesn't detect the context of {% trans %} strings with a
filter
--------------------------------------+------------------------------------

Reporter: apocquet | Owner: nobody
Type: Bug | Status: new
Component: Internationalization | Version: 1.8

Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted

Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1

Easy pickings: 0 | UI/UX: 0

--------------------------------------+------------------------------------
Changes (by timgraham):

* needs_better_patch: 0 => 1
* has_patch: 0 => 1

Comment:

[https://github.com/django/django/pull/5203 PR] (but has a test failure).

--
Ticket URL: <https://code.djangoproject.com/ticket/25297#comment:4>

[Django]

#25297: makemessages doesn't detect the context of {% trans %} strings with a
filter
--------------------------------------+------------------------------------

Reporter: apocquet | Owner: nobody
Type: Bug | Status: new
Component: Internationalization | Version: 1.8

Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 0

Easy pickings: 0 | UI/UX: 0

--------------------------------------+------------------------------------
Changes (by apocquet):

* needs_better_patch: 1 => 0

--
Ticket URL: <https://code.djangoproject.com/ticket/25297#comment:5>

[Django]

#25297: makemessages doesn't detect the context of {% trans %} strings with a
filter
--------------------------------------+------------------------------------

Reporter: apocquet | Owner: nobody
Type: Bug | Status: new
Component: Internationalization | Version: 1.8

Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0

Needs tests: 0 | Patch needs improvement: 1

Easy pickings: 0 | UI/UX: 0

--------------------------------------+------------------------------------
Changes (by timgraham):

* needs_better_patch: 0 => 1

Comment:

Left some comments for improvement on the pull request.

--
Ticket URL: <https://code.djangoproject.com/ticket/25297#comment:6>

[Django]

#25297: makemessages doesn't detect the context of {% trans %} strings with a
filter
-------------------------------------+-------------------------------------
Reporter: apocquet | Owner: nobody
Type: Bug | Status: new
Component: | Version: 1.8
Internationalization |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Ready for
| checkin

Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1

Easy pickings: 0 | UI/UX: 0

-------------------------------------+-------------------------------------
Changes (by apocquet):

* stage: Accepted => Ready for checkin

--
Ticket URL: <https://code.djangoproject.com/ticket/25297#comment:7>

[Django]

#25297: makemessages doesn't detect the context of {% trans %} strings with a
filter
-------------------------------------+-------------------------------------

Reporter: apocquet | Owner: nobody
Type: Bug | Status: closed
Component: | Version: 1.8
Internationalization |
Severity: Normal | Resolution: fixed

Keywords: | Triage Stage: Ready for
| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1

Easy pickings: 0 | UI/UX: 0

-------------------------------------+-------------------------------------
Changes (by Tim Graham <timograham@…>):

* status: new => closed
* resolution: => fixed

Comment:

In [changeset:"e7b7f94678c632791f0323f8c61ce611275605c5" e7b7f946]:
{{{
#!CommitTicketReference repository=""
revision="e7b7f94678c632791f0323f8c61ce611275605c5"
Fixed #25297 -- Allowed makemessages to work with {% trans %} tags that
use template filters.
}}}

--
Ticket URL: <https://code.djangoproject.com/ticket/25297#comment:8>