[Django] #24257: The trans template tag fails to get a message when there is a % character in the string
Django
in the string
--------------------------------------+------------------------------------
Reporter: aboudreault | Owner: nobody
Type: Bug | Status: new
Component: Internationalization | Version: 1.7
Severity: Normal | Keywords: trans templatetag i18n
Triage Stage: Unreviewed | Has patch: 0
Easy pickings: 0 | UI/UX: 0
--------------------------------------+------------------------------------
I am using a nested template with translation in my django template file.
This is my basic html code:
{{{
<script id="invoice-date-modal-template" type="text/html">
<div id="modal" class="reveal-modal tiny " data-reveal>
<h5><%= client %> - <%= date %></h5>
<h6>{% trans "Modify the invoice state of <%= client %> dated
of <%= date %>?" %}</h6>
<p>{% trans "Select the session invoice date and confirm."
%}</p>
<div class="row">...</div>
</div>
</script>
}}}
makemessages && compilemessages.
In django.po: I see that the % characters has been doubled:
{{{
msgid ""
"Modify the invoice state of <%%= client %%> of <%%= date %%>?"
msgstr ""
"Changer l'état de facturation de <%%= client %%> du <%%= date %%>?"
}}}
In my web application, I can see that my first trans is not translated,
but the second one is ok.
I've found a workaround for that: using blocktrans instead of trans. Doing
it this way work in my application:
{{{
<h6>{% blocktrans %}Modify the invoice state of <%= client %> of <%= date
%>?{% endblocktrans %}</h6>
}}}
Looks like something is different or missing with the trans template tag.
--
Ticket URL: <https://code.djangoproject.com/ticket/24257>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
in the string
Reporter: aboudreault | Owner: nobody
Type: Bug | Status: new
Internationalization |
Severity: Normal | Resolution:
Keywords: trans templatetag | Triage Stage: Accepted
i18n |
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by claudep):
* needs_better_patch: => 0
* needs_docs: => 0
* needs_tests: => 0
* stage: Unreviewed => Accepted
Comment:
Sigh... will we get this one right one day :-/ (see #11240). I was able to
reproduce it.
--
Ticket URL: <https://code.djangoproject.com/ticket/24257#comment:1>
Django
in the string
Reporter: aboudreault | Owner: nobody
Type: Bug | Status: new
Internationalization |
Severity: Normal | Resolution:
Keywords: trans templatetag | Triage Stage: Accepted
i18n |
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by claudep):
The problem with the current solution of doubling percents in `trans`
string content (to prevent msgfmt check errors, as explained in #11240),
is that it's almost impossible to make the reverse operation by replacing
double percents by single ones, because the result is a lazy translation
(and should be kept lazy), hence post-processing the string is not
possible. `blocktrans` does that implicitely during the placeholder
substitution operation (`result % data`).
The only solution I can see currently is to stop doubling percents in
`trans` strings, and document that in case of translation problems, a
translator comment should be added before the string: `# Translators:
xgettext:no-python-format` (to prevent msgfmt from complaining).
--
Ticket URL: <https://code.djangoproject.com/ticket/24257#comment:2>
Django
in the string
Reporter: aboudreault | Owner: nobody
Type: Bug | Status: new
Internationalization |
Severity: Normal | Resolution:
Keywords: trans templatetag | Triage Stage: Accepted
i18n |
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by claudep):
See that commit where the above solution is applied:
https://github.com/claudep/django/compare/24257
The downside is that a same string translated with `trans` or with
`blocktrans` does not appear identically in the po file:
{{{
{% trans "50% of something" %} => msgid "50% of something"
{% blocktrans %}50% of something{% endblocktrans %} => msgid "50%% of
something"
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/24257#comment:3>
Django
in the string
Reporter: aboudreault | Owner: nobody
Type: Bug | Status: new
Internationalization |
Severity: Normal | Resolution:
Keywords: trans templatetag | Triage Stage: Accepted
i18n |
Has patch: 0 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Comment (by beck):
This was the ticket I decided to tackle at the pycon sprints (claudep wish
you were there so I could thank you in person for your patience on my
previous pull... and this one too).
This issue turned out to be quite difficult. I've tried multiple
approaches.
== Long story short ==
The simplest solution is to ensure that all translation messages that have
a percent signs get python formatting.
== Long story long ==
In an ideal world we would be able to use the template string (with it's
curly brace variables) as our msgid. Since that is not possible, template
copy is coerced into a python formatted / sprintf string.
This is a source of much pain and confusion, especially with percent
signs. Why?
'''xgettext is awkward.'''
xgettext will identify all python-format strings, example:
{{{
echo 'gettext("%s");' | xgettext --language=python --output=- -
#, python-format
msgid "%s"
msgstr ""
}}}
This is all good. It gets awkward when you pass it an invalid python fmt
string, say:
{{{
echo 'gettext("%s costs 10%");' | xgettext --language=python --output=- -
msgid "%s costs 10%"
msgstr ""
}}}
This is awkward because the single `%` is seen as invalid python format
and so the message id is not marked as we would expect.
'''sprintf is awkward.'''
Since humans are bad parsers, when I look at the format, `% %s` I see a
percent followed by a string specifier. What this actually means a percent
sign specifier, with a whitespace conversion flag, followed by an `s`
character.
Example:
{{{
>>> "% %s" % ()
'%s'
>>> "% 10%s" % ()
' %s'
>>>
}}}
These two bits awkwardness in gettext and sprinf has caused some confusion
as past developers have tried to shoehorn the template language into these
technologies.
Example:
{{{
{% blocktrans with a=1 %}Blocktrans extraction shouldn't double escape
this: %%, a={{ a }}{% endblocktrans %}
}}}
The correct msgid is:
{{{
Blocktrans extraction shouldn't double escape this: %%%%, a=%(a)s
}}}
Not:
{{{
Blocktrans extraction shouldn't double escape this: %%, a={{ a }}
}}}
== The solution ==
First get some robust tests that capture all corner cases of awkwardness.
I've added a sample app to the i18n tests to make it easier to write tests
that evaluate extraction and translation using the same gettext catalogs.
I then refactored tests addressing extraction and translation when a `%`
is involved.
Second, knock out any special handling of percents and ensure valid
python-formatting on strings.
Technically not all template strings with a percent should be python-
formatted. If the template was only `{% trans "10%" %}` this could go
into the gettext catalog with msgid `10%` but such things is not possible
with how `blocktrans` is rendered.
When using `trans` and `blocktrans` with the same copy, the two should
always extract the same msgid and render identically.
The pull: https://github.com/django/django/pull/4549
--
Ticket URL: <https://code.djangoproject.com/ticket/24257#comment:4>
Django
in the string
Reporter: aboudreault | Owner: nobody
Type: Bug | Status: new
Internationalization |
Severity: Normal | Resolution:
Keywords: trans templatetag | Triage Stage: Accepted
i18n |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
Changes (by timgraham):
* has_patch: 0 => 1
--
Ticket URL: <https://code.djangoproject.com/ticket/24257#comment:5>
Django
in the string
Reporter: aboudreault | Owner: nobody
Type: Bug | Status: new
Internationalization |
Severity: Normal | Resolution:
Keywords: trans templatetag | Triage Stage: Accepted
i18n |
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by timgraham):
* needs_better_patch: 0 => 1
Comment:
Left comments for improvement on the PR.
--
Ticket URL: <https://code.djangoproject.com/ticket/24257#comment:6>
Django
in the string
Reporter: aboudreault | Owner: nobody
Type: Bug | Status: new
Internationalization |
Severity: Normal | Resolution:
Keywords: trans templatetag | Triage Stage: Accepted
i18n |
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by timgraham):
* needs_better_patch: 1 => 0
--
Ticket URL: <https://code.djangoproject.com/ticket/24257#comment:7>
Django
in the string
Reporter: aboudreault | Owner: nobody
Type: Bug | Status: closed
Component: | Version: 1.7
Internationalization |
Severity: Normal | Resolution: fixed
Keywords: trans templatetag | Triage Stage: Accepted
i18n |
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
Changes (by Tim Graham <timograham@…>):
* status: new => closed
* resolution: => fixed
Comment:
In [changeset:"b7508896fbe19ec2cdeb81565cd587091b6b68d0" b7508896]:
{{{
#!CommitTicketReference repository=""
revision="b7508896fbe19ec2cdeb81565cd587091b6b68d0"
Fixed #24257 -- Corrected i18n handling of percent signs.
Refactored tests to use a sample project.
Updated extraction:
* Removed special handling of single percent signs.
* When extracting messages from template text, doubled all percent signs
so they are not interpreted by gettext as string format flags. All
strings extracted by gettext, if containing a percent sign, will now
be labeled "#, python-format".
Updated translation:
* Used "%%" for "%" in template text before calling gettext.
* Updated {% trans %} rendering to restore "%" from "%%".
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/24257#comment:8>