Seems to be a bug in makemessages management command

[Андрей Меньков]

Django management command doesn't recognize multiline strings as strings to localize. So such strings don't appear in resulting django.po files after running command

For example for such a string

{code}

{{ _(''' fhsdafjkshdfjkasdhfksj

         ytyerwtuiwyertuiywert''') }}

{code}

I might be the bug of GNU xgettext utility, because makemessages uses it to produce *.po files, but after running xgettext I became sure that the problem is in the makemessages command, not xgettext.

I ran xgettext using the following command

xgettext --keyword=gettext_noop --keyword=gettext_lazy --keyword=ngettext_lazy:1,2 --keyword=ugettext_noop --keyword=ugettext_lazy --keyword=ungettext_lazy:1,2 --keyword=pgettext:1c,2 --keyword=npgettext:1c,2,3 --keyword=pgettext_lazy:1c,2 --keyword=npgettext_lazy:1c,2,3 --from-code UTF-8 --add-comments=Translators --from-code utf-8 -d django -p ./output_dir -L Python temp_i18n_template.html

After running this command I got file output_dir/django.po with specified string inside it.

What I can do now? What do you think about this "bug" ?

P.S. It's my first attempt to contribute to Django somehow. So please do not judge strictly). I haven't found anywhere about this problem and about its' fixes too.

[Андрей Меньков]

May be it would better to post this to "Django developers" mailing list?

пятница, 30 мая 2014 г., 15:53:56 UTC+3 пользователь Андрей Меньков написал:

[Russell Keith-Magee]

Hi Андрей,

On Fri, May 30, 2014 at 9:50 PM, Андрей Меньков <nothingel...@gmail.com> wrote:

May be it would better to post this to "Django developers" mailing list?

Possibly - if you think you've found a bug, django-developers is a better audience for that. django-users is a better audience if you're just unsure if you're using a command/API correctly. 

 

пятница, 30 мая 2014 г., 15:53:56 UTC+3 пользователь Андрей Меньков написал:

Django management command doesn't recognize multiline strings as strings to localize. So such strings don't appear in resulting django.po files after running command

For example for such a string

{code}

{{ _(''' fhsdafjkshdfjkasdhfksj

         ytyerwtuiwyertuiywert''') }}

{code}

I'm a little unclear what's going on here. Are you using Django templates, or Jinja2? Django's template language doesn't allow multi-line template tags, and _() isn't valid translation syntax in a Django template. 

If you're using Jinja2, then the source of the bug may be in whatever layer you're using to integrate Jinja2 templates into Django.

 

I might be the bug of GNU xgettext utility, because makemessages uses it to produce *.po files, but after running xgettext I became sure that the problem is in the makemessages command, not xgettext.

I ran xgettext using the following command

xgettext --keyword=gettext_noop --keyword=gettext_lazy --keyword=ngettext_lazy:1,2 --keyword=ugettext_noop --keyword=ugettext_lazy --keyword=ungettext_lazy:1,2 --keyword=pgettext:1c,2 --keyword=npgettext:1c,2,3 --keyword=pgettext_lazy:1c,2 --keyword=npgettext_lazy:1c,2,3 --from-code UTF-8 --add-comments=Translators --from-code utf-8 -d django -p ./output_dir -L Python temp_i18n_template.html

After running this command I got file output_dir/django.po with specified string inside it.

What I can do now? What do you think about this "bug" ?

P.S. It's my first attempt to contribute to Django somehow. So please do not judge strictly). I haven't found anywhere about this problem and about its' fixes too.

The make messages is really just a thin wrapper around xgettext; the command you issued manually seems to match pretty closely what Django's makemessages command should be doing. If you're not getting the right output from Django's management command, it possible you've found a bug. If I had to guess, it will be with the code that is discovering the templates that need to be processed, but it's impossible to say for sure without your code in front of me.

If you want to do some digging, the relevant code is in django/core/management/commands/makemessages.py. Most of the code in that file is dedicated to building a command line and executing a call to xgettext; it should be fairly straightforward to understand. If you *do* find a problem, please open a ticket and let us know; or follow up with a thread on the django-dev if you're uncertain if the problem is a bug in Django or your own usage.

Yours,

Russ Magee %-)

[Андрей Меньков]

Hi, Russ.

Thank you for such a detailed response.

I think I have ever found the place where this really occurs. I was posted this message more for knowing if this really a bug or not (maybe I don't know something about template translation and multiline strings in them). 

I have used Jinja2 for template engine. But I think that at this level it's now important whether we use Django Template language or Jinja2, because both are later processed by xgettext with Python option specified.

I thing the problem is in function django.utils.translation.templatize. It's implementation could be found in django.utils.translation.trans_real module. 

From docstring for it:

Turns a Django template into something that is understood by xgettext. It
    does so by translating the Django translation tags into standard gettext
    function invocations.

Function calling results:
>> templatize(" {{ _('kfhdsajfkh') }} ")
>> ('kfhdsajfkh')

>> templatize(""" {{ _(''' abcdefg
                            hhjkfhsdajfk
                            fsdauyuirywer
                       ''') }}
  
              """)
>>  XX XXXXX XXXXXXX
                         XXXXXXXXXXXX
                         XXXXXXXXXXXXX
                     XXXX XX


So it can be seen that templatize function marks multiline strings as not needed to translate.

суббота, 31 мая 2014 г., 5:09:17 UTC+3 пользователь Russell Keith-Magee написал:

[Ramiro Morales]

On Sat, May 31, 2014 at 7:50 AM, Андрей Меньков
<nothingel...@gmail.com> wrote:
>
> I thing the problem is in function django.utils.translation.templatize. It's
> implementation could be found in django.utils.translation.trans_real module.
>
> From docstring for it:
>
> Turns a Django template into something that is understood by xgettext. It
> does so by translating the Django translation tags into standard gettext
> function invocations.
>
> Function calling results:
>>> templatize(" {{ _('kfhdsajfkh') }} ")
>>> ('kfhdsajfkh')
>
>>> templatize(""" {{ _(''' abcdefg
> hhjkfhsdajfk
> fsdauyuirywer
> ''') }}
>
> """)
>>> XX XXXXX XXXXXXX
> XXXXXXXXXXXX
> XXXXXXXXXXXXX
> XXXX XX
>
>
> So it can be seen that templatize function marks multiline strings as not
> needed to translate.

As Russ says, it's not clear to me if the _() template construct you
are using is from Django i18n or from Jinja.

Django template tags (including trans) aren't multi-line. There is a
ongoing discussion about this.

If you use the Django blocktrans template tag, multi-line literals are
perfectly suported by templatize():

In [1]: from django.utils.translation import templatize

In [2]: templatize("""{% blocktrans %}foo
bar
baz
{% endblocktrans %}""")
Out[2]: " gettext(u'foo\\nbar\\nbaz\\n') SSS\nSSS\nSSS\n"

In [3]:

--
Ramiro Morales
@ramiromorales

[Андрей Меньков]

I'm using Jinja2. There _() construct is equivalent to ugettext().

I tried 'blocktrans' tag - it really works, that right).

But what to do with Jinja2 templates? Or may be this is not a bug but some sort of improvement for compatibility with Jinja2?

--
You received this message because you are subscribed to a topic in the Google Groups "Django users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/django-users/q2KNl0Bha6o/unsubscribe.
To unsubscribe from this group and all its topics, send an email to django-users...@googlegroups.com.
To post to this group, send email to django...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAO7PdF8%3DrNwR%3DOxAGs7TVv8NXA6c8Q%3DA5kD71eOb2mY8mWixCA%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

[Russell Keith-Magee]

Hi Андрей,

Unless you can reproduce this with Django's own template language -- which *doesn't* support multiline content, it's not a bug. However, Jinja2 use is common in the community, so if we can apply a patch that improves Jinja2 support, then we'd certainly consider that patch.

Yours,

Russ Magee %-)

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.

To post to this group, send email to django...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/CAL40Fptj%3DuHZ9qOdCgRpLBdsJw93%3DVEX7JCBO2BFzQuh_DV3NA%40mail.gmail.com.