Compilemessages and percent char in msgids

[Marcin Nowak]

Hi.

I'm not sure what is going on, but I've updated recently Django to 1.11.13 and two things seems to be broken (from my perspective):

• behaviour of translation messages containing % is inconsistent, for example:
  
  
• prevously used msgis are staying untranslated with trans/blocktrans:
  
  msgid "Source %"
  msgstr "Translated %"
  
  ugettext("Source %") -> "Translated %"  [OK]
  trans "Source %" -> "Source %"  [FAIL, expected "Translated %"]
  blocktrans "Source %" -> "Source %"  [FAIL, expected "Translated %"]
  
  
• msgids with double percent chars:
  
  msgid "Source2 %%"
  msgstr "Translated2 %"
  
  ugettext("Source2 %") -> "Source2 %"  [FAIL, should fail probably?]
  
  ugettext("Source2 %%") -> "Translated2 %"  [OK]
  trans "Source2 %" -> "Translated2 %"  [OK]
  blocktrans "Source2 %" -> "Source2 %" [FAIL, expected "Translated2 %"]
  blocktrans "Source2 %%" -> "Source2 %%"  [FAIL, expected "Source2 %"]
  
  activate(original) /* no msgid/msgstrs defined */
  
  ugettext("Source2 %%") -> "Source2 %%"  [FAIL, expected "Source2 %"]
  
  
• msgids and msgstr with double percent chars:
  
  msgid "Source3 %%"
  msgstr "Translated3 %%"
  
  ugettext("Source3 %%") -> "Translated3 %%" [FAIL, expected "Translated3 %"]
  ugettext("Source3 %") -> "Source3 %" [FAIL]
  trans "Source3 %" -> "Translated3 %" [OK]
  blocktrans "Source3 %" -> "Translated3 %" [OK]
  blocktrans "Source3 %%" -> "Source3 %%" [FAIL]
  
  activate(original) /* no msgid/msgstrs defined */
  
  ugettext("Source3 %%") -> "Source3 %%" [FAIL, expected "Source3 %"]
  ugettext("Source3 %")-> "Source3 %" [OK]
  trans "Source3 %" -> "Source3 %" [OK]
  trans "Source3 %%" -> "Source3 %%" [OK]
  blocktrans "Source3 %" -> "Source3 %" [OK]
  blocktrans "Source3 %%" -> "Source3 %%" [OK]
  
  
• and second is wrong compilemessages behaviour, because it scans and compiles all files starting from cwd. Eariler only django apps were scanned. Now this command is trying to compile everything, also unused or already compiled files from 3rd party packages.
  

First issue must be fixed. Translations behaviour isn't consistent. I cannot make any reliable translations having % in msgids anymore. Assuming that double percent in msgid is required (one for escape, one for percent sign), this makes wrong translations with "p/n/u/gettext" functions if msgstrs with single % were not defined. Please imagine switching to not fully translated language, and user will get double % in messages (excl. templates with trans, but not blocktrans).

Second should be probably fixed by reverting a patch which adds a tree scan of files. Currently I did  workaround by copying older version of compilemessages into my project's app. But this is not how Django should work. I'm using buldout where all external eggs,parts and other projects lies in cwd.

Any thoughts?

Kind Regards,

Marcin

[Ramiro Morales]

Marcin,

You have upgraded recently to version 1.11.13?

IIRC handling of percent symbols in translatable literals has been the subject of a few commits along the history trying to get it right for as many use cases as possible. It's entirely possible it was broken in some way during the 1.11 release cycle.

Please examine the history for the compilemessages module source file. Perhaps you can cherry pick one or more commit to your local copy.

If memory doesn't fail me there is some complexity related to the fact that xgettext (used by makemessages) auto detects some strings that contain % symbols as literals using 'traditional' Python string interpolation and there is no way to control such detection.

HTH

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/5918116a-319d-4d5b-b0dd-dcfdbf7b3928%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Ramiro Morales
@ramiromorales

[Marcin Nowak]

W dniu wtorek, 24 lipca 2018 14:43:51 UTC+2 użytkownik Ramiro Morales napisał:

Marcin,

You have upgraded recently to version 1.11.13?

About month ago. It was litlle painful process, BTW.

 

IIRC handling of percent symbols in translatable literals has been the subject of a few commits along the history trying to get it right for as many use cases as possible. It's entirely possible it was broken in some way during the 1.11 release cycle.

I saw these commits and some tickets (#24257 for example). I tried to match these changes, but I'm still getting bug reports about translations containing a percent char.

Please examine the history for the compilemessages module source file. Perhaps you can cherry pick one or more commit to your local copy.

I did that. I'm using older version, copied into my project/app.

 

If memory doesn't fail me there is some complexity related to the fact that xgettext (used by makemessages) auto detects some strings that contain % symbols as literals using 'traditional' Python string interpolation and there is no way to control such detection.

But the problem is that translations are inconsistent. It would be better to replace/fix/workaround xgettext instead of making translation process invalid...

BR,

Marcin

[Marcin Nowak]

BUMP