[Django] #36980: TranslationCatalog.__getitem__ gives wrong priority to non-plural translations when plural forms mismatch

[Django]

#36980: TranslationCatalog.__getitem__ gives wrong priority to non-plural
translations when plural forms mismatch
-------------------------------------+-------------------------------------
Reporter: UHHHHHHHHHHHHHH | Type: Bug
Status: new | Component:
| Internationalization
Version: 5.1 | Severity: Normal
Keywords: i18n translation | Triage Stage:
plural TranslationCatalog | Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
= Description =

When a third-party package (e.g. django-allauth) has a slightly different
`Plural-Forms` header than the project's own translations (e.g. `plural=n
!= 1` vs `plural=(n != 1)`), `TranslationCatalog.update()` correctly
prepends a separate catalog to preserve plural lookup correctness.

However, `TranslationCatalog.__getitem__()` iterates catalogs in order and
returns the '''first match'''. This means the prepended catalog wins for
'''all''' lookups, including non-plural `gettext()` calls where plural
form separation is irrelevant. This silently overrides higher-priority
translations (from `LOCALE_PATHS` or later `INSTALLED_APPS`) with lower-
priority ones.

= Steps to reproduce =

1. Create a Django project with a translation in `LOCALE_PATHS`:
{{{
# project/locale/nl/LC_MESSAGES/django.po
# Plural-Forms: nplurals=2; plural=(n != 1);
msgid "Activate"
msgstr "Activeren"
}}}

2. Install a third-party app (e.g. django-allauth) that defines the same
msgid with a different `Plural-Forms` header:
{{{
# allauth/locale/nl/LC_MESSAGES/django.po
# Plural-Forms: nplurals=2; plural=n != 1;
msgid "Activate"
msgstr "Activeer"
}}}

Note: `n != 1` and `(n != 1)` are functionally identical and compile to
identical bytecode, but Python's `gettext.c2py()` produces functions with
different `__code__` objects. Django's `TranslationCatalog.update()`
compares `__code__` to decide whether to merge or prepend.

3. `gettext("Activate")` returns `"Activeer"` (allauth's version) instead
of `"Activeren"` (the project's version), even though `LOCALE_PATHS`
should have highest priority per Django's documentation.

= Root cause =

In `django/utils/translation/trans_real.py`:

{{{#!python
class TranslationCatalog:
def update(self, trans):
# Mismatched plural -> prepend to position 0
for cat, plural in zip(self._catalogs, self._plurals):
if trans.plural.__code__ == plural.__code__:
cat.update(trans._catalog)
break
else:
self._catalogs.insert(0, trans._catalog.copy())
self._plurals.insert(0, trans.plural)

def __getitem__(self, key):
# First catalog wins for ALL lookups
for cat in self._catalogs:
try:
return cat[key]
except KeyError:
pass
raise KeyError(key)
}}}

The `update()` prepend is correct for plural lookups (each catalog needs
its own plural function). But `__getitem__` shouldn't give the prepended
catalog priority for non-plural string lookups — these should follow the
normal priority order (LOCALE_PATHS > INSTALLED_APPS > Django built-in).

= Expected behavior =

Non-plural `gettext()` lookups should respect the documented translation
priority order regardless of plural form differences between catalogs.

= Actual behavior =

A third-party package with a cosmetically different `Plural-Forms` header
silently overrides all matching non-plural translations from higher-
priority sources.

= Impact =

In our project, this causes 28 translations to be silently wrong. The
project has no way to fix this without either:
* Monkey-patching `TranslationCatalog`
* Patching the third-party package's `.po` files in the Docker build
* Adding `pgettext` context to every conflicting string

= Environment =

* Django 5.1 (also affects 5.0, likely all versions since #34221 was
fixed)
* Python 3.12
* The issue was introduced/amplified by the fix for #34221 which changed
`update()` to only merge with the top catalog
--
Ticket URL: <https://code.djangoproject.com/ticket/36980>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.

[Django]

#36980: TranslationCatalog.__getitem__ gives wrong priority to non-plural
translations when plural forms mismatch
-------------------------------------+-------------------------------------

Reporter: UHHHHHHHHHHHHHH | Owner: (none)
Type: Bug | Status: closed
Component: | Version: 5.1
Internationalization | Resolution:
Severity: Normal | worksforme

Keywords: i18n translation | Triage Stage:
plural TranslationCatalog | Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------

Changes (by Jacob Walls):

* resolution: => worksforme
* status: new => closed

Comment:

I couldn't reproduce:

{{{#!py
In [4]: gettext('Activate')
Out[4]: 'Activeren'
}}}

After removing my LOCALE_PATHS translations, I get allauth's translation:

{{{#!py
In [3]: gettext('Activate')
Out[3]: 'Activeer'
}}}

Did you recompile messages?
--
Ticket URL: <https://code.djangoproject.com/ticket/36980#comment:1>