[Django] #29973: compilemessages misses ignore option, compiles more than needed
Django
-------------------------------------+-------------------------------------
Reporter: stitch | Owner: nobody
Type: | Status: new
Cleanup/optimization |
Component: Core | Version: 2.1
(Management commands) | Keywords: compilemessages,
Severity: Normal | ignore, translation,
Triage Stage: | internationalization
Unreviewed | Has patch: 0
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 1
UI/UX: 0 |
-------------------------------------+-------------------------------------
The makemessages command contain an ignore flag, this allows you to skip
directories like 'vendor'.
In our project we have a directory called .tox, this directory is ignored
during makemessages. Yet, when compilemessages is run, the .tox directory
(which contains a lot of sources) is included with compiling translations.
Now we might have things wrong, but it seems strange to me that
makemessages has ignore options, while compilemessages doesn't. Even if
it's wrong, we might still want to ignore this folder because we're clumsy
and it works just 'good enough'.
The command we'd like to use looks like:
{{{
django compilemessages --ignore vendor --ignore .tox -l EN
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/29973>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
-------------------------------------+-------------------------------------
Reporter: stitch | Owner: nobody
Cleanup/optimization |
Component: Core (Management | Version: 2.1
commands) |
Severity: Normal | Resolution: invalid
Keywords: compilemessages, | Triage Stage:
ignore, translation, | Unreviewed
internationalization |
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: new => closed
* resolution: => invalid
Comment:
I guess you want to use the `--exclude` option.
{{{
$ python manage.py compilemessages --ignore vendor
usage: manage.py compilemessages [-h] [--locale LOCALE] [--exclude
EXCLUDE]
[--use-fuzzy] [--version] [-v {0,1,2,3}]
[--settings SETTINGS]
[--pythonpath PYTHONPATH] [--traceback]
[--no-color] [--force-color]
manage.py compilemessages: error: unrecognized arguments: --ignore vendor
}}}
See TicketClosingReasons/UseSupportChannels for places to get help before
filing a bug report.
--
Ticket URL: <https://code.djangoproject.com/ticket/29973#comment:1>
Django
-------------------------------------+-------------------------------------
Type: | Status: new
Cleanup/optimization |
Component: Core (Management | Version: 2.1
commands) |
Severity: Normal | Resolution:
ignore, translation, | Unreviewed
internationalization |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: closed => new
* resolution: invalid =>
Comment:
The --exclude command is about locales. This issue is about having too
many directories being used in compilemessages.
Here is an example where i'm in need of an --ignore during
compilemessages, including the line where the issue resides and an example
output of where things go wrong:
Using this command:
{{{
django-admin compilemessages -l NL --exclude DE
}}}
And this filter on what directory constitutes a 'locale directory':
https://github.com/django/django/blob/eac9ab7ebb1ce0cbbc79c4cf65e8f70b0635a240/django/core/management/commands/compilemessages.py#L69
It's clear that the filter is far too greedy. And in my case it results in
a list of directories that should not be included. Hence the request for
an --ignore option just like in makemessages.
The directory filter currently delivers this output on my machine, again:
everything named 'locale' is included.
{{{
['conf/locale',
'locale',
'locale',
'./locale',
'./.tox/default/lib/python3.6/site-packages/import_export/locale',
'./.tox/default/lib/python3.6/site-packages/jet/locale',
'./.tox/default/lib/python3.6/site-packages/jet/dashboard/locale',
'./.tox/default/lib/python3.6/site-packages/constance/locale',
'./.tox/default/lib/python3.6/site-packages/adminsortable2/locale',
'./.tox/default/lib/python3.6/site-packages/leaflet/locale',
'./.tox/default/lib/python3.6/site-packages/dal_select2/locale',
'./.tox/default/lib/python3.6/site-packages/sphinx/locale',
'./.tox/default/lib/python3.6/site-packages/django_countries/locale',
'./.tox/default/lib/python3.6/site-packages/django_extensions/locale',
'./.tox/default/lib/python3.6/site-packages/django/contrib/auth/locale',
'./.tox/default/lib/python3.6/site-packages/django/contrib/admin/locale',
'./.tox/default/lib/python3.6/site-
packages/django/contrib/flatpages/locale',
'./.tox/default/lib/python3.6/site-packages/django/contrib/sites/locale',
'./.tox/default/lib/python3.6/site-
packages/django/contrib/postgres/locale',
'./.tox/default/lib/python3.6/site-
packages/django/contrib/redirects/locale',
'./.tox/default/lib/python3.6/site-
packages/django/contrib/sessions/locale',
'./.tox/default/lib/python3.6/site-
packages/django/contrib/humanize/locale',
'./.tox/default/lib/python3.6/site-
packages/django/contrib/contenttypes/locale',
'./.tox/default/lib/python3.6/site-packages/django/contrib/gis/locale',
'./.tox/default/lib/python3.6/site-
packages/django/contrib/admindocs/locale',
'./.tox/default/lib/python3.6/site-packages/django/conf/locale',
'./.tox/default/lib/python3.6/site-packages/debug_toolbar/locale',
'./failmap/locale',
'./failmap/organizations/locale',
'./failmap/game/locale',
'./failmap/map/locale',
'./vendor/dnscheck/engine/locale']
}}}
You can also run it for yourself, by creating a 'locale' directory
somewhere and then run this part of the django source code:
https://github.com/django/django/blob/eac9ab7ebb1ce0cbbc79c4cf65e8f70b0635a240/django/core/management/commands/compilemessages.py#L67
{{{
basedirs=[]
for dirpath, dirnames, filenames in os.walk('.', topdown=True):
for dirname in dirnames:
if dirname == 'locale':
basedirs.append(os.path.join(dirpath, dirname))
basedirs
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/29973#comment:2>
Django
-------------------------------------+-------------------------------------
Type: | Pascual
Cleanup/optimization | Status: assigned
Component: Core (Management | Version: 2.1
commands) |
Severity: Normal | Resolution:
Keywords: compilemessages, | Triage Stage:
ignore, translation, | Unreviewed
internationalization |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: new => assigned
* owner: nobody => Manu Pascual
--
Ticket URL: <https://code.djangoproject.com/ticket/29973#comment:3>
Django
-------------------------------------+-------------------------------------
Type: | Pascual
Cleanup/optimization | Status: assigned
Component: Core (Management | Version: 2.1
commands) |
Severity: Normal | Resolution:
Keywords: compilemessages, | Triage Stage:
ignore, translation, | Unreviewed
internationalization |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Ryan Siemens):
Hey Manu,
I have a patch in place for this. Do you mind if I take over?
--
Ticket URL: <https://code.djangoproject.com/ticket/29973#comment:4>
Django
-------------------------------------+-------------------------------------
Type: | Pascual
Cleanup/optimization | Status: assigned
Component: Core (Management | Version: 2.1
commands) |
Severity: Normal | Resolution:
Keywords: compilemessages, | Triage Stage:
ignore, translation, | Unreviewed
internationalization |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Manu Pascual):
Of course, I would have liked to do it this last weekend, but I was sick.
So all your sir.
--
Ticket URL: <https://code.djangoproject.com/ticket/29973#comment:5>
Django
-------------------------------------+-------------------------------------
Type: | Status: new
Cleanup/optimization |
Component: Core (Management | Version: 2.1
commands) |
Severity: Normal | Resolution:
Keywords: compilemessages, | Triage Stage:
ignore, translation, | Unreviewed
internationalization |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: assigned => new
* owner: Manu Pascual => (none)
--
Ticket URL: <https://code.djangoproject.com/ticket/29973#comment:6>
Django
-------------------------------------+-------------------------------------
Type: | Siemens
Cleanup/optimization | Status: assigned
Component: Core (Management | Version: 2.1
commands) |
Severity: Normal | Resolution:
Keywords: compilemessages, | Triage Stage:
ignore, translation, | Unreviewed
internationalization |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------
* owner: (none) => Ryan Siemens
* status: new => assigned
--
Ticket URL: <https://code.djangoproject.com/ticket/29973#comment:7>
Django
-------------------------------------+-------------------------------------
Type: | Siemens
Cleanup/optimization | Status: assigned
Component: Core (Management | Version: 2.1
commands) |
Severity: Normal | Resolution:
Keywords: compilemessages, | Triage Stage:
ignore, translation, | Unreviewed
internationalization |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------
* cc: Ryan Siemens (added)
* has_patch: 0 => 1
--
Ticket URL: <https://code.djangoproject.com/ticket/29973#comment:8>
Django
-------------------------------------+-------------------------------------
| Siemens
Type: New feature | Status: assigned
Component: Core (Management | Version: 2.1
commands) |
Severity: Normal | Resolution:
Keywords: compilemessages, | Triage Stage:
ignore, translation, | Unreviewed
internationalization |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------
* type: Cleanup/optimization => New feature
--
Ticket URL: <https://code.djangoproject.com/ticket/29973#comment:9>
Django
-------------------------------------+-------------------------------------
| Siemens
Type: New feature | Status: assigned
Component: Core (Management | Version: 2.1
commands) |
Severity: Normal | Resolution:
Keywords: compilemessages, | Triage Stage:
ignore, translation, | Unreviewed
internationalization |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Ryan Siemens):
I had initially implemented this to just drop a directory, by name, but I
think thats actually incorrect.
Consider a directory structure like so
{{{
./
|- locale/
|- foo/
|- locale/
|- bar/
|- foo/
|- locale/
}}}
With my current patch doing `compilemessages --ignore=foo` would drop
`./foo` and `./bar/foo` from being searched, but I think it should only
drop dirs relative to the path. So in this case only `./foo` wouldn't be
searched. Also I think I it should support globbing similar to
`makemessages --ignore`. Thoughts or opinions?
Thanks.
--
Ticket URL: <https://code.djangoproject.com/ticket/29973#comment:10>
Django
-------------------------------------+-------------------------------------
| Siemens
Type: New feature | Status: assigned
Component: Core (Management | Version: 2.1
commands) |
Severity: Normal | Resolution:
ignore, translation, |
internationalization |
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------
* stage: Unreviewed => Accepted
Comment:
OK, yes. This seems a reasonable addition to mirror `makemessages`.
--
Ticket URL: <https://code.djangoproject.com/ticket/29973#comment:11>
Django
-------------------------------------+-------------------------------------
| Siemens
Type: New feature | Status: assigned
Component: Core (Management | Version: 2.1
commands) |
Severity: Normal | Resolution:
Keywords: compilemessages, | Triage Stage: Accepted
ignore, translation, |
internationalization |
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------
* cc: Simon Charette (added)
* needs_better_patch: 0 => 1
Comment:
Left a few comments for non-critical improvements.
--
Ticket URL: <https://code.djangoproject.com/ticket/29973#comment:12>
Django
-------------------------------------+-------------------------------------
| Siemens
Type: New feature | Status: assigned
Component: Core (Management | Version: 2.1
commands) |
Severity: Normal | Resolution:
Keywords: compilemessages, | Triage Stage: Accepted
ignore, translation, |
internationalization |
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 1 | UI/UX: 0
-------------------------------------+-------------------------------------
* needs_better_patch: 1 => 0
--
Ticket URL: <https://code.djangoproject.com/ticket/29973#comment:13>
Django
-------------------------------------+-------------------------------------
| Siemens
Type: New feature | Status: assigned
Component: Core (Management | Version: 2.1
commands) |
Severity: Normal | Resolution:
ignore, translation, | checkin
internationalization |
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
-------------------------------------+-------------------------------------
Changes (by Simon Charette):
* easy: 1 => 0
* stage: Accepted => Ready for checkin
Comment:
Except for the conflict in the 2.2 release notes that'll to be either
fixed or changed to target 3.0 the patch looks RFC to me.
--
Ticket URL: <https://code.djangoproject.com/ticket/29973#comment:14>
Django
-------------------------------------+-------------------------------------
Reporter: Elger Jonker | Owner: Ryan
| Siemens
Type: New feature | Status: assigned
Component: Core (Management | Version: 2.1
commands) |
Severity: Normal | Resolution:
Keywords: compilemessages, | Triage Stage: Ready for
ignore, translation, | checkin
internationalization |
Needs tests: 0 | Patch needs improvement: 0
-------------------------------------+-------------------------------------
--
Ticket URL: <https://code.djangoproject.com/ticket/29973#comment:15>
Django
-------------------------------------+-------------------------------------
Reporter: Elger Jonker | Owner: Ryan
| Siemens
Component: Core (Management | Version: 2.1
commands) |
Keywords: compilemessages, | Triage Stage: Ready for
ignore, translation, | checkin
internationalization |
Needs tests: 0 | Patch needs improvement: 0
-------------------------------------+-------------------------------------
* status: assigned => closed
* resolution: => fixed
Comment:
In [changeset:"a168e5616c85dfd658f20ffbb69a3872c2029dd3" a168e56]:
{{{
#!CommitTicketReference repository=""
revision="a168e5616c85dfd658f20ffbb69a3872c2029dd3"
Fixed #29973 -- Added compilemessages --ignore option.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/29973#comment:17>
Django
-------------------------------------+-------------------------------------
Reporter: Elger Jonker | Owner: Ryan
| Siemens
Component: Core (Management | Version: 2.1
commands) |
Severity: Normal | Resolution:
ignore, translation, | checkin
internationalization |
Needs tests: 0 | Patch needs improvement: 0
-------------------------------------+-------------------------------------
Comment (by Tim Graham <timograham@…>):
In [changeset:"bc9f0b32039325e96683e0e4a3ea84428a336c8d" bc9f0b3]:
{{{
#!CommitTicketReference repository=""
revision="bc9f0b32039325e96683e0e4a3ea84428a336c8d"
Refs #29973 -- Extracted helper functions from makemessages.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/29973#comment:16>