[Django] #31692: compilemessages needlessly runs msgfmt on unchanged .po files
Django
-------------------------------------+-------------------------------------
Reporter: cederlys | Owner: nobody
Type: | Status: new
Cleanup/optimization |
Component: Core | Version: master
(Management commands) |
Severity: Normal | Keywords:
Triage Stage: | Has patch: 0
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
I have a project where running `django-admin compilemessages` takes 1.75
seconds. Running it again, when all the `.mo` files already exists and are
up-to-date, also takes 1.75 seconds.
I propose that `compilemessages.py` is changed so that it only invokes
`msgfmt` when it would do anything useful. This can be implemented by
checking the mtime of the `.po` file and the corresponding `.mo` file. (If
statting the `.mo` file fails, treat that as if the mtime was 0.) Only
submit the command to the `executor` if the mtime of the `.po` file is
greater than that of the `.mo` file. In effect: don't do anything if the
`.mo` file is newer than the `.po` file.
There is one issue with this: the way the code currently uses the
`is_writable` function. Since it modifies the mtime of the `.mo` file, you
would have to perform the stat of the `.mo` file before you check if it is
writable. (Or, you could just remove the `is_writable` function and its
use. That feature is, in my opinion, of dubious value, and it doesn't
appear to be documented.)
After I made the changes above, the runtime in the common case where
nothing needs to be done was reduced from 1.75 seconds to 0.2 seconds.
(Unfortunately, I doubt that I will be able to get a Corporate Contributor
License Agreement signed, so I can unfortunately not contribute my
change.)
1.75 seconds may not be much, but when a CI system does it repeatedly, it
adds up.
--
Ticket URL: <https://code.djangoproject.com/ticket/31692>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
-------------------------------------+-------------------------------------
Type: | Status: new
Cleanup/optimization |
Component: Core (Management | Version: master
commands) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* stage: Unreviewed => Accepted
Comment:
Proposal makes sense.
About the CLA, I don't remember one time where we refused a contribution
because of that (should be time to drop it?).
--
Ticket URL: <https://code.djangoproject.com/ticket/31692#comment:1>
Django
-------------------------------------+-------------------------------------
Type: | Paroz
Cleanup/optimization | Status: assigned
Component: Core (Management | Version: master
commands) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* owner: nobody => Claude Paroz
* status: new => assigned
--
Ticket URL: <https://code.djangoproject.com/ticket/31692#comment:2>
Django
-------------------------------------+-------------------------------------
Type: | Paroz
Cleanup/optimization | Status: assigned
Component: Core (Management | Version: master
commands) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* has_patch: 0 => 1
Comment:
[https://github.com/django/django/pull/13085 PR]
--
Ticket URL: <https://code.djangoproject.com/ticket/31692#comment:3>
Django
-------------------------------------+-------------------------------------
Type: | Paroz
Cleanup/optimization | Status: assigned
Component: Core (Management | Version: master
commands) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Mariusz Felisiak <felisiak.mariusz@…>):
In [changeset:"ed0a040773f5bad187170ab4e3b094fe3108d702" ed0a040]:
{{{
#!CommitTicketReference repository=""
revision="ed0a040773f5bad187170ab4e3b094fe3108d702"
Refs #31692 -- Updated compilemessages and tests to use pathlib.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/31692#comment:4>
Django
-------------------------------------+-------------------------------------
Type: | Paroz
Component: Core (Management | Version: master
commands) |
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: assigned => closed
* resolution: => fixed
Comment:
In [changeset:"e62d55a4fe6555d18eebda638de402447618b500" e62d55a4]:
{{{
#!CommitTicketReference repository=""
revision="e62d55a4fe6555d18eebda638de402447618b500"
Fixed #31692 -- Prevented unneeded .po file compilation.
Thanks Nick Pope and Simon Charette for the reviews.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/31692#comment:5>
Django
-------------------------------------+-------------------------------------
Type: | Paroz
Cleanup/optimization | Status: closed
Component: Core (Management | Version: master
commands) |
Severity: Normal | Resolution: fixed
Keywords: | Triage Stage: Accepted
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by GitHub <noreply@…>):
In [changeset:"02ea98bc2ff972f7705bfffb24843494086d011f" 02ea98bc]:
{{{
#!CommitTicketReference repository=""
revision="02ea98bc2ff972f7705bfffb24843494086d011f"
Refs #31692 -- Fixed compilemessages crash on Windows with Python < 3.8.
Regression in ed0a040773f5bad187170ab4e3b094fe3108d702.
See https://bugs.python.org/issue31961
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/31692#comment:6>