[Django] #29026: Make makemigrations scriptable / script-friendly
Django
-------------------------------------+-------------------------------------
Reporter: Chris | Owner: nobody
Jerdonek |
Type: | Status: new
Cleanup/optimization |
Component: | Version: master
Migrations | Keywords:
Severity: Normal | makemigrations,scripting,stderr,stdout
Triage Stage: | Has patch: 0
Unreviewed |
Needs documentation: 0 | Needs tests: 0
Patch needs improvement: 0 | Easy pickings: 0
UI/UX: 0 |
-------------------------------------+-------------------------------------
Currently, the `makemigrations` management command doesn't lend itself
well to scripting. For example, it writes its progress output to `stdout`
rather than `stderr`. Also, there doesn't appear to be a structured /
programmatic way to figure out what files it has created.
My use case is that in my development environment, I'd like to be able to
run `makemigrations` in a Docker container, find out what files were added
(e.g. from `makemigrations`'s output), and then copy those files from the
Docker container to my development machine so they can be added to source
control.
Currently, there doesn't seem to be an easy way to do this. One way, for
example, is to manually read `makemigrations`'s output to find out what
apps were affected, and then inspect the directories yourself for the new
files.
Better, for example, would be if `makemigrations` could write the paths to
the created files to `stdout`.
--
Ticket URL: <https://code.djangoproject.com/ticket/29026>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
-------------------------------------+-------------------------------------
Type: | Status: new
Cleanup/optimization |
Component: Migrations | Version: master
Severity: Normal | Resolution:
Keywords: | Triage Stage:
makemigrations,scripting,stderr,stdout| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by astandley):
The current makemigrations command would seem to allow for everything you
have asked.
{{{
>> python -m manage makemigrations
Migrations for 'newspaper':
newspaper\migrations\0001_initial.py
- Create model People
>> python -m manage makemigrations
Migrations for 'newspaper':
newspaper\migrations\0001_initial.py
- Create model People
}}}
Addressing each point:
Output written to stdout rather than stderr.
You can redirect stdout to stderr for the execution of the command if you
want the output in stderr. (Using sys in python, or pipes for a shell
script)
A structured / programmatic way to figure out what files it has
created.
Run a regex on the output of the migration command. Example pattern:
{{{r'Migrations for '(?P<app_name>[^']*)':\n
(?P<migration_file>[^\n]*)'}}}
Is there a reason this would not meet your needs?
--
Ticket URL: <https://code.djangoproject.com/ticket/29026#comment:1>
Django
-------------------------------------+-------------------------------------
Type: | Status: new
Cleanup/optimization |
Component: Migrations | Version: master
Severity: Normal | Resolution:
Keywords: | Triage Stage:
makemigrations,scripting,stderr,stdout| Unreviewed
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Chris Jerdonek):
I don't consider parsing log output with regexes to be structured output
or a programmatic API. That seems brittle and unnecessarily complicated,
and also would be hard for Django to make backwards compatibility
guarantees about.
What I had in mind was something like log output going to stderr, and the
paths of the created files going to stdout -- one per line. If something
fancier was needed, json could be outputted. With something like that,
there wouldn't be any need for regex parsing and the API would be well-
defined.
--
Ticket URL: <https://code.djangoproject.com/ticket/29026#comment:2>
Django
-------------------------------------+-------------------------------------
Type: | Status: new
Cleanup/optimization |
Component: Migrations | Version: master
Severity: Normal | Resolution:
Keywords: | Triage Stage:
makemigrations,scripting,stderr,stdout| Unreviewed
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Tim Graham):
It seems complicated. For example, what if `makemigrations` requires
interactive input from the questioner?
--
Ticket URL: <https://code.djangoproject.com/ticket/29026#comment:3>
Django
-------------------------------------+-------------------------------------
Type: | Status: new
Cleanup/optimization |
Component: Migrations | Version: master
Severity: Normal | Resolution:
Keywords: | Triage Stage:
makemigrations,scripting,stderr,stdout| Unreviewed
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Chris Jerdonek):
My original use case was a non-interactive one (inside a container). But
again, you raise a good point. Obviously, piping to `stdout` won't work if
interactivity is required (because you'd want user prompts to go to
`stdout`). This is true of any script, not just Django management
commands. Other than that, I don't think the changes I've described would
''hurt'' things in that case, aside from possibly the "reliability" issue
you mentioned [https://code.djangoproject.com/ticket/29040#comment:3
here]. That though could be addressed by my
[https://code.djangoproject.com/ticket/29040#comment:4 follow-up comment]
to yours. If we wanted a fancier solution, the "structured" `stdout` could
be outputted only in non-interactive mode.
--
Ticket URL: <https://code.djangoproject.com/ticket/29026#comment:4>
Django
-------------------------------------+-------------------------------------
Type: | Status: new
Cleanup/optimization |
Component: Migrations | Version: master
Severity: Normal | Resolution:
makemigrations,scripting,stderr,stdout|
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* stage: Unreviewed => Accepted
Comment:
I'm skeptical, but I guess if you have a patch to propose, we can evaluate
it.
--
Ticket URL: <https://code.djangoproject.com/ticket/29026#comment:5>
Django
-------------------------------------+-------------------------------------
Type: | Status: new
Cleanup/optimization |
Component: Migrations | Version: master
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
makemigrations,scripting,stderr,stdout|
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Tim Graham):
I closed #29470 as a duplicate.
--
Ticket URL: <https://code.djangoproject.com/ticket/29026#comment:6>
Django
-------------------------------------+-------------------------------------
Type: | Status: new
Cleanup/optimization |
Component: Migrations | Version: master
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
makemigrations,scripting,stderr,stdout|
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Amir Hadi):
If you are developing with Docker, why are you not just mounting your
development machines source directory into the container, execute
`makemigrations` and then you have the migrations directly on your
machine. This way you can save yourself from parsing anything.
--
Ticket URL: <https://code.djangoproject.com/ticket/29026#comment:7>
Django
-------------------------------------+-------------------------------------
Type: | Walls
Cleanup/optimization | Status: assigned
Component: Migrations | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
makemigrations,scripting,stderr,stdout|
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* owner: nobody => Jacob Walls
* status: new => assigned
--
Ticket URL: <https://code.djangoproject.com/ticket/29026#comment:8>
Django
-------------------------------------+-------------------------------------
Type: | Walls
Cleanup/optimization | Status: assigned
Component: Migrations | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
makemigrations,scripting,stderr,stdout|
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* has_patch: 0 => 1
--
Ticket URL: <https://code.djangoproject.com/ticket/29026#comment:9>
Django
-------------------------------------+-------------------------------------
Type: | Walls
Cleanup/optimization | Status: assigned
Component: Migrations | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
makemigrations,scripting,stderr,stdout|
Needs tests: 1 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* needs_tests: 0 => 1
Comment:
Until I remedy the test failure on Windows.
--
Ticket URL: <https://code.djangoproject.com/ticket/29026#comment:10>
Django
-------------------------------------+-------------------------------------
Type: | Walls
Cleanup/optimization | Status: assigned
Component: Migrations | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
makemigrations,scripting,stderr,stdout|
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* needs_better_patch: 0 => 1
* needs_tests: 1 => 0
Comment:
I'm going to take a look at implementing Chris's suggested `log()` method.
(See PR)
--
Ticket URL: <https://code.djangoproject.com/ticket/29026#comment:11>
Django
-------------------------------------+-------------------------------------
Type: | Walls
Cleanup/optimization | Status: assigned
Component: Migrations | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
makemigrations,scripting,stderr,stdout|
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* needs_better_patch: 1 => 0
--
Ticket URL: <https://code.djangoproject.com/ticket/29026#comment:12>
Django
-------------------------------------+-------------------------------------
Type: | Walls
Cleanup/optimization | Status: assigned
Component: Migrations | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
makemigrations,scripting,stderr,stdout|
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* needs_better_patch: 0 => 1
--
Ticket URL: <https://code.djangoproject.com/ticket/29026#comment:13>
Django
-------------------------------------+-------------------------------------
Type: | Walls
Cleanup/optimization | Status: assigned
Component: Migrations | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
makemigrations,scripting,stderr,stdout|
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* needs_better_patch: 1 => 0
Comment:
Up to date w/r/t feedback but for commit reorganization and suggestion to
put the new log() method in a standalone PR, which I am willing to do!
--
Ticket URL: <https://code.djangoproject.com/ticket/29026#comment:14>
Django
-------------------------------------+-------------------------------------
Type: | Walls
Cleanup/optimization | Status: assigned
Component: Migrations | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
makemigrations,scripting,stderr,stdout|
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Jacob Walls):
Got around to reorganizing the commits and opening a first-step
[https://github.com/django/django/pull/14936 refactoring PR to add log()]
per Chris's suggestion.
If for whatever reason this is not desired, by all means, close the
smaller PR and we can just work on one branch. Cheers.
--
Ticket URL: <https://code.djangoproject.com/ticket/29026#comment:15>
Django
-------------------------------------+-------------------------------------
Type: | Walls
Cleanup/optimization | Status: assigned
Component: Migrations | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
makemigrations,scripting,stderr,stdout|
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Mariusz Felisiak <felisiak.mariusz@…>):
In [changeset:"f153e9214f17cb3bc1f7046a48ff3f563e88483b" f153e92]:
{{{
#!CommitTicketReference repository=""
revision="f153e9214f17cb3bc1f7046a48ff3f563e88483b"
Refs #29026 -- Added log() to makemigrations.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/29026#comment:16>
Django
-------------------------------------+-------------------------------------
| Walls
Type: New feature | Status: assigned
Component: Migrations | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
makemigrations,scripting,stderr,stdout|
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* type: Cleanup/optimization => New feature
--
Ticket URL: <https://code.djangoproject.com/ticket/29026#comment:17>
Django
-------------------------------------+-------------------------------------
| Walls
Type: New feature | Status: assigned
Component: Migrations | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
makemigrations,scripting,stderr,stdout|
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* needs_better_patch: 0 => 1
--
Ticket URL: <https://code.djangoproject.com/ticket/29026#comment:18>
Django
-------------------------------------+-------------------------------------
| Walls
Type: New feature | Status: assigned
Component: Migrations | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
makemigrations,scripting,stderr,stdout|
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* needs_better_patch: 1 => 0
--
Ticket URL: <https://code.djangoproject.com/ticket/29026#comment:19>
Django
-------------------------------------+-------------------------------------
| Walls
Type: New feature | Status: assigned
Component: Migrations | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
makemigrations,scripting,stderr,stdout|
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Mariusz Felisiak <felisiak.mariusz@…>):
In [changeset:"0ab58c120939093fea90822f376e1866fc714d1f" 0ab58c12]:
{{{
#!CommitTicketReference repository=""
revision="0ab58c120939093fea90822f376e1866fc714d1f"
Refs #29026 -- Allowed customizing InteractiveMigrationQuestioner's prompt
destination.
Previously, the questioner did not obey the value of stdout provided
to the command.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/29026#comment:20>
Django
-------------------------------------+-------------------------------------
| Walls
Type: New feature | Status: assigned
Component: Migrations | Version: dev
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
makemigrations,scripting,stderr,stdout|
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* needs_better_patch: 0 => 1
--
Ticket URL: <https://code.djangoproject.com/ticket/29026#comment:21>
Django
-------------------------------------+-------------------------------------
| Walls
Type: New feature | Status: assigned
Component: Migrations | Version: dev
Severity: Normal | Resolution:
makemigrations,scripting,stderr,stdout| checkin
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* needs_better_patch: 1 => 0
* stage: Accepted => Ready for checkin
--
Ticket URL: <https://code.djangoproject.com/ticket/29026#comment:22>
Django
-------------------------------------+-------------------------------------
| Walls
Component: Migrations | Version: dev
Severity: Normal | Resolution: fixed
Keywords: | Triage Stage: Ready for
makemigrations,scripting,stderr,stdout| checkin
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: assigned => closed
* resolution: => fixed
Comment:
In [changeset:"6f78cb6b13fca2512f80dcd2bce7838bf33b70f0" 6f78cb6b]:
{{{
#!CommitTicketReference repository=""
revision="6f78cb6b13fca2512f80dcd2bce7838bf33b70f0"
Fixed #29026 -- Added --scriptable option to makemigrations.
}}}
--
Ticket URL: <https://code.djangoproject.com/ticket/29026#comment:23>