Re: [Django] #11927: Allow manage.py dumpdata to dump YAML in block style
Django
--------------------------------------+------------------------------------
Reporter: sampablokuper | Owner:
Type: New feature | Status: new
Component: Core (Serialization) | Version: SVN
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 1
Needs tests: 1 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
--------------------------------------+------------------------------------
Changes (by aviraldg):
* owner: aviraldg =>
* status: reopened => new
--
Ticket URL: <https://code.djangoproject.com/ticket/11927#comment:14>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
Django
--------------------------------------+------------------------------------
Reporter: sampablokuper | Owner:
Type: New feature | Status: new
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 1
Needs tests: 1 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
--------------------------------------+------------------------------------
Comment (by blockstyle@…):
--blockstyle option not found?!! I'm dumping my fixtures as yaml. The
flowstyle is a pita to see.
Why isn't it available? I'm using Django 1.6
--
Ticket URL: <https://code.djangoproject.com/ticket/11927#comment:15>
Django
--------------------------------------+------------------------------------
Reporter: sampablokuper | Owner:
Type: New feature | Status: new
Component: Core (Serialization) | Version: master
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 1
Needs tests: 1 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
--------------------------------------+------------------------------------
Comment (by charettes):
This ticket patch needs improvement based on @russellm comments, tests and
documentation.
--
Ticket URL: <https://code.djangoproject.com/ticket/11927#comment:16>
Django
Reporter: sampablokuper | Owner: xiaohanyu
Type: New feature | Status: assigned
Component: Core | Version: master
(Serialization) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 1
Needs tests: 1 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
Changes (by xiaohanyu):
* owner: => xiaohanyu
* status: new => assigned
--
Ticket URL: <https://code.djangoproject.com/ticket/11927#comment:17>
Django
-------------------------------------+-------------------------------------
| Hanyu
Type: New feature | Status: assigned
Component: Core | Version: master
(Serialization) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 1
Needs tests: 1 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* cc: tehfink (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/11927#comment:18>
Django
-------------------------------------+-------------------------------------
| Kooijman
Type: New feature | Status: assigned
commands) |
Severity: Normal | Resolution:
Keywords: | Triage Stage:
Has patch: 1 | Needs documentation: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* cc: Matthijs Kooijman (added)
* needs_better_patch: 1 => 0
* component: Core (Serialization) => Core (Management commands)
* needs_tests: 1 => 0
* owner: Xiao Hanyu => Matthijs Kooijman
* stage: Accepted => Unreviewed
Comment:
I've taken the time time to redo this patch and address the comments on
the previous patch. The patch now also adds tests and documentation.
The PyYAML docs say the following:
> By default, PyYAML chooses the style of a collection depending on
whether it has nested collections. If a collection has nested collections,
it will be assigned the block style. Otherwise it will have the flow
style.
>
> If you want collections to be always serialized in the block style, set
the parameter default_flow_style of dump() to False. For instance,
In other words, this option configures the style for collections, and
there are three options: Always use the flow style, always use the block
style, or the default of automatically selecting. To match that, I've
added a `--yaml-collection-style` option that takes `auto`, `flow` or
`block` option.
Some uncertainties / notes when reviewing the patch:
- The option name is a lot more verbose than the `--style` option Russel
[[https://code.djangoproject.com/ticket/11927#comment:10|proposed]].
However, since it is only relevant to the yaml formatter, I've made the
option name more verbose.
- The code in django/core/management/commands/dumpdata.py only passes the
option on to the serializer when the current formatter is 'yaml'. This
might be a bit hardcoded, but I didn't see an easy and elegant way to do
this otherwise?
- I've left the default at "`auto`" as before, but I suspect that a
default of "`block`" might be more appropriate for loaddata. The current
mixed style likely produces more user surprise that the more verbose block
style would, as well as being more appropriate for version control. Would
it make sense to change the default? If so, should the default of the
serializer class be changed as well, or just the dumpdata command?
Given this is a new feature, I've targeted the docs at 2.2.
The patch is available on github:
https://github.com/django/django/pull/10117
--
Ticket URL: <https://code.djangoproject.com/ticket/11927#comment:19>
Django
-------------------------------------+-------------------------------------
Reporter: sampablokuper | Owner: Matthijs
| Kooijman
Type: New feature | Status: assigned
Component: Core (Management | Version: master
commands) |
Severity: Normal | Resolution:
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* needs_docs: 1 => 0
* stage: Unreviewed => Accepted
--
Ticket URL: <https://code.djangoproject.com/ticket/11927#comment:20>
Django
-------------------------------------+-------------------------------------
Reporter: sampablokuper | Owner: Matthijs
| Kooijman
Type: New feature | Status: assigned
Component: Core (Management | Version: master
commands) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Changes (by Tim Graham):
* needs_better_patch: 0 => 1
--
Ticket URL: <https://code.djangoproject.com/ticket/11927#comment:21>
Django
-------------------------------------+-------------------------------------
Reporter: sampablokuper | Owner: Matthijs
| Kooijman
Type: New feature | Status: assigned
Component: Core (Management | Version: master
commands) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Matthijs Kooijman):
Meanwhile, on github Tim Graham has reviewed the patch and suggested to
generalize the concept of format-specific commandline options, rather than
hardcoding it as I had done. I've implemented this, but ran into some
problems with the testing framework. See the PR for comments and further
discussion.
Meanwhile, I also tried changing the default style from the current mixed
style, to always using the block style (which I think is a better style
for dumping model data). However, I noticed that this style also changes
plain lists, not just mappings. E.g., a model with a many2manyfield ends
up being serialized in block style too, which might not be a good default.
For example the categories field in:
{{{
- model: serializers.article
pk: 1
fields:
author: 2
headline: Poker has no place on ESPN
pub_date: 2006-06-16 11:00:00
categories:
- 3
- 1
meta_data: []
}}}
Better would be to use the flow style for such simple lists:
{{{
categories: [3, 1]
}}}
My current patch has three styles, 'flow' (everything flow style, single
line), 'block' (everything block style, like above) and 'mixed' (the
PyYAML default with flow style for containers with scalars and block style
for containers with other containers.
I can imagine a fourth style, which uses block for all mappings, flow
style for sequences with scalars and block style for sequences with
containers. This is is essentially the PyYAML default style, except that
mappings always use the block style. A quick test shows this is easy to
implement by overriding the `represent_mapping()` method on
`DjangSafeDumper` to force `flow_style=False`. Not sure how to name this
style, though. It could also be described as "mixed", but I do not want to
drop the PyYAML default style. The latter could be renamed to "default" or
even "pyyaml-default", or perhaps this new style could be more explicitly
called "block-mappings" to indicate it uses block style for mappings
(without indicating the rest).
Any other thoughts?
--
Ticket URL: <https://code.djangoproject.com/ticket/11927#comment:22>
Django
-------------------------------------+-------------------------------------
Reporter: sampablokuper | Owner: Matthijs
| Kooijman
Type: New feature | Status: assigned
Component: Core (Management | Version: master
commands) |
Severity: Normal | Resolution:
Keywords: | Triage Stage: Accepted
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* cc: Florian Demmer (added)
--
Ticket URL: <https://code.djangoproject.com/ticket/11927#comment:23>
Django
-------------------------------------+-------------------------------------
Reporter: sampablokuper | Owner: Matthijs
| Kooijman
Component: Core (Management | Version: dev
commands) |
Severity: Normal | Resolution: wontfix
Keywords: | Triage Stage:
| Unreviewed
Has patch: 1 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 1
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
* status: assigned => closed
* resolution: => wontfix
* stage: Accepted => Unreviewed
Comment:
Given the difficulty of progressing this, and the amount of time it's sat
untouched, I can't see that adding pass-through CLI options for
serialisers is a good way forward.
Rather, I'd think that creating a serialiser subclass, encapsulating your
project options and using the existing
[https://docs.djangoproject.com/en/4.0/ref/settings/#serialization-modules
SERIALIZATION_MODULES] setting is the way forward.
--
Ticket URL: <https://code.djangoproject.com/ticket/11927#comment:24>