[django.contrib.postgres] Would this be accepted and how hard would it be to do?

[Some Developer]

Hi,

I'm a long term Django user but have never contributed to the main
Django code base mainly because I haven't really needed any extra
features. Until now.

I only use PostgreSQL as my database and the recent addition of
django.contrib.postgresql got me thinking about other features of
PostgreSQL that could be added to the Django code base that would be
very useful for me and probably others.

Those features are PostgreSQLs built in support for the XML data type.
These could be supported in Django with the addition of an XMLField.
Because PostgreSQL supports querying the XML data directly some
additions would have to be made to support these operations in the ORM
(I think at least).

I've just checked out the latest Django source code and notice that
there is a jsonb.py file in the django.contrib.postgres.fields package
which I assume will support JSON in PostgreSQL in version 1.9. I'd like
to do the same thing but with the XML data type.

Is this a project that a) would be accepted b) would be fairly straight
forward and c) would be useful to the wider community? JSON is great and
all but XML still has its uses in the world (I'm one of those people who
prefer JSON but XML just offers more for the project that I am working on).

So thoughts?

[Josh Smeaton]

I'm not quite up to date on XML in databases (I've never had to use XML in a database), but don't the majority of core backends support XML in some way? If so, I'd probably like to see an attempt to build out XML support such that all/most core databases could benefit from the new feature, rather than being locked away in contrib.postgres.

So, to answer your questions:

Is this a project that; 

a) would be accepted - Probably, if someone were to implement it and drive it until quality was acceptable

b) would be fairly straight forward and - I'm not sure how straightforward it would be. The addition of Lookups and Transforms will help tremendously though.

c) would be useful to the wider community? - Not for me personally, but I'm sure there'd be others that would find good use for XML support. Hopefully we can get some more feedback to accurately determine that though.

Cheers,

[Russell Keith-Magee]

On Mon, Jun 29, 2015 at 7:32 AM, Josh Smeaton <josh.s...@gmail.com> wrote:

I'm not quite up to date on XML in databases (I've never had to use XML in a database), but don't the majority of core backends support XML in some way? If so, I'd probably like to see an attempt to build out XML support such that all/most core databases could benefit from the new feature, rather than being locked away in contrib.postgres.

 

Heh... 

https://github.com/django/django/blob/stable/1.3.x/django/db/models/fields/__init__.py#L1135

So, to answer your questions:

Is this a project that; 

a) would be accepted - Probably, if someone were to implement it and drive it until quality was acceptable

b) would be fairly straight forward and - I'm not sure how straightforward it would be. The addition of Lookups and Transforms will help tremendously though.

c) would be useful to the wider community? - Not for me personally, but I'm sure there'd be others that would find good use for XML support. Hopefully we can get some more feedback to accurately determine that though.

Broadly, I agree with Josh. 

Given that we have contrib.postgres, we should be aiming at 100% coverage of PostgreSQL features in core (not saying that is possible or likely, but it's an aspirational goal). 

In cases where it is plausible to implement a feature for all backends, we should aim to do that. 

However, I don't think we should be cross platform at all costs. It would be possible to implement XMLField as a light layer over TextField (hey - we used to do exactly this), but if that's the only way to implement it, I don't think we should implement it at all. I'm not in favour of implementing known-to-be-slow design patterns for core features. Unless SQLite and MySQL offer a native XML data type, with reasonably rich query syntax, then I don't think a cross-platform is desirable.

Yours,

Russ Magee %-)

[Collin Anderson]

I think XMLField was removed because the security of xml parsing and validating is hard to get right.

[Marc Tamlyn]

In that case the ability to support it would depend on the support in psycopg2 and how safe we consider their implementation.

Marc

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/72777a63-a357-44ba-a067-34a7a4af1b1f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Russell Keith-Magee]

Correct - our XMLField was just a TextField with our own parsing on top of it, so given all the DDOS problems related to "billion laughs" etc, it was just too dangerous to include any more.

As Marc notes, in this case, we'd be looking at an implementation of XMLField that is provided by someone else; it's up to PostgreSQL to determine the risks and mitigate them. 

Yours,

Russ Magee %-)