Postgres specific fields and third party apps.

[Tom Christie]

I'm not sure if there'll be an actionable outcome of this, but wanted to raise it for discussion.

We've started supporting some of the PostgreSQL specific fields in Django REST framework, and I ran into a roadblock when adding a new testcase.

Unsurprisingly we're not able to test them without also switching the test database to PostgreSQL.

As a third party app that's not something I want to have to do - it'd mean a bunch of work, an extra dependency, slower tests, and (unacceptablly) more complication for our contributors.

The only way I can see to resolve it would be if we had support for those fields with the sqlite backend.

Technically I assume that's feasible - use JSON encoding for storing the data (probably easy), and do awfully non-optimal in-memory filtering for the field-specific lookups (probably very hard).

There'd also be the obstacle requiring someone with the time and motivation to make it happen.

Options I can see:

* The status quo - don't support fields these in SQLite. (I assume this is by far the most likely.)

* Support the fields in SQLite, but don't support the lookups. (This'd actually hit the sweet spot for me - I can still test against these fields, and test cases will run just fine, so long as you're not hitting any field specific fitlering in querysets.)

* Support the fields and lookups in SQLite. (Sounds highly unlikely.)

Any thoughts?

Cheers!

   Tom

Aside: I'm not interested in the "you should be testing with the same database as you use in production" gumpf here. It's a third party app.

[Marc Tamlyn]

I would strongly disagree with supporting these on SQLite, in any format. A fully supported version would be complex to support and enormously inefficient, and I'm not in favour of Django shipping half baked features with major holes in them. I certainly don't have the motivation to build this.

Mark any relevant tests to be skipped unless you're using postgres, and run postgres as well on your CI. While it's perfectly reasonable for your developers to not be needing the extra complication of installing additional databases etc, I see no reason why you shouldn't be checking everything works on the main production databases. This is similar to the contribution story for Django.

Given that your travis builds currently run in under 40 seconds, I don't think their speed on postgres will be problematic for your pull requests etc.

I would probably have the motivation to create a PR for DRF showing a possible travis/testing setup for what I have proposed.

Marc

Aside: In my opinion, a third party app that does significant database work should be testing with the same database(s) as its users are likely to use in production.

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/f08447af-395c-4fb0-9f01-d2af3b4b8504%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Tom Christie]

Thanks Marc, that's helpful input.

> I would probably have the motivation to create a PR for DRF showing a possible travis/testing setup for what I have proposed.

No, don't worry about that. We actually don't have any need to *hit* the database in those tests at all, we just need to make sure that the model fields get correctly mapped to the equivalent serializer fields. There will be mocking-type approaches to those tests that mean less work than switching the CI database.