New external package - is there any duplication

[Dan Davis]

I am planning an external package to assist my developers in getting their special-purpose DDL out of the database and into git.   Some teams can handle it, but some teams could use help making sure that they use operations.RunSQL well and wisely.

My biggest questions about what I'm doing are:

* Are there any packages that already attempt to do what I'm thinking of?

* What gotchas have I not considered?

* What should it be called?   What should the management command be called?

* How do I test it well?

Here is a more detailed idea of the packages key contributions (https://github.com/danizen/django-sqlextras/):

* Create a migration operation that is a subclass of RunSQL, but takes a forward object that consists of a path and an expected checksum, and a similar optional backwards checksum.

* Create a management command that can build migrations using this operation (or you can do it yourself, of course).

Feedback is welcome.

[Adam Johnson]

Hi Dan,

I think this has not got much of a response because it might be a bit niche - it really depends where your teams are struggling to use RunSQL.

Looking for other packages at https://djangopackages.org/grids/g/database-migration/ , nothing seems to be related. They are all precursors/replacements for Django Migrations. Maybe there are other packages somewhere on Django Packages though.

Some responses to your ideas:

1. Rather than subclass RunSQL to load sql files, just embed SQL in Python with triple quoted strings? The checksumming would be unnecessary then, presuming team members understand that (applied) migration files are immutable.

2. You could "sync" a folder of (idemptotent) SQL files with a post_migrate signal handler. For example, Django syncs the Content Type table with such a handler: https://github.com/django/django/blob/master/django/contrib/contenttypes/apps.py#L20 . You could loop over all the SQL files in a given folder to check and run them. They could contain custom idempotent SQL such as "CREATE OR REPLACE FUNCTION" - presuming you're using RunSQL for DDL that Django Migrations doesn't support. I've seen this approach a couple times for things like stored procedures, though I'd prefer to use vanilla RunSQL in a migration myself. At least one can avoid yet another easily forgotten management command.

Thanks,

Adam

--
You received this message because you are subscribed to the Google Groups "Django developers (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-develop...@googlegroups.com.
To post to this group, send email to django-d...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-developers.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/73a56671-3904-4eef-b82a-d04fe175c993%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Adam

[Dan Davis]

Thank you for the review, Adam.

I like the idea of the post-migrate signal, but it can be complicated, and sometimes doing things in sequence is important:

With Oracle, if function B runs function A (and so on) then after we run:

CREATE OR REPLACE FUNCTION A(...) ....

We would need to, in Oracle anyway, run:

ALTER FUNCTION B COMPILE REUSE SETTINGS;

Hard then to be an idempotent change then, even with create and replace.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAMyDDM0FhWyZrgZRVw1i_OGQ6qMGu8xArL8xwxWZbghC%3Ds6bsg%40mail.gmail.com.

[Adam Johnson]

Oh dear, Oracle being awkward again! Maybe you could list and ten recompile all functions at the end of the handler code?

To view this discussion on the web visit https://groups.google.com/d/msgid/django-developers/CAFzonYaipXbb%2BD6YrCEZx4HDf%2BqqxNOEDykU3JdiP0TSOJiSfw%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--

Adam