Using AWS Redshift database Django 1.6

[jpk]

Greetings,

I'm attempting to set up a django project using amazon redshift as the database backend.  I started a fresh 1.6.5 project and set up the database portion of settings.py to use the redshift cluster. With just the default apps (admin, auth, contenttypes, sessions, messages, and staticfiles), I ran syncdb and got this: https://dpaste.de/y98N.  Compared to vanilla postgre, redshift doesn't support some things, including some constraints, which is what's making syncdb blow up.

What worries me in that backtrace is, the table creation sql that has the offending constraint in it is being generated in commands/syncdb.py, so that's before we even get to the postgresql_psycopg2 backend code, right?  So hacking on the postgres backend to make it jive with redshift wouldn't help, here.  (Correct me if I'm wrong!)

Anyway, what wisdom can you share?  What's the best way to approach this?  Is trying to get this to work even a good idea?  Any guidance would be greatly appreciated.


Thanks!
jpk

--
john p. kiffmeyer

[C. Kirby]

Wow, redshift lets you save quite a bit of data.
It looks like someone started in on a redshift backend, I found something on github (https://github.com/binarydud/django-redshift)
It appears to be unmaintained, and maybe not even complete, but you might get some idea's of where work needs to be done from it.

Kirby

[jpk]

Yeah, I came across that as well, but it didn't work on 1.6.  I cloned and debugged it up until I got the exact same error as the paste in my first message. :(

So, a few questions:

• Am I correct in my analysis that the syncdb command alone is generating the table creation sql that includes the constraint redshift is barfing on?
• If I were to patch the syncdb command code, is it reasonable to consult the database backend for settings related to which constraints are allowed/not?
• Does it make more sense for the database backend code to do the table creation instead of syncdb?  (As in, db backends provide might an abstracted create table method that syncdb could just call instead of building its own sql.)

I know more than I did a day ago, but I'm still not super hip on how db backends are implemented and how the rest of the framework interacts with them.  So if the stuff I'm asking doesn't make sense then do put me on the right path. :)

Thanks!

jpk

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.
To post to this group, send email to django...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/8bd4e6b5-42c5-4779-86af-cb04b708540d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
john p. kiffmeyer

[C. Kirby]

I'll answer what I can from a quick review of the codebase.

On Thursday, May 29, 2014 12:36:39 PM UTC-5, jpk wrote:

Yeah, I came across that as well, but it didn't work on 1.6.  I cloned and debugged it up until I got the exact same error as the paste in my first message. :(

So, a few questions:

• Am I correct in my analysis that the syncdb command alone is generating the table creation sql that includes the constraint redshift is barfing on?

No, syncdb (or in 1.7+ migrate) eventually calls the correct db backend to generate the sql. You can see from the code that there is no SQL there, just calls to the backend sql generation (In particular look at line 96)

• If I were to patch the syncdb command code, is it reasonable to consult the database backend for settings related to which constraints are allowed/not?

I would not change the sync code. A better path would probably be to start with the postgres backend and make the changes necessary for redshift (I assume that is what the django-redshift project started to do)

• Does it make more sense for the database backend code to do the table creation instead of syncdb?  (As in, db backends provide might an abstracted create table method that syncdb could just call instead of building its own sql.)

I know more than I did a day ago, but I'm still not super hip on how db backends are implemented and how the rest of the framework interacts with them.  So if the stuff I'm asking doesn't make sense then do put me on the right path. :)

Thanks!

jpk

 
Kirby

[jpk]

Ah, I see!  I was mistaken, then.  So the base db backend's sql_create_model is what generates the create table with the constraint?  Then a db backend subclass just needs to override that to do whatever dbms-specific stuff it wants?  Is there documentation on what all the methods in a django.db.backends module are supposed to do?  (The docstrings in the code seem decent enough, just wondering if there's anything more detailed than that.)

Thanks!

jpk

--

You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.
To post to this group, send email to django...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.

To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/5a094392-088f-4062-b85a-2a0a24bf192b%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
john p. kiffmeyer

[Varsha Mane]

Hi

Anyone knows how to integrate,https://github.com/binarydud/django-redshift i Django 1.6 based project..

Thanks in Advance..