How to design High Scalability architecture with Django in cost effective fashion

[Mihir Sevak]

Hi All,

      I am wondering if there are any books, articles or any other resources from where I can learn about how to create a highly scalable architecture using Django. We are talking about distributed systems and most likely one which is hosted on a third party cloud solution like AWS. That being the case cost effectiveness is a concern as well.

Several years ago I used to work with LexisNexis on their super computer with 6400 nodes. We had designed the system in a way that depending on load, nodes will be turned on and off automatically saving us bandwidth usage and electricity/power usage. I am envisioning similar architecture using Django. However, I have no clue how to put multiple database back end with same django frontend or what does middleware do or if we have 15 different django applications running to create a whole functional website how to distribute those 15 applications among nodes etc etc.

I am really a newbee in django architecture and because I don't know enough about django I don't know how to leverage distributed architecture and make django work with it.

If anyone can share any education material about how Django is architectured in order to build distributed systems effectively and reliably that would be absolutely wonderful. If there are any open source projects which I can take a look at and learn that would be great as well. 

Thanks.

Have a great day and wonderful weekend.

[Matthew Pava]

Hi Mihir,

Though I am no expert on distributed systems, you may want to check out the possible DATABASES settings.  You can specify multiple databases in your settings file.

See https://docs.djangoproject.com/en/1.10/topics/db/multi-db/

I would read about Django database routing to get started.  I believe there is a Django package out there that handles routing in a more convenient manner than out of the box; it is mentioned somewhere in this mailing list, but I can’t remember the name of it offhand.

Best wishes!

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.
To post to this group, send email to django...@googlegroups.com.
Visit this group at https://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/45e97ff1-4e6f-4bd8-a0b9-b62d6f42d533%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Nate Granatir]

I am also not even close to being an expert on this topic, but I do know that it's super easy to deploy a Django app on Heroku, which will get you scalable app deployment and Postgres dbs with very little upfront effort.

https://devcenter.heroku.com/articles/deploying-python

Nate

[Joe Landrigan]

Disclaimer: I've never even made a site in Django.  That said, I was looking at some of the same issues because of a project I'd like to do, and these looked interesting: https://github.com/JBKahn/django-sharding and https://github.com/mmalone/django-multidb