New app - architecture

75 views
Skip to first unread message

Radek Svarz

unread,
Oct 18, 2014, 10:01:09 AM10/18/14
to django...@googlegroups.com
Hi,

just imagine you have a chance to create a new application - on the greenfield (hurray! :)

The application would be Software as a Service ala Github, Bitbucket, JIRA, Mailchimp, Zendesk, Freshdesk

I.e. friendly, direct frontend + some background tasks (initiated both by cron and users) + API for webservices integrations.

Users are from corporates and SMEs. App has designed architecture for ideal zero downtime.

App would have both URL separated parts and SPA (Single Application parts for modules) via # in URL. UI should behave like Mailchimp. 

Due to the continuous development the app would have configurable feature flipping (for both FrontEnd, and BackEnd, might also have for webservice APIs).

Usage is on PC browser (IE9+ and others - yes corporates:( ), also for browsers in tablets (Android Chrome, Ipad Safari).

# How would you design the app architecture TODAY? #
-------------------------------------------------------------------------------

Browser Front End (please answer at least this):

  UI for interactive special graphs: D3JS (we have tested several others - conclusion we must use it)

  UI other: bootstrap / kendo UI / dojo / jquery UI / ... ?

  JS framework: angularJS / backbone.js / ember ? (btw, I prefer declarative over algorithmic coding)


Other::::::(back end,...)::::::::::::::::::

BE API for browser FE: just pure django views or anything else?

BE API for web services integrations: TastyPie? (API will have to be versioned, I would like to have the auto documentation, too)

Cache: reddit? memcached?

BE - app server: Django (for sure :), Celery / Rabit MQ?

Authentication: (it is not a social app, therefore the own login is acceptable. Eventually there will be needed some sort of SSO from corporate environment - how to do that?) + eventually python-social-auth for public SSO (e.g. for SMEs, who use GApps?)

Search: Whoosh? (there will be searchable cca 100 000 records, every record is ala github ticket. Nevertheless search should form some sort of group-based selection wizard, therefore I do not like the Haystack limitations. Solr etc. seams to be overkill now.

BE - DB: PostgreSQL?

Deployment: Fabric?

Supporting: Sentry, New Relic?

BE autotests: ???

FE autotests: selenium?

CDN: google, maxcdn, other? (for JS, fonts etc.)


Overall constraint - we love Django, so we want to build it on top of it. :)


We have the vision that the application would be there for a long term period. Therefore we need ingredients which are maintainable.

Please provide the structural answers :)

Thanks,

Radek

Avraham Serour

unread,
Oct 19, 2014, 8:06:55 AM10/19/14
to django...@googlegroups.com
 UI other: bootstrap / kendo UI / dojo / jquery UI / ... ?
whatever you feel more comfortable, get something working first, but I wouldn't use jqueryUI and wouldn't pay for kendo at least to begin, after I got something working and the situation proves I need kendo I would buy it.

JS framework: angularJS / backbone.js / ember ? (btw, I prefer declarative over algorithmic coding)
If you need to start learning something I would choose angularjs, but if you are already familiar or prefer something just use what you like

BE API for browser FE: just pure django views or anything else?
what other option would you have other than django views?

BE API for web services integrations: TastyPie? (API will have to be versioned, I would like to have the auto documentation, too)
I like tastypie, but use it only if you actually need it

Cache: reddit? memcached?
how would you use reddit for cache?

BE - app server: Django (for sure :), Celery / Rabit MQ?
It seems you are confusing the things here, none of them are application servers
django is a framework, the application itself would be using it
to serve the application I would use nginx+uwsgi, nginx is a webserver, uwsgi is the application server

Authentication: (it is not a social app, therefore the own login is acceptable. Eventually there will be needed some sort of SSO from corporate environment - how to do that?) + eventually python-social-auth for public SSO (e.g. for SMEs, who use GApps?)
So don't worry for now, worry then, there are django apps for that, you can plug additional auth backends later or even write your own

Search: Whoosh? (there will be searchable cca 100 000 records, every record is ala github ticket. Nevertheless search should form some sort of group-based selection wizard, therefore I do not like the Haystack limitations. Solr etc. seams to be overkill now.
My suggestions here is to create a stage server, populate and test with some of the most popular, it looks like you already ruled out haystack
In any case this shouldn't influence on the alpha version, use the DB for search for now, while you are only making the prototype after you start seeing performance problems start looking for a dedicated engine, remember the first 3 rules on optimizing

BE - DB: PostgreSQL?
I like postgres, I strongly recommend it

Deployment: Fabric?
yes, but on the other hand this doesn't have much meaning, fabric just makes easier automating ssh commands, it doesn't provide recipes for deployment, you still have to plan this

Supporting: Sentry, New Relic?
what kind of support are you talking about?

BE autotests: ???
>FE autotests: selenium?
unit tests are very nice, but if you have no experience on it you may waste too much time learning this instead of making the product itself, ideally you should have them before making a release, but make an alpha version first and start thinking about them during beta

CDN: google, maxcdn, other? (for JS, fonts etc.)
just serve them from your server, if you see performance problems after releasing the first version you can consider them, and test.
this depends on your users location, before using CDN's make the content cachable, in any case refer to the first 3 rules on optimizing

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.
To post to this group, send email to django...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/fd22c91e-0474-4266-af39-2740ef3d2370%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Scot Hacker

unread,
Oct 19, 2014, 3:37:48 PM10/19/14
to django...@googlegroups.com

All good advice from Avraham Serour - just a couple of additional comments:


On Sunday, October 19, 2014 1:06:55 AM UTC-7, Avraham Serour wrote:


BE API for web services integrations: TastyPie? (API will have to be versioned, I would like to have the auto documentation, too)
I like tastypie, but use it only if you actually need it

Django REST Framework eclipses the functionality and ease of use of Tastypie, IMO, and has become the de facto standard for Django API generation. Its documentation is stellar and it is quite actively developed. I'm hoping that, now that migrations are in core, a discussion about integrating API generation into core is a next big topic for Django core (this is an area where Django lags behind Rails).

 
Deployment: Fabric?
yes, but on the other hand this doesn't have much meaning, fabric just makes easier automating ssh commands, it doesn't provide recipes for deployment, you still have to plan this

Fabric is awesome, but as you move past the "automating ssh commands" into "we need a real build system," check out Ansible.

./s
 

Radek Svarz

unread,
Oct 19, 2014, 3:52:55 PM10/19/14
to django...@googlegroups.com
@Avraham - thanks a lot

To clarify some of my mistakes. Overall we have our 1st version prototype done. Now I would like to change it to the better architecture.

>>>what other option would you have other than django views?
API JSON responses made views - ala Tastypie  for SPA parts

>>> how would you use reddit for cache?

>>> BE - app server: Django (for sure :), Celery / Rabit MQ?
>It seems you are confusing the things here, none of them are application servers
>django is a framework, the application itself would be using it
t>o serve the application I would use nginx+uwsgi, nginx is a webserver, uwsgi is the application server

Correct, I just abstracted that into one box ;) So if I split it:

framework: django :)
app server: uwsgi
http server: nginx (also depending on the PaaS env. we will use)

bkg tasks:  Celery or Rabit MQ?

>>> SSO + open auth.
>>> So don't worry for now, worry then, there are django apps for that, you can plug additional auth backends later or even write your own
It will come sooner than later (cca 2 months to demonstrate).

>>> Search - remember the first 3 rules on optimizing
Do you mean 1) do not optimize 2) do not optimize 3) do not optimize yet ? :)

Btw, notes from other sources: 

Deployment Ansible - seams to be more flexible than Fabric.
Local tasks directly in Django mng. commands, FE tasks via Grunt.
Tests in Pytest with Django dynamic fixtures. 
Splinter as a frontend for Selenium. Karma runner for running tests written in Jasmine.
E2E tests in Protractor.

Radek
Reply all
Reply to author
Forward
0 new messages