Swampdragon, the realtime framework for Django seems interesting. But will it scale ?

[Suren Sth]

I recently came across Swampdragon ( Visit official site). I am curious can it be used in production sites and will it scale ? If it can be, what is the best way?

[Kakar Nyori]

Oooh! Nice one. But its very new though. I would also like to hear from the experts!!! Thanks for the post though

On Wed, Nov 12, 2014 at 10:07 AM, Suren Sth <srn...@gmail.com> wrote:

I recently came across Swampdragon ( Visit official site). I am curious can it be used in production sites and will it scale ? If it can be, what is the best way?

--
You received this message because you are subscribed to the Google Groups "Django users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to django-users...@googlegroups.com.
To post to this group, send email to django...@googlegroups.com.
Visit this group at http://groups.google.com/group/django-users.
To view this discussion on the web visit https://groups.google.com/d/msgid/django-users/74092e59-801e-48f6-ba99-9d6231516b84%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[hags...@gmail.com]

You should be able to load balance SwampDragon (though I haven't actually tried this out yet).

I ran some simple tests with 300 connections to a small linode instance, where I subscribed over 300 users (I think it was in the realms of 500 but I don't have the results anymore).

I had each subscribed connection push a message (so each connection sent one message and received as many messages as there were connections) and it held up fine.

The tests are probably rather inconclusive, but it was pretty quick and there was no problems.

It was also #4 on Hacker news a few days ago and it held up under the load. There were over 6,000 users (I think it was close to 6,400 users) across that day playing around with the chat demo.

I honestly doubt that 6,000 users using it at the same time would hold up though (it is a small server).

But let's say you build a site with SwampDragon, and you get on average 200 users using it per hour, and that would be consistent across a whole day (you would have peaks and dips of course), that would be 4,800 users a day. If you get 4,800 users a day you are doing pretty well.

[Aaron Reabow]

the thing i am trying to figure out is how this works with my existing setup.

I get that you need to install redis. but how does the rest of the stack change

mine is through the good people at webfaction:

apache tomcat

nginx (static)

postgres

python2.7

django1,6

html/css/js

how much of this can i leave alone, while installing redis?

[Shazwi Suwandi]

Do the demos on the main site work for any of you?

[Aaron Reabow]

Work on his site (yes)

[Cal Leeming]

This actually looks like an interesting framework, and I'd like to start off with the good points.

The author (Jonas) has very kindly shared his work the community, and I really do applaud the effort he has put into this. SPAs (single page applications) are becoming much more common, and frameworks like this help raise awareness about why they are so awesome. Using websockets can help in many ways, it can greatly reduce the overhead of each request by reducing the total req/resp size and increase async throughput by pipelining potentially thousands of queries per second on the same connection (using pub/sub channels). It also lets you think outside the box in terms of API design, if you use a guaranteed pub/sub delivery mechanism then recovering from broken connection state becomes super easy.

This project does have good intentions, but the question of "can it be used in production" doesn't just come down to performance, and upon code inspection I do have some concerns;

* Lack of maturity, initial commit was March 2014, not many closed/opened issues, the author has not made it clear how long this has been in dev for.

* Reasonable amount of downloads on pypi and "stars" on github

* Code is not up to PEP8 standards

* No public CI testing

* JS does not have modular structure, does not have a sane design pattern, and does not adhere to require/AMD

* JS has some unit tests, but they don't appear to be functional or recently updated. On first glance, I'd say coverage is minimal and that TDD probably hasn't been followed.

* API implementation looks a bit wonky

* Deeply integrated with Django, rather than a modular fashion which could then be used as standalone or with other frameworks/ORMs.

For my own taste/use cases, I would say this is lacks the maturity and stability to be considered production ready. This project feels to me like a "prototype draft", something that could be used for a hobby project, bleeding edge experimentation, prototype inspiration etc. If this came up for internal code review at our work, it wouldn't pass.

But again - huge kudos to the author for what he's done so far, and I really hope he continues to improve on it.

tl;dr - It's feels pretty alpha, YMMV :)

Cal

On Wed, Nov 12, 2014 at 4:37 AM, Suren Sth <srn...@gmail.com> wrote:

I recently came across Swampdragon ( Visit official site). I am curious can it be used in production sites and will it scale ? If it can be, what is the best way?

--

[jonas hagstedt]

You are right about pep8, there were two pep8 violations in there (fixed that after reading this).

*  There is a CI setup with Codeship. After reading this I made the badge available on Github (good point).

*  It is deeply integrated with Django by design.

*  I am reviewing the JavaScript, but it works so there is nothing stopping anyone from using it (but you are right about the JS).

*  I honestly don't remember the exact date I started developing SwampDragon but I don't think it was long before the initial commit.

I am interested in knowing what it is about the API implementation that looks wonky. 

It's always good with fresh eyes on a project, so I would love to hear more details on this.

It's been good to read these comments as you tend to get tunnel vision when you work on something after a while.

None of these points are actually qualifying if you should use it in production or not (don't get me wrong, the points are all good).

Should you use this in production?:

*  Does it perform well enough?

*  Is it secure enough?

and these two questions depends a lot on your code base / project requirements.

I would love to hear more about the wonky API though if you wouldn't mind.

Cheers

[Cal Leeming]

Thanks for the reply, I was almost sure there were more PEP8 violations, line length and such, but perhaps my eyes failed me so I'll recheck that. 

And sure I'd be more than happy to go into detail/specifics on my reasoning about the API, I've got two other promised reviews in the queue (Django itself and DRF) so it'll be a couple of weeks before I can spend some time on this.

I completely agree on the tunnel vision, I've worked on projects in the past where I was so sure it was the right way, and then others came along and absolutely destroyed my code with their review (and they were right to do so). Code reviews can be a bit demoralizing sometimes, especially when you have vested a lot of time/effort into them, hence why I tried to focus on the positives more than the negatives.

Cal

[jonas hagstedt]

I ignore E501 as I find it a silly rule, but if I don't then there will be pep8 issues a plenty, so that would explain it.

Thanks for taking the time to looking at the code though. 

Next version will have some JS rewrite (hopefully without breaking backwards compatibility)