Introducing Hotdot: project combining Django + Orbited + Twisted. Questions and Feedback

[Alex Clemesha]

Hi all,

I have a project going called "Hotdot" ( http://github.com/clemesha/hotdot ), which is
aimed at combining Django+Orbited+Twisted in the "best" possible way, with the
end goal being to make creating real world comet apps, in Python, easier to make.

Recent improvements in Twisted (specifically WSGI support, etc) and the increasing
interest in a "Comet Frameworks" (see Tornado http://www.tornadoweb.org) lead
me to believe that if Django+Orbited+Twisted were combined in an intelligent way,
we would have a "Comet Framework" that is more full-featured that Tornado
(because of the vibrant Django community, and the large scope of Twisted that goes
far beyond just HTTP).  Furthermore, if done right, this combination would *scale down*
very nicely, as well as *scale up*.  I think scaling down (such that is it easy
for people for getting started and run smaller sites with the defaults) is hugely important
for mindshare/adoption.

The code (here: http://github.com/clemesha/hotdot ) and is currently just a glorified
example, but has some very nice parts, as well as some crufty parts, and possibly
some parts that are "just plain wrong" from a Orbited/MorbidQ usage perspective,
and I'd love any feedback that anyone has to offer.

Some interesting code parts:

The "server", which starts everything (where everything is a Twisted service):
http://github.com/clemesha/hotdot/blob/master/server.py

Some overriding of MorbidQ defaults because of (believed) limitations in Morbid:
http://github.com/clemesha/hotdot/blob/master/realtime/stompfactory.py

And here is the Twisted cred checker that uses Django models to do auth:
http://github.com/clemesha/hotdot/blob/master/realtime/auth.py
(there is a limitation is MorbidQ such that I could not just "plug this in", see
the above comment and the file "stompfactory.py")

The intermeditary process than hands off Messages:
http://github.com/clemesha/hotdot/blob/master/realtime/restq.py#L69
To the "MESSAGE_HANDLERS"
http://github.com/clemesha/hotdot/blob/master/realtime/message_handlers.py
This is where all the Application (Message related) business logic lives, aka
the place where you do message modifying / filtering / logging / etc.


So anyways, the README file ( http://github.com/clemesha/hotdot/blob/master/README.rst )
shows how to get started and try it out, which is pretty easy. The first thing to do would
be create 2 different Users, login in as the 2 user from different browsers,
make a "Poll" and edit and vote from both browsers, and see the realtime behavior.
I'd greatly appreciate any feedback, criticism, or possible improvements.


thanks for your time, and Orbited!
-Alex



--
Alex Clemesha
clemesha.org

[skylar....@gmail.com]

Cool!

...

---

From: Alex Clemesha <clem...@gmail.com>

Date: Fri, 4 Dec 2009 15:01:05 -0800

To: <orbite...@googlegroups.com>

Subject: [Orbited] Introducing Hotdot: project combining Django + Orbited + Twisted. Questions and Feedback

--
You received this message because you are subscribed to the
Orbited discussion group.
To post, send email to
<orbite...@googlegroups.com>
To unsubscribe, send email to
<orbited-user...@googlegroups.com>
For more options, visit
<http://groups.google.com/group/orbited-users>

[desmaj]

Hi Alex,

On Dec 4, 6:01 pm, Alex Clemesha <cleme...@gmail.com> wrote:
> I have a project going called "Hotdot" (http://github.com/clemesha/hotdot), which is

> aimed at combining Django+Orbited+Twisted in the "best" possible way, with
> the
> end goal being to make creating real world comet apps, in Python, easier to
> make.

Nice job!

> very nicely, as well as *scale up*.  I think scaling down (such that is it
> easy
> for people for getting started and run smaller sites with the defaults) is
> hugely important
> for mindshare/adoption.

I think you're probably right, and I look forward to seeing what you
and the other Django folks can do with this. Good luck!

> Some overriding of MorbidQ defaults because of (believed) limitations in
> Morbid:http://github.com/clemesha/hotdot/blob/master/realtime/stompfactory.py
>
> And here is the Twisted cred checker that uses Django models to do auth:http://github.com/clemesha/hotdot/blob/master/realtime/auth.py
> (there is a limitation is MorbidQ such that I could not just "plug this in",
> see
> the above comment and the file "stompfactory.py")

Could you propose a patch to MorbidQ to address this limitation? It
would be great if you could plug your DatabaseChecker into MorbidQ
without having to work around anything.

Thanks,
Matthew

[Mario Balibrera]

Great work, Alex. I listed your project here: http://orbited.org/wiki/Sites .

> And here is the Twisted cred checker that uses Django models to do auth:http://github.com/clemesha/hotdot/blob/master/realtime/auth.py

> (there is a limitation is MorbidQ such that I could not just "plug this in",
> see
> the above comment and the file "stompfactory.py")

Could you propose a patch to MorbidQ to address this limitation? It
would be great if you could plug your DatabaseChecker into MorbidQ
without having to work around anything.

Good call, Matthew. Ken Whitesell would probably know the best way to integrate it into his auth module. Ken, thoughts? I've cc'ed the MorbidQ list. We should finish this subthread there. ;)

-mario 

[Alex Clemesha]

On Sat, Dec 5, 2009 at 5:03 PM, Mario Balibrera <mario.b...@gmail.com> wrote:

Great work, Alex. I listed your project here: http://orbited.org/wiki/Sites .

> And here is the Twisted cred checker that uses Django models to do auth:http://github.com/clemesha/hotdot/blob/master/realtime/auth.py

> (there is a limitation is MorbidQ such that I could not just "plug this in",
> see
> the above comment and the file "stompfactory.py")

Could you propose a patch to MorbidQ to address this limitation? It
would be great if you could plug your DatabaseChecker into MorbidQ
without having to work around anything.

Absolutely.
 

Good call, Matthew. Ken Whitesell would probably know the best way to integrate it into his auth module. Ken, thoughts? I've cc'ed the MorbidQ list. We should finish this subthread there. ;)

I'd appreciate any thoughts on the best way to plug this in, but whatever the final decision is, I'm happy to contribute a patch.


thanks,
Alex

--
Alex Clemesha
clemesha.org

[Alex Clemesha]

Hi,

I've found a couple small bugs in Orbited as result of
the work that I'm doing on Hotdot.

They effect "orbited/transports/base.py" and "orbited/cometsession.py",
here is the svn diff with respect to the orbited trunk:
http://gist.github.com/256582

Is there a more formal process you guys would like me
to post these fixes as? They are pretty small, so not sure if that is needed.


thanks,
Alex

--
Alex Clemesha
clemesha.org

[Mario Balibrera]

Hey Alex. Great work spotting a bug! It looks quite promising. For the sake of the project's long-term health, I encourage you to file a ticket here: http://orbited.org/wiki/Development#FilingaBug. The process is simple: create a ticket documenting the undesirable behavior; submit a patch, providing both your diff and a concise test case. The more we adopt this practice, the clearer we'll understand our code base a year from now, and the better our debugging toolkit will be. Besides, don't you want to go down in history as an Orbited Contributor (link: http://orbited.org/wiki/About#Contributors)?

You probably won't need them, but here are the official ticket guidelines: http://orbited.org/wiki/Bugs. Thank you for your past and future contributions to the community.

-mario

[Dmitry Amelchenko]

I'm trying to reverse proxy orbited with ssl and get "upstream prematurely closed connection while reading response header from upstream"

So, i have an nginx config that proxyes port 443 to orbited 8043
The orbited config looks good and seems to work, for now I have both of these ports open on the server, so if i connect to orbited on port 8043, it works, but port 443 gives the error.

Any specific suggestions/recommendations how to configure ssl proxying for orbited?

[Alex Clemesha]

On Dec 14, 8:59 pm, Mario Balibrera <mario.balibr...@gmail.com> wrote:
> Hey Alex. Great work spotting a bug! It looks quite promising. For the sake
> of the project's long-term health, I encourage you to file a ticket here:http://orbited.org/wiki/Development#FilingaBug. The process is simple:
> create a ticket documenting the undesirable behavior; submit a patch,
> providing both your diff and a concise test case. The more we adopt this
> practice, the clearer we'll understand our code base a year from now, and
> the better our debugging toolkit will be. Besides, don't you want to go down
> in history as an Orbited Contributor (link:http://orbited.org/wiki/About#Contributors)?
>
> You probably won't need them, but here are the official ticket guidelines:http://orbited.org/wiki/Bugs. Thank you for your past and future
> contributions to the community.

Alright, tickets have been created:
http://orbited.org/ticket/131
and:
http://orbited.org/ticket/132

The fixes, which are pretty trivial (all the work was in tracking them
down :) are included.
It would be very easy to add tests in for these bugs if some sort of
initial test harnesses
already existed for this bit of code. Are there any such tests? If
not, I'll most likely
just roll up my sleeves and do them myself, but I thought I would ask.

thanks,
Alex

>
> -mario

>
> On Mon, Dec 14, 2009 at 6:19 PM, Alex Clemesha <cleme...@gmail.com> wrote:
> > Hi,
>
> > I've found a couple small bugs in Orbited as result of
> > the work that I'm doing on Hotdot.
>
> > They effect "orbited/transports/base.py" and "orbited/cometsession.py",
> > here is the svn diff with respect to the orbited trunk:
> >http://gist.github.com/256582
>
> > Is there a more formal process you guys would like me
> > to post these fixes as? They are pretty small, so not sure if that is
> > needed.
>
> > thanks,
> > Alex
>

> > On Fri, Dec 4, 2009 at 3:01 PM, Alex Clemesha <cleme...@gmail.com> wrote:
> > > Hi all,
>
> > > I have a project going called "Hotdot" (
> >http://github.com/clemesha/hotdot
> > > ), which is
> > > aimed at combining Django+Orbited+Twisted in the "best" possible way,
> > with
> > > the
> > > end goal being to make creating real world comet apps, in Python, easier
> > to
> > > make.
>
> > > Recent improvements in Twisted (specifically WSGI support, etc) and the
> > > increasing

> > > interest in a "Comet Frameworks" (see Tornadohttp://www.tornadoweb.org)

> > > lead
> > > me to believe that if Django+Orbited+Twisted were combined in an
> > intelligent
> > > way,
> > > we would have a "Comet Framework" that is more full-featured that Tornado
> > > (because of the vibrant Django community, and the large scope of Twisted
> > > that goes
> > > far beyond just HTTP).  Furthermore, if done right, this combination
> > would
> > > *scale down*
> > > very nicely, as well as *scale up*.  I think scaling down (such that is
> > it
> > > easy
> > > for people for getting started and run smaller sites with the defaults)
> > is
> > > hugely important
> > > for mindshare/adoption.
>

> > > The code (here:http://github.com/clemesha/hotdot) and is currently

> > just a
> > > glorified
> > > example, but has some very nice parts, as well as some crufty parts, and
> > > possibly
> > > some parts that are "just plain wrong" from a Orbited/MorbidQ usage
> > > perspective,
> > > and I'd love any feedback that anyone has to offer.
>
> > > Some interesting code parts:
>
> > > The "server", which starts everything (where everything is a Twisted
> > > service):
> > >http://github.com/clemesha/hotdot/blob/master/server.py
>
> > > Some overriding of MorbidQ defaults because of (believed) limitations in
> > > Morbid:
> > >http://github.com/clemesha/hotdot/blob/master/realtime/stompfactory.py
>
> > > And here is the Twisted cred checker that uses Django models to do auth:
> > >http://github.com/clemesha/hotdot/blob/master/realtime/auth.py
> > > (there is a limitation is MorbidQ such that I could not just "plug this
> > in",
> > > see
> > > the above comment and the file "stompfactory.py")
>
> > > The intermeditary process than hands off Messages:
> > >http://github.com/clemesha/hotdot/blob/master/realtime/restq.py#L69
> > > To the "MESSAGE_HANDLERS"
>

> >http://github.com/clemesha/hotdot/blob/master/realtime/message_handle...

> >    <orbited-user...@googlegroups.com<orbited-users%2Bunsu...@googlegroups.com>

[sste...@gmail.com]

On Dec 21, 2009, at 5:04 AM, Alex Clemesha wrote:

>
>
> On Dec 14, 8:59 pm, Mario Balibrera <mario.balibr...@gmail.com> wrote:
>> Hey Alex. Great work spotting a bug! It looks quite promising. For the sake
>> of the project's long-term health, I encourage you to file a ticket here:http://orbited.org/wiki/Development#FilingaBug. The process is simple:
>> create a ticket documenting the undesirable behavior; submit a patch,
>> providing both your diff and a concise test case. The more we adopt this
>> practice, the clearer we'll understand our code base a year from now, and
>> the better our debugging toolkit will be. Besides, don't you want to go down
>> in history as an Orbited Contributor (link:http://orbited.org/wiki/About#Contributors)?
>>
>> You probably won't need them, but here are the official ticket guidelines:http://orbited.org/wiki/Bugs. Thank you for your past and future
>> contributions to the community.
>
> Alright, tickets have been created:
> http://orbited.org/ticket/131
> and:
> http://orbited.org/ticket/132
>
> The fixes, which are pretty trivial (all the work was in tracking them
> down :) are included.
> It would be very easy to add tests in for these bugs if some sort of
> initial test harnesses
> already existed for this bit of code. Are there any such tests? If
> not, I'll most likely
> just roll up my sleeves and do them myself, but I thought I would ask.

As far as I could tell, there weren't any tests at all yet, last time I looked. I have a few I've developed that I've been meaning to throw in there, when there's a there to throw them, as it were...

Steve