Combining streaming and long polling

[Makoto]

Hi, everyone.

This is my first post to node.js group and I am very excited about the
potential of node.js and real time web.

As an experimentation to learn more about node.js, I am currently
working on "tail web app" which simply tails error log on web server
(http://github.com/makoto/log_watcher/).

Unlike normal "web chat system" which keeps waiting until someone says
anything,
my tail server keeps outputting the tail result to browser forever,
which is more like streaming model which has been discussed in node.js
threads a few times before.

My initial trial was almost copying & pasting the example found on
the jsconf.eu slide like this.

http://github.com/makoto/log_watcher/blob/20bf6038ad53eb6e4c72737aaa9b2cb87477f251/watcher.js

This actually works ,but browser eventually times out, which is not
good.

so, I tried to combine the above method with ajax long polling(eg:
server calls res.finish after 10 sec, and client reconnects
immediately after using ajax) like below, but they do not seem working
either.

http://github.com/makoto/log_watcher/blob/ede1e21b90aff5b2c945cf8367979969c124ff94/watcher.js
http://github.com/makoto/log_watcher/blob/2fc3138c350d101d6e356d4820becc5244871b57/index.html

I assume this is because ajax callback fires only when the entire
response ends(by doing res.final()), not when new data from tail is
pushed into response.

I also looked int streaming file upload example (http://debuggable.com/
posts/streaming-file-uploads-with-node-js:4ac094b2-b6c8-4a7f-
bd07-28accbdd56cb),
but I don't think this is relevant caz I am not using multipart file
upload (correct me if I am wrong)

Is my approach of combining streaming and log-polling correct
approach? Should I rather use js.io or HTML5 WebSockets(which are not
really implemented in any major browsers) for this? Any suggestions
are welcome.

Thanks.

Makoto

[Irakli Gozalishvili]

--
Irakli Gozalishvili
Web: http://rfobic.wordpress.com/
Phone: +31 614 205275
Address: Taksteeg 3 - 4, 1012PB Amsterdam, Netherlands

WebSocket is implemented in Latest Chromium and Webkit for the Firefox there is a working patch but it's ot integrated to the tree yet, hopefully it will be in for 3.6. There is also shim using flash for other browsers http://github.com/gimite/web-socket-js. I have narwhal based implementation for the server side. http://github.com/Gozala/narwhal-xulrunner/blob/experimental/examples/websocket.js Having it on node would be great, actually it's on my todo list but I have too much stuff on my plate now...



 

Thanks.

Makoto

--

You received this message because you are subscribed to the Google Groups "nodejs" group.
To post to this group, send email to nod...@googlegroups.com.
To unsubscribe from this group, send email to nodejs+un...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/nodejs?hl=en.

[Makoto]

Hi, Irakli.

Thank you for your info.
I tried the combination of Chromium & http://github.com/alexanderte/websocket-server-node.js
and worked fine.

The code I ended up writing is so simple. I am very impressed by
websocket-server-node.js
http://gist.github.com/247741

I also tried web-socket-js and this seems working fine, too. I just
need websocket-server-node.js to return flash policy file which I
haven't figured out how to. Once I set the policy file, I should be
able to run WebSocket on normal browser (using flash though).

Thanks.

Makoto

On Dec 1, 11:07 am, Irakli Gozalishvili <rfo...@gmail.com> wrote:
> --
> Irakli Gozalishvili
> Web:http://rfobic.wordpress.com/
> Phone: +31 614 205275
> Address: Taksteeg 3 - 4, 1012PB Amsterdam, Netherlands
>
>
>
>
>
> On Tue, Dec 1, 2009 at 02:22, Makoto <inoue...@googlemail.com> wrote:
> > Hi, everyone.
>
> > This is my first post to node.js group and I am very excited about the
> > potential of node.js and real time web.
>
> > As an experimentation to learn more about node.js, I am currently
> > working on "tail web app" which simply tails error log on web server
> > (http://github.com/makoto/log_watcher/).
>
> > Unlike normal "web chat system" which keeps waiting until someone says
> > anything,
> > my tail server keeps outputting the tail result to browser forever,
> > which is more like streaming model which has been discussed in node.js
> > threads a few times before.
>
> > My initial trial was  almost copying & pasting the example found on
> > the jsconf.eu slide like this.
>

> >http://github.com/makoto/log_watcher/blob/20bf6038ad53eb6e4c72737aaa9...

>
> > This actually works ,but browser eventually times out, which is not
> > good.
>
> > so, I tried to combine the above method with ajax long polling(eg:
> > server calls res.finish after 10 sec, and client reconnects
> > immediately after using ajax) like below, but they do not seem working
> > either.
>

> >http://github.com/makoto/log_watcher/blob/ede1e21b90aff5b2c945cf83679...
>
> >http://github.com/makoto/log_watcher/blob/2fc3138c350d101d6e356d4820b...

>
> > I assume this is because ajax callback fires only when the entire
> > response ends(by doing res.final()), not when new data from tail is
> > pushed into response.
>
> > I also looked int streaming file upload example (http://debuggable.com/
> > posts/streaming-file-uploads-with-node-js:4ac094b2-b6c8-4a7f-

> > bd07-28accbdd56cb<http://debuggable.com/%0Aposts/streaming-file-uploads-with-node-js:4a...>
> > ),

> > but I don't think this is relevant caz I am not using multipart file
> > upload (correct me if I am wrong)
>
> > Is my approach of combining streaming and log-polling correct
> > approach? Should I rather use js.io or HTML5 WebSockets(which are not
> > really implemented in any major browsers) for this? Any suggestions
> > are welcome.
>
> WebSocket is implemented in Latest Chromium and Webkit for the Firefox there
> is a working patch but it's ot integrated to the tree yet, hopefully it will

> be in for 3.6. There is also shim using flash for other browsershttp://github.com/gimite/web-socket-js. I have narwhal based implementation
> for the server side.http://github.com/Gozala/narwhal-xulrunner/blob/experimental/examples...

> it on node would be great, actually it's on my todo list but I have
> too much stuff on my plate now...
>
>
>
> > Thanks.
>
> > Makoto
>
> > --
>
> > You received this message because you are subscribed to the Google Groups
> > "nodejs" group.
> > To post to this group, send email to nod...@googlegroups.com.
> > To unsubscribe from this group, send email to

> > nodejs+un...@googlegroups.com<nodejs%2Bunsu...@googlegroups.com>
> > .

[Irakli Gozalishvili]

On Thu, Dec 3, 2009 at 00:53, Makoto <inou...@googlemail.com> wrote:

Hi, Irakli.

Thank you for your info.
I tried the combination of Chromium & http://github.com/alexanderte/websocket-server-node.js
and worked fine.

Cool I did not knew about this one http://github.com/alexanderte/websocket-server-node.js
 

The code I ended up writing is so simple. I am very impressed by
websocket-server-node.js
http://gist.github.com/247741

I also tried web-socket-js and this seems working fine, too. I just
need websocket-server-node.js to return flash policy file which I
haven't figured out how to. Once I set the policy file, I should be
able to run WebSocket on normal browser (using flash though).

For flash policy you can give a look on my code probably most of it can be reused

http://github.com/Gozala/narwhal-xulrunner/blob/experimental/lib/websocket-server.js#L317

Cheers!
 

To unsubscribe from this group, send email to nodejs+un...@googlegroups.com.

[Makoto]

Hi, Irakli.

I managed to change websocket-server-node.js to return flash policy
file, though probably I don't need it if I can serve the client html
from node.js directly. Thank you for your info.

Anyway, my "tail app" turned into "iostat" app which looks exactly
like OSX Activity Monitor, but as web app.

Here is the blog post about my experimentation. Hope some people find
it useful info.

http://blog.new-bamboo.co.uk/2009/12/7/real-time-online-activity-monitor-example-with-node-js-and-websocket

Makoto

On Dec 3, 1:19 am, Irakli Gozalishvili <rfo...@gmail.com> wrote:

> On Thu, Dec 3, 2009 at 00:53, Makoto <inoue...@googlemail.com> wrote:
> > Hi, Irakli.
>
> > Thank you for your info.
> > I tried the combination of Chromium &
> >http://github.com/alexanderte/websocket-server-node.js
> > and worked fine.
>

> Cool I did not knew about this onehttp://github.com/alexanderte/websocket-server-node.js

>
> > The code I ended up writing is so simple. I am very impressed by
> > websocket-server-node.js
> >http://gist.github.com/247741
>
> > I also tried web-socket-js and this seems working fine, too. I just
> > need websocket-server-node.js to return flash policy file which I
> > haven't figured out how to. Once I set the policy file, I should be
> > able to run WebSocket on normal browser (using flash though).
>
> For flash policy you can give a look on my code probably most of it can be
> reused
>

> http://github.com/Gozala/narwhal-xulrunner/blob/experimental/lib/webs...

> > <nodejs%2Bunsu...@googlegroups.com<nodejs%252Bunsubscribe@googlegroups. com>