Socket.IO and Clojure?

[Sean Corfield]

At work we're starting down the path of building a new piece of
functionality based on WebSockets and the external team we're working
with is a Node.js shop so their go to solution is Socket.IO and
they've produced a very nice front end in CoffeeScript and a prototype
back end on Node.js.

I'd really like to have our back end on the JVM, of course, and so I'd
like to find a JVM-based Socket.IO solution that I use from/with
Clojure...

This seems like a reasonable option:

https://github.com/mrniko/netty-socketio

A little bit of experimentation with lein-try (Thank you Ryan!) shows
that it's pretty easy to get a basic server up and running in the REPL
- and I was able to get several of their demos running unchanged
against Clojure, instead of their Java applications, so that was
promising.

Are there other folks out there doing Socket.IO stuff with Clojure?
What approaches have you taken?

Obviously, we could run Node.js and have it hit a Clojure-based REST
API to do the integration, and that might be less pain long term
but...
--
Sean A Corfield -- (904) 302-SEAN
An Architect's View -- http://corfield.org/
World Singles, LLC. -- http://worldsingles.com/

"Perfection is the enemy of the good."
-- Gustave Flaubert, French realist novelist (1821-1880)

[Anand Prakash]

Hi Sean,

We are in process for going with a similar solution on clojure. Lets keep sharing notes.

We were on heroku till last week. Websockets wont work on heroku. Lack of sticky session caused other issues. So we moved to elabstic beanstalk. However elastic beanstalk doesn't support jetty and I do not like tomcat. Plus on elastic beanstalk whenever you push code they have a down time upto 5 minutes.

So over the past 2 days, we moved things to chef on ec2 which I am really happy with. Now we have a system as easy to use as heroku, with much more control. I am mentioning these because you or others might go through the same steps.

Coming to your main point about Socket.IO. We spent around a week exploring Socket.IO. It was super easy to hack up a demo, however we ended up leaving it for couple of reasons:

1. Though the community looks very active in terms of forks etc, the development on Socket.IO is completely stopped. The dev team is focussed on engine.IO and they are very opaque. When you post on socket.io google group, posts do not have get approved. If you propose to help, you do not get any response.

2. The library has lot of features - like rooms etc. However they are not built to scale. I looked at the code and was very disappointed. User->Room mapping, Room->User mapping were stored in memory which will make it not scale beyond a point. Bad coding (e.g. using lists traversal where hash maps should be used) is very common in nodejs community and I am very wary of developing on that platform.

3. There was a memory leak in Socket.IO which would cause memory to increase linearly with usage. Given that most projects using nodejs dont move beyond prototypes these kind of issues do not get highlighted. I used to manage the mobile server at LinkedIn and we had to deal with a lot of memory leaks with node.js.

4. Javascript

So finally, if you really have to use Socket.IO, I would recommend using https://github.com/einaros/ws for just websocket and built everything else yourselves.

I am not a big fan of keeping two different setups at the stage of our product. We are fine with long-poll for now. So we will start there and eventually get to web sockets. We are considering http://http-kit.org/ for that. For communication between message generators and open client sockets we are exploring some pub sub solution.

Hope this helps.

Thanks

Anand

[Angel Java Lopez]

If you leave Socket.IO, another option is direct WebSocket in Java

https://github.com/TooTallNate/Java-WebSocket

http://java-websocket.org/

--
--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Sean Corfield]

Just to clarify, we don't _want_ to use Socket.IO on Node.js but
combined with the client-side library it has a lot of benefits
(multiple transports, automatic fallback, good cross-browser support,
namespaces and rooms match our problem domain). The server component
is sufficiently complex that we don't want to replicate all of that
functionality ourselves.

We don't need huge scalability (at least, not in foreseeable future)
so the in-memory aspect of Node.js's implementation isn't really a
concern if we have to fallback on it and expose our services via REST
to it. We'd rather have the whole server side on the JVM however.

We can't use direct WebSockets because browser support is not good
enough (and won't be for our target market for a long time I suspect).

That's why we are looking at netty-socketio since it seems to support
the Socket.IO client "out of the box" and is reasonably easy to use
from Clojure.

The reasons stated above are also why we can't just use http-kit
(which was my first thought) - we need more machinery than just
WebSockets.

I had found this by Chris Granger -
https://github.com/ibdknox/clj-socketio - but it's outdated and when I
reached out to him, he confirmed the first point above - that the
server side of Socket.IO is too complex to warrant trying to replicate
ourselves (and that Socket.IO had evolved sufficiently that his
project wouldn't be a useful starting point either).

Sean

> --
> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clo...@googlegroups.com
> Note that posts from new members are moderated - please be patient with your
> first post.
> To unsubscribe from this group, send email to
> clojure+u...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups
> "Clojure" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to clojure+u...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

[Clifton King]

My company has been using with Socket.IO for async communication in production for a little while, and while it worked great in private beta, it has serious performance issues (as well as memory leaks) with only ~ 500 concurrent users. Ouch. The trouble with Socket.IO is that it tries to do too much. It doesn't live at the right level of abstraction.

The ideal solution would be a networking layer with proper fallbacks (websocket -> flashsocket -> xhr -> jsonp, et al) that handles client handshaking and *nothing else.* After dealing with a lot of issues around a load-balanced socket.io server [1], I believe the better library to utilize would be SockJS [2]. SockJS has a spec, a solid client library and some server implementations (netty version in beta, vertx supposedly works out of the box).

I haven't used SockJS yet (it will require a few small server/client libraries to do proper multiplexing), but it's likely the direction we're going to be taking since the situation with SocketIO appears to be pretty dire as of late. Our software usage really kicks up 2-3x during the fall (higher-ed focused company), so we'll be forced to move sooner rather than later.

What would be really-really awesome would be a core.async + http-kit powered library. Just throwing that out there :P

[1] https://github.com/learnboost/socket.io/issues/438

[2] http://sockjs.org

Shameless plug: if you enjoy problems like these and would consider moving to Dallas to solve them, reach out to me: clifton [at] orgsync [dot] com

[Hoàng Minh Thắng]

I had the same concern and have searched for solutions. IMO sockjs is well designed and perfectly fits my needs.
You can use sockjs with Clojure, though we'll need more tweaks to make it pass all the sockjs specs:
https://github.com/jenshaase/methojure/tree/master/methojure-sockjs
If you really want to work with sockjs and Clojure, invest on that.

Personally I'm developing Chlorinejs to fill the gap Clojurescript left.
Here's a demo using sockjs in both client and server (nodejs)
https://github.com/chlorinejs-demos/angular-sockjs-im

[c...@bitemyapp.com]

If you want something robust I'd recommend something like http-kit or Netty on the backend + using websockets (and a shim) directly on the frontend, or possibly browserchannel if you need to worry about firewalls and ancient browsers.

Socket.IO is a moving target intended for a single audience (not multiplatform). It's not going to win you more than you'll pay in bugs and performance issues in the long run.

On Tuesday, July 16, 2013 10:07:34 PM UTC-7, Sean Corfield wrote:

[Christopher Martin]

Shameless plug: I recently shared a Clojure library for WebSockets and HTTP Kit: http://cljwamp.us 

It provides some features similar to Socket.IO in regards to pubsub/multiplexing events, and uses the WAMP spec which has several multi-platform options: http://wamp.ws/implementations

While clj-wamp does not support long polling, a separate solution could potentially run along side it (since HTTP Kit itself supports long polling) though I haven't tried this yet.

This library probably isn't a good option for Sean (the OP), but might be of interest for those looking into WebSockets with HTTP Kit.

Cheers,

~Christopher Martin

[Anand Prakash]

This is an awesome discussion. Hope this leads to some good frameworks in the community.

I was looking at Socket.IO webpage and they seem to support the following:

Websocket, Adobe Flash Socket, AJAX long polling, AJAX multipart streaming, Forever Iframe, JSONP polling.

I understand some people are not comfortable with Websockets, because if the early stage of this technology.

So there are two time of use-cases:

a. pseudo real time - e.g. chat apps - they should all be fine with polling and long poll is an optimization.

b. real-time - e.g. multi player games, stock ticker etc - they would need some kind of socket.

To answer Sean's question - I understand there are benefits with the socket.io client library. However you said that you do not want to go for Websockets. So if that is the case, isn't AJAX long polling sufficient for you. If that is the case then no client side library is required. You just need to keep polling from the client. It's server's choice with it wants return right away or implement long-poll.

I am new to this space - so just try to make sure if I am missing something obvious.

Thanks

Anand

[Anand Prakash]

BTW one more thing. If I am building anything serious. Anything which might be user facing and could potentially be used by even thousands of users, I would never consider Socket.IO. I just feel that it's a super hack which tries to do too many things with no guarantees on anything.

[Ustun Ozgur]

A bit off-topic, but in some situations where bidirectional communication is 

not necessary, one might consider EventSource instead of Websockets 

to push data from server to the client. For some reason, EventSource is not 

as well known as Websockets.

It has some advantages like being more backwards compatible.

http://www.html5rocks.com/en/tutorials/eventsource/basics/

http://stackoverflow.com/questions/5195452/websockets-vs-server-sent-events-eventsource

https://github.com/Yaffle/EventSource

Ustun