SocketStream 0.3 update

[Owen B]

Hi all

I gave a presentation at KrtConf on Tuesday detailing the new features and ideas going into SocketStream 0.3. In summary these are:

* A complete rewrite of 0.2 to make the core more modular to encourage contributions and 3rd party addons. This is the top priority!

* New custom websocket responders. We will ship with Events and RPC as per 0.2, but you will easily be able to make your own responders for high speed gaming and models. Rather than trying to 'solve' models for you, this approach allows multiple ideas to bloom - allowing us all to figure out what works and support it in the future.

* New Client Asset Manager. Allows you to serve different single page clients to different devices, depending upon URLs and User Agent. We will also support a greater variety of templating libs.

* Websocket Middleware, works like Connect middleware allowing you to handle logging and authentication before any of the RPC functions are called

* Transport Agnostic. I demoed on stage a simple app working in SocketIO then instantly switching to Pusher Pipe by changing one line of config in your app, instantly enabling front-end scaling and the ability to deploy to Heroku. As the transport layer is also modular it will be easy to use Sock JS and other libraries in the future without changing any of your application code. This is a killer feature and unique to SocketStream

I will continue working on the 0.3 code this month and hope to get something on github within 3-4 weeks for people to comment on and play with. Expect the first proper 0.3 release by the end of the year. In the mean time we will continue to refine 0.2, including support for Node 0.6 when Connect 2.0 is stable.

I will also post a link to the video of the presentation in this thread once it's available.

If anyone would like to help in anyway with SocketStream 0.3 please get in touch. We've already had a great contribution from Tadek who has written an independent module which loads a directory into an 'API Tree' and is now included within the 0.3 core. We've also got a great new logo for SocketStream 0.3 which has been kindly contributed by Craig Jordan Muir, the creator of the excellent NodeUp podcast.

Thanks to everyone for the support and the kind comments after the presentation. Really looking forward to getting 0.3 to you soon.

Owen

[Jana]

Hi Owen,

Cool stuff and I can't wait to watch your krtconf presentation
video! :-)

Btw, wondering whether or not SocketStream 0.3 will support that
latest node v0.6.0 which is awesome!

Thanks,
Jana

.

[alia...@tradermail.info]

<3 SS.3

[Owen B]

Thanks guys

Been working hard on 0.3 this week. So much still to think through but it's shaping up nicely.

I know a few of you have been asking about Node 0.6 compatibility with SocketStream 0.2. I'd very much like to support this once Connect 2.0 is officially released and on npm. This likely means SocketStream 0.2.6 will targeted at Node 0.6 only.

Just wondered if anyone is interested in working on this (using the current version of Connect 2.0 on github) whilst I focus purely on 0.3? Feel free to send me a private message on github anytime you need help.

Owen

[Tadeusz Wójcik]

Can't wait too see video from krtconf and v0.3!

[Owen B]

Hi guys

Taken a break from 0.3 development today to release SocketStream 0.2.6 for Node 0.6. No other major changes.

Your apps should work without modification - just make sure you pass the full absolute path to @session.authenticate() as require.paths is no longer present.

Owen

[Owen B]

To all those eagerly awaiting SocketStream 0.3, first of all an apology: I've decided against putting a preview of the code onto Github just yet.

I've been working hard on the new release almost every day since I came back from KrtConf.com last month and I'm very excited about the progress so far; but getting it right is a big big deal and there are several important design ideas which are just not there yet.

So I'm going to use the rest of December, including the long period I have away from the office over the holidays, to continue to refine the code and ideas, plus write new documentation to what is effectively a brand new product.

My goal now is to release a feature-complete working version of 0.3 (an alpha release) by the end of the year. This will not be a 0.2 replacement overnight, and it will still have plenty of rough edges, but it will allow you to start writing apps and experiment with the new API.

Given 0.3 is a big change from 0.2, there will be parts you like and parts you don't. Rest assured nothing is set in stone. Based upon your feedback and suggestions, I will work with other contributors to refine the design until I'm happy with it, before releasing 0.3.0 and retiring 0.2.

2012 will be the year when everyone and his dog is launching a new realtime single-page framework. I want to make very sure we have made the best design decisions we can to give developers the confidence they need to choose SocketStream.

Thanks for your support,

Owen

PS: I'm still waiting for the 0.3 presentation I gave at KrtConf to be uploaded, but I'll let you know when it is.

[Dirk Dressel]

No problem. Good to hear you put so much effort into it. Its better to
start right.

So we are just waiting for a late christmas this year!

[Tim Reha]

If anyone is interested in Socketstream and real-time web/mobile apps in Seattle connect with me to setup a meetup. I have a big loft and additional studio space with live stream capabilities to host an event.  Cheers, Tim

[Owen B]

Thanks Tim for the kind offer. I was in Seattle a few months ago and I love the city. Sure to return soon.

Just to let everyone know, I'm really happy with the way 0.3 is shaping up. Lots still to do, especially around documenting/showing off the new cool features, but I'm still on track to release the first alpha by the end of the month.

[Johnny Weng Luu]

Can't wait for it!

Will you be announcing the alpha release in this mailing list?

[Owen B]

Hi Johnny

Yup, I'll announce it here and on Twitter. Likely to be just before
New Year.

Much still to do but between now and then, but I'm very excited about
getting it out there and working towards the first production-ready
release.

Owen

[Owen B]

Hi all

Well, I tried hard to get the first SocketStream 0.3 alpha release out before the end of the year but sadly it wasn't to be; mostly due to the large amount of work left to do, plus the fact I had my iPhone stolen a few days ago and lost a lot of time sorting it all out. All fine now thankfully.

However, I really didn't want to leave you all empty handed. So while the code is not ready yet, over the last few days I've been working on the start of the www.socketstream.org website which demonstrates all the new features in SocketStream 0.3 with example code.

The website is now live (running on 0.3 of course!) for the purpose of showing you the new slide tour (please don't advertise it widely just yet). I'll be finessing the design and building-out other sections of the site in the near future, leveraging as many of the cool realtime features within SocketStream as possible.

You'll also notice the awesome new SocketStream logo I announced at KRTConf. Many thanks to Craig Jordan Muir for designing it.

All the code shown in the feature tour currently works with the alpha release but is likely to change plenty before 0.3.0 is ready. I'll be adding many more slides, graphics, section indexes and more to the tour over the coming days and weeks, especially in response to questions and feedback.

I'm really not keen to set myself another deadline when the alpha code will be released on Github - there is still plenty to do and the list is likely to grow as a result of showing the API today; but I know I have a good number of free days coming up to work on it. With luck you'll be able to play with the code within the next two weeks.

It's worth pointing out that most of the features in 0.2 have been carried through to 0.3, though not necessarily provided in the core. However please note:

• The 'Users Online' functionality has been removed from the core and will now be a separate optional module (which I've already started working on).
  
  
• The HTTP API will not be supported in the first alpha release for a number of reasons I will discuss here at some point in the future. There will definitely be some form of HTTP API in the 0.3.0 release, but I still need to decide upon the best way to implement it.
  
  
• Most importantly, the scaling feature built into 0.2 (the ZeroMQ internal RPC layer) has deliberately not made its way into 0.3. After a lot of thought I decided this approach was too complex to maintain and attract contributors. It also incorrectly treated all incoming requests in the same way, even though you may often care more about latency than distributing requests to multiple servers.
  
  0.3 will not come with any inbuilt scaling solutions but the new modular websocket transport layer coupled with the new RPC middleware and modular websocket message responders should provide us with all the tools we need to experiment with different ways to scale out in the future.

All comments and suggestions on the new ideas welcome. The API syntax may change slightly before the first alpha is released (in which case the slides will be updated), but in general the code you see is the code you'll be using.

Have a really great New Year everyone. I'll reply to all comments tomorrow.

Owen

[deizel]

Very excited to hear about all this! I realise this isn't the best timing, but the website doesn't appear to be up at the moment. Anyway, have a great new year, I'll check back next year. :)

[Owen B]

Ah bad timing indeed! It's up again now. Please try again when you get chance.

Just goes to show the enormous gulf between 'works fine for me' and production ready software.

[Quang Anh Trịnh]

I'm really glad something had showed up after a long wait. FYI, I'm betting the future of my project on SocketStream, and it is an MMO game.

[Andrey Tarantsov]

I'm planning to rewrite one of my projects on SocketStream too (as soon as I get to updating it). But, I guess even 0.2 is good enough for me there, so I'm not adding to the 0.3 rush :)

[Quang Anh Trịnh]

I think it might be nice to update new code to the master branch as it grows... so that everyone is kept up to date, and be well prepared for 0.3.

[christophe vidal]

" I demoed on stage a simple app working in SocketIO then instantly switching to Pusher Pipe by changing one line of config in your app" 

Could you elaborate on this line which needs to be added in order to switch to Pusher Pipe? I'd like to test that on Heroku but didn't find the explanation in the doc.

thanks!

[Owen B]

Hey Christophe

While the experiment with Pusher Pipe worked, it proved to be a bad idea in reality as the latency was too high. Therefore I took the decision to discontinue support.

Heroku is good, but NodeJitsu provides hosting with full WebSocket support, so I would advise you to take a look there.

Owen

[christophe vidal]

Hi Owen,

oki, that makes sense. But using Heroku is usually a requirement for me hence ... In the meantime while Heroku doesn't support Sockets, I followed this fix and it works quite well: https://github.com/Anephenix/ss-engine.io/issues/7 . As for scalability and performance, that's a good question.

Christophe