SocketStream 0.4 Preview

[Owen B]

Hi all

I'd like to give you a quick preview of what you can expect to see in SocketStream 0.4, our next major release:

• A new Stream API! Pipe() news feeds and more over the websocket to all connected clients or Private Channels
  
• Vastly improved Request Responder API. Define new responders inline with (optional) integrated JSON and callback support
  
• Bundled with Engine.IO by default. Easily switch to SockJS as before
  
• Run multiple instances of SocketStream within one process
  
• Allow packing of assets without starting the HTTP server - great for a pre-deploy Makefile
  
• Option to Gzip assets and cache all minified JS & CSS in RAM
  
• Server-side Event Bus. Allows you to run clean-up code after a websocket closes and much more
  
• Specify a different 'entry' file for each Single Page Client
  
• Silent output. No longer writes anything to console.log by default. Pass your own logging functions if desired
  
• Allow any Request Responder to be unit tested and accessed over ss-console
  
• Improved internal architecture - leaner and faster than 0.3

Of course, there are likely to be many other new features and changes as development progresses. I just want to give you a taster of what's to come.

Evolution not Revolution

As many of your know, SocketStream 0.3 was a complete rewrite/rethink which caused me to drop many of the features in earlier versions. Rest assured this will not be the case with SocketStream 0.4.

SocketStream 0.4 will build upon the many solid ideas in 0.3 by improving the implementation as we progress towards a stable 1.0 API.

One more important change

SocketStream 0.4 will be written in vanilla Javascript.

What!? Does it mean I've suddenly gone off CoffeeScript? Far from it. I love writing in CoffeeScript and will still keep using it in my apps; however, an open source project is nothing without a steady stream of contributions. Changing from CoffeeScript to vanilla JS will encourage more people to contribute without the confusion/overhead of running `SS_DEV=1` or `make build`.

Availability

So when can you start to play with 0.4?

Development is already well underway but there's plenty of thinking and coding still to do. Although it will ultimately be ready when it's ready, there's a very good chance a working alpha will hit Github in time for my LXJS talk at the end of September.

Contributing

I frequently get asked how best to contribute to SocketStream. Now the 0.3 API is stable, here are a few suggestions:

• Continue submitting pull requests - always greatly appreciated!
  
• Scour the 0.3 code for possible bugs and security exploits. I will make sure any changes make it into 0.4
  
• Write unit tests (especially for things like the RPC Responder which won't change much in 0.4)
  
• Improve the documentation, especially for complete beginners
  
• Join existing efforts to write Request Responders for Backbone, Angular, etc (search our Google Group for details)
  
• Help improve www.socketstream.org (I'll be putting the source code on Github soon)

I'm also looking to establish a core team of talented developers who are able to make regular contributions and help shape forthcoming releases. Please get it touch if this sounds interesting to you.

Feel free to tell me what you would like to see in 0.4. I'm always open to new ideas - bigger and bolder the better :)

Owen

[David Rosen]

This is very exciting to read. All of these are interesting new features that I'd love to test. I will really enjoy following along with the progress of 0.4. I'd like to suggest something that could improve community involvement.

My suggestion is to create the 0.4 branch now. When it's ready to preview as an alpha, branch master to 0.3 and merge 0.4 into master. This will enable you to commit more frequently, even if it doesn't build yet or tests are failing. I believe that seeing weekly or even daily progress will encourage others to contribute while you're writing it.

Cheers,
Dave

[Eddie]

Just curious - will the event bus feature allow for an "on connection" handler as well?

On Thursday, August 2, 2012 5:19:16 AM UTC-7, Owen B wrote:

[Owen B]

Hey guys

Eddie: Yes, it will indeed.

David: Glad you like the list of features :) As soon as I am happy to put some code on github, I will. But right now I need complete freedom to try-out (and often disregard), new ideas in private.

Owen

[David Rosen]

Hi Owen,

No worries, I fully understand and respect your need to code in private. I recently started using ss.load.code() to load some code on demand, and I have some ideas:

1. Have you considered adding a similar capability for on-demand template loading?

2. It would be nice to be able to specify an array of files or folders to all load asynchronously. For example, page A and B both have a box X. When page A is requested, I'd like to be able to do something like:

    ss.load(
      code: ['/demand/page-a', '/demand/box-x', '/demand/dependency-i.coffee']
      tmpl: ['/demand/page-a', '/demand/box-x', '/demand/dependency-j.jade', '/demand/dependency-k.jade']
    , ->
      # gets called after all dependencies are loaded
    )

where, for instance, the box X assets may already be loaded from previously viewing page B.

3. Having a common folder to put demand-loaded code and templates would eliminate the repetition of '/demand/' above:

    ss.load(
      code: ['page-a', 'box-x', 'dependency-i.coffee']
      tmpl: ['page-a', 'box-x', 'dependency-j.jade', 'dependency-k.jade']
    , ->
      # gets called after all dependencies are loaded
    )

and it's similar to ss.client.define().

4. It would be great if the namespace was relative to that common 'code/demand/' folder. So 'code/demand/page-a/model.coffee' would be required with:

    require('/page-a/model')

regardless of whether it was loaded as 'page-a' or 'page-a/model.coffee'.

5. How about support for minifying (and gzipping and RAM caching) on-demand code in production mode? It would need to happen only on the first request for the file and be erased when the other packed assets are erased. A possible path suggestion for these assets would be something like:

    static/assets/demand/[random number]-code/[subfolders]/[filename].js
    static/assets/demand/[random number]-tmpl/[subfolders]/[filename].html

where the random number could be the same one as for the other packed asset filenames, and the subfolder and filenames are the same as those of the original files.

Dave

[David Rosen]

An alternate suggestion that's probably easier to implement: Use ss.client.define() to define a bundle of code to load, which enables the files to be packed exactly as they are currently. In app.js:

ss.client.define('main-page-a-bundle', {
  code: ['page-a', 'dependency-i.coffee'],
  tmpl: ['page-a', 'dependency-j.jade']
});

ss.client.define('box-x-bundle', {
  code: 'box-x',
  tmpl: ['box-x', 'dependency-k.jade']
});

and in the client:

ss.load.bundle(['main-page-a-bundle', 'box-x-bundle'], ->
  # gets called once specified bundles are loaded
)

[Robert]

My two cents in response to David's recommendations:

It's a very good idea, in resource handling, and in any framework to be able to handle file dependencies in "stacks" -- I'm thinking of Modernizr and YepNope as an example. Require.js does this to, in it's own flavor. Making that easy to handle in SocketStream is a good idea, especially as you bulk up multiple instances (great idea!). Totally enthused by the direction this is taking. SocketStream already handles dependencies fairly well, but this kind of organization is so key, and making it the best possible -- thumbs up.

For what it's worth, I like both Meteor.js and Derby.js, but I think you've steered this in the right direction by looking more for developer input, and truly trying to make things more configurable and not less. And not trying to do everything. :)

[Oleg Anashkin]

Hi Owen,

Please also consider adding a new asset type, similar to js: client-side templates which are server-compiled to javascript. They should be served after the regular js files instead of being included into head tag.

Thank you,

Oleg

On Thursday, August 2, 2012 5:19:16 AM UTC-7, Owen B wrote: