Faye Redis Protocol (for posting messages via C#)

[anon]

I'm looking into using Faye for the real-time portion of my application. However, my code is set up right now so that all of my backend calls are made via C#, but my front end is served via node.

I was hoping to find a way to such that users would subscribe to the front end servers using the Faye Node client and Redis engine for subscribing to messages, but the C# api would be responsible for actually posting the messages into Redis for the Node servers to consume and send to the client. Is this something that is possible with the current Faye setup?

Is there a defined protocol that I can use for posting messages to Redis from non-node code (C# in this case, but I assume it doesn't matter as long as there as a Redis adapter for the language in question)?

Thanks!

[anon]

Of course there's also the option of communicating with the Faye servers via HTTP. But that's an extra layer of latency. 

[James Coglan]

On 2 April 2012 19:58, anon <psel...@gmail.com> wrote:

I was hoping to find a way to such that users would subscribe to the front end servers using the Faye Node client and Redis engine for subscribing to messages, but the C# api would be responsible for actually posting the messages into Redis for the Node servers to consume and send to the client. Is this something that is possible with the current Faye setup?

Is there a defined protocol that I can use for posting messages to Redis from non-node code (C# in this case, but I assume it doesn't matter as long as there as a Redis adapter for the language in question)?

This is certainly possible, although the Redis interface has not been standardized. It should be fairly clear from the faye-redis source how to do this, but bear in mind you're coding against something that's not supposed to be a public interface.

My stock answer to this question is that you should use HTTP/WebSocket to send messages to Faye, and Redis should be considered an implementation detail of running multiple webservers, not an API service to integrate with. 

[Paul]

Thanks for the response. If I did write directly to Redis -- would the Node servers be notified that there are messages waiting on that channel, or do I have to explicitly tell it (via the HTTP endpoint) about that data?

As an unrelated question -- is it possible to horizontally scale the storage engine (in this case Redis), or are we stuck with vertically for this?

On Monday, April 2, 2012 5:59:03 PM UTC-4, James Coglan wrote:

[James Coglan]

Thanks for the response. If I did write directly to Redis -- would the Node servers be notified that there are messages waiting on that channel, or do I have to explicitly tell it (via the HTTP endpoint) about that data?

If you do the same thing that the Redis engine in Faye does, the Node server will pick it up this basically involves pushing the message onto each target client's message queue, then sending a message on the Redis pub/sub channel /notifications so other servers are notified. If you go through Faye's HTTP interface you don't have to touch Redis at all.

As an unrelated question -- is it possible to horizontally scale the storage engine (in this case Redis), or are we stuck with vertically for this?

The common strategy for this is to shard the channel space, i.e. you run multiple Faye servers with in-memory storage, or with a Redis DB for each Faye server. Each server is responsible for a distinct set of channels, and clients need to figure out which servers to connect to for the channels they need. You could even load-balance each shard by having multiple servers within a shard share a Redis DB. 

[Paul]

Interesting approach. This would make the client have to hold multiple connections (one for each shard), right? I wonder if you could throw another layer of servers in front of it that are responsible for figuring out which shards to connect to, so the client would still only have to make one connection.

[James Coglan]

On 13 April 2012 17:35, Paul <psel...@gmail.com> wrote:

Interesting approach. This would make the client have to hold multiple connections (one for each shard), right? I wonder if you could throw another layer of servers in front of it that are responsible for figuring out which shards to connect to, so the client would still only have to make one connection.

I've seen it used best when each page of a site only subscribes to a small subset of the channel space, or just one channel, and you shard such that each page only makes one connection. The server can make as many connects as needed to push when it needs to, but this is less of a problem than client-side connections. 

[James Coglan]

On 20 April 2012 00:15, Jeff Robert Dagala <jrda...@gmail.com> wrote:

Hi James, I am a newbie on C# and I'm wondering if you could teach me
how to subscribe, send message and wait for response on faye using
WebSocket

I don't know C# but I can describe briefly how to do it in Ruby if that helps. Of course, for a robust solution you will probably end up porting the whole Faye/cometD client codebase to C#.

Here's the simplest thing you can do, with no error handling, transport negotation etc. The handshake must be done over normal HTTP, not sockets:

require 'uri'

require 'net/http'

require 'json'

require 'faye/websocket'

uri = URI.parse('http://localhost:8000/bayeux')

message = JSON.dump('channel' => '/meta/handshake',

                    'version' => '1.0',

                    'supportedConnectionTypes' => ['long-polling','websocket'])

response = Net::HTTP.post_form(uri, :message => message)

handshake = JSON.parse(response.body).first

client_id = handshake['clientId']

Once you've got a client ID, you can open a socket to the server, register subscriptions and receive messages. The protocol requires that you poll the /meta/connect channel even when using WebSocket; message delivery is not tied to this but it serves as a keep-alive mechanism to keep the session active.

EM.run {

  ws = Faye::WebSocket::Client.new('ws://localhost:8000/bayeux')

  

  ws.onmessage = lambda do |event|

    message = JSON.parse(event.data).first

    case message['channel']

      when '/meta/connect'

        # poll /meta/connect again

        ws.send(JSON.dump('channel'        => '/meta/connect',

                          'clientId'       => client_id,

                          'connectionType' => 'websocket'))

      when '/meta/subscribe'

        # check message['successful'] and message['error']

      else

        # handle any other pushed message

    end

  end

  

  ws.onopen = lambda do |event|

    # send initial /meta/connect message

    ws.send(JSON.dump('channel'        => '/meta/connect',

                      'clientId'       => client_id,

                      'connectionType' => 'websocket'))

    

    # register a subscription for channel /foo

    ws.send(JSON.dump('channel'      => '/meta/subscribe',

                      'clientId'     => client_id,

                      'subscription' => '/foo'))

  end

}

[TJ Singleton]

If you are just posting messages you could push messages onto a redis list, pop them off on the node side, and then publish them via a faye client. Then you don't have to worry about a private protocol. 

[James Coglan]

On 25 April 2012 04:37, TJ Singleton <tjsin...@me.com> wrote:

If you are just posting messages you could push messages onto a redis list, pop them off on the node side, and then publish them via a faye client. Then you don't have to worry about a private protocol.

I'm not sure why you'd do this instead of just publishing directly to Faye via HTTP. Am I missing something? 

[TJ Singleton]

On Wednesday, April 25, 2012 4:43:03 AM UTC-4, James Coglan wrote:

I'm not sure why you'd do this instead of just publishing directly to Faye via HTTP. Am I missing something? 

If he didn't want to implement the handshake/websocket in C#. 

[James Coglan]

I might have got confused because this thread actually has two separate topics in it. Just to note that you need to handshake to publish messages -- you can publish without a clientId, it's as simple as:

curl -X POST example.com/faye -H 'Content-Type: application/json' -d '{"channel":"/foo","data":{"hello":"world"}}'

[TJ Singleton]

On Apr 25, 2012, at 6:16 PM, James Coglan wrote:

I might have got confused because this thread actually has two separate topics in it. Just to note that you need to handshake to publish messages -- you can publish without a clientId, it's as simple as:

curl -X POST example.com/faye -H 'Content-Type: application/json' -d '{"channel":"/foo","data":{"hello":"world"}}'

That's handy to know! 

[James Coglan]

On 25 April 2012 23:16, James Coglan <jco...@gmail.com> wrote:

you need to handshake to publish messages

Should have been: you *don't* need to handshake to publish messages