Testing websockets app in isolation

[Wai Lee Chin Feman]

Hello,

I have encountered an interesting problem testing components of a websockets server.  My application looks something like this:

-----------------------------------------------------------------------------------------

(defn my-app [request-channel request]

  (receive

    request-channel

    (fn [client-specified-name]      

       (siphon

          (transfom-to-response request-channel client-specified-name)

            request-channel)))))

(defn transform-to-response [channel name]

   (map* ................))

-----------------------------------------------------------------------------------------

And my test code looks like this:

-----------------------------------------------------------------------------------------

(def karl-channel (named-channel "karl"))

(my-app karl-channel {})

(enqueue karl-channel "karl")

(enqueue karl-channel request-message)

-----------------------------------------------------------------------------------------

Now, the app works fine when I start up a server and test it through a browser.  However, I am having trouble testing 'my-app in isolation.  When I enqueue a message into karl-channel, the "response" gets re-enqueued into karl-channel!  It then makes its way into transform-to-response, and causes my app to crash.

Is there a way to test at the 'my-app level without having the obvious issue of "this channel is siphoned into itself and the best case is for an infinite loop"?

I could start up a server and connect to it using the 'websocket-client function, but I would like to test at a lower level.

Thanks!

[Wai Lee Chin Feman]

The breaking tests are included in the last commit to the project:

https://github.com/skatenerd/moose

[Zach Tellman]

The confusion here is due to the difference between a unidirectional and bidirectional channel.  When channels represent a network connection, messages sent into the channel cannot be consumed from that channel.  This is what lets us do something like:

(siphon ch ch)

to create an echo server.  If that were a normal channel, we'd simply be looping a channel back into itself, creating an infinite loop.  In fact, that's exactly what you're doing in the code above: 

(siphon

  (transfom-to-response request-channel client-specified-name)

  request-channel)

The issue here is that you're not creating a bidirectional channel using (channel-pair).  A message sent into a paired channel can be read from the other channel, and vise-versa.  As a result, this is perfectly fine:

(let [[a b] (channel-pair)]

  (def a a)

  (def b b))

(siphon a a)

In your tests, pass in one of the paired channels to the handler, and use the other to send and receive messages.

Zach

--
You received this message because you are subscribed to the Google Groups "Aleph" group.
To unsubscribe from this group and stop receiving emails from it, send an email to aleph-lib+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[Sid Kurias]

Could you clarify what you mean by
 'When channels represent a network connection, messages sent into the channel cannot be consumed from that channel.'

Also does this imply that when connecting from the browser, aleph knows this is a network connection and automatically creates a channel pair?

I am not clear when I should use a channel pair. Quite a few examples seen in postings in this group  seem to be working with websockets without explicitly instantiating channel pairs. Or have I missed something obvious?

Thank you
Sid

[Zach Tellman]

Hi Sid,

For every bidirectional connection, Aleph creates a channel-pair under the covers.  Examples of bidirectional connections include TCP, WebSockets, and UDP sockets with receive turned on.  Examples of unidirectional connections include the response channel for normal HTTP requessts, and anything derived from (channel).  

The short answer, then, is that you only use channel-pairs when you want to model bidirectional communication between two different parts of your code, or when you want to mock out part of Aleph.

Let me know if any of the above doesn't make sense, and I can go into more detail.

Zach

[Wai Lee Chin Feman]

I think the important distinction here is that I was asking specifically about testing code.  In the non-test code, this is all relatively transparent.  You simply enqueue messages intended for the client back into the client's channel.  Or, you read out of that channel to get messages sent by the client.

When it comes time to test, things get just a little bit more hairy.  You have to simulate the convenience above manually.  You can see the test code I wrote here:  https://github.com/skatenerd/moose/blob/master/test/moose/integration_test.clj

--
You received this message because you are subscribed to a topic in the Google Groups "Aleph" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/aleph-lib/3Z5lE1nUG8Q/unsubscribe?hl=en.
To unsubscribe from this group and all its topics, send an email to aleph-lib+...@googlegroups.com.

[Sid Kurias]

Zach, Wai

Thank you. Both the answers helped, as did the test code.
But more details will be welcome!!!

Specifically on the wiki page for Connections it says

'Lamina represents a network socket using one of these channels' (of a channel pair)

My question is which one and does it matter?

It seems that when one creates a channel-pair you get
channel c => (splice a b)
channel d => (splice b a)

in all there are 4 channels involved.

When one en-queues data onto c, it gets en-queued to b and  when one consumes from c one is consuming from a.
And vice-versa for channel d. I hope that understanding is right

In the original post when the other end is a browser and websockets are being used aleph creates a channel-pair for you.So now the server has a way to send and receive data to the client or rather to the websocket that represents a client connection, because aleph has created a channel-pair. Where as if one were not using a network connection- as in the tests-, then one could only read from the channel i.e. consume from the channel and not send data to the client. Hence the example in the original post fails. It is a unidirectional channel used in a bidirectional manner which leads to the infinite loop situation. Or at least that is my understanding.

In response to the original post, you do mention

In your tests, pass in one of the paired channels to the handler, and use the other to send and receive messages.

Is this simply because
a. The handler can only receive one channel
b. The handler modifies the channel and creates a new channel and the reference to this new channel is not available outside the handler. Is the reference to the new channel important? I guess the same results can be obtained by holding on to the original reference. What I mean is suppose I do the following....

   (defn my-handler [ch req]
           (map* #() ch) ;; this should create a new channel which modifies the output of channel ch .....
Somewhere in my code if I have

      (let [ [a b] (channel-pair)]
           (my-handler a req)....
            (read-channel a))
this read-channel should return data that is modified as per the map* call in the handler. Is that correct? If so I really do not need the reference to the new channel. i.e, point b above is not relevant.
or
c. is there another reason for this specific recommendation that you have made.

Sorry for the long post.
Thanks
Sid

[Zach Tellman]

I'm a little unclear on what you're asking, so let me try to just explain what's going on another way.

Each channel is two halves, an emitter which creates messages, and a receiver which accepts messages.  In a normal unidirectional channel, these are the same thing.  However, via (splice ...) you can create a channel which sends messages once place, and receives them from somewhere else.

Thus, (splice a b) doesn't really create a new channel, so much as a view into two pre-existing channels.  The channels created by channel-pair are mirror images of each other, so it doesn't matter which you send into the handler and which you keep for yourself.  Think of them as cups connected by string; which cup you get shouldn't matter.

For bidirectional communication, using a unidirectional channel doesn't make sense.  As you point out, it will create an infinite loop.  This is more readily understood using the tools in lamina.viz.  It's notable that if you pass in half a channel-pair, lamina.viz/view-graph will render that as two separate channels, since really it's two channels masquerading as one.

As for your second question, the reason is that the handler can only take a single channel, but furthermore that for the purposes of bidirectional communication it only cares about half of each unidirectional channel (the sending half of one, and the receiving half of the other).  If each handler took two full channels, there's a lot of potential for confusion if the two channels ever get mixed up.  Splicing together only the halves that the handler cares about takes away any potential for confusion.

Your 'B' answer seems to be based on a misunderstanding of Lamina's semantics.  Remember that calling map* on a channel doesn't modify that channel, it returns a new channel, and under the covers extends the topology.  This is explained in detail in Lamina's wiki.

Hope that helps,

Zach

[Sid Kurias]

hmmm....let there be light!!!!

I think your wiki pages are quite well documented. I understood the semantics behind pipelines and channels after a couple of reads.
It was when I got to Connections that I got confused. I did not understand how the channel pair worked to give you a bidirectional connection, since I  did not understand how the splicing worked. I think your explanation clears it up quite well. It was the missing piece for me.

Now I feel confident I can  implement with aleph/lamina and that the abstractions you have chosen are right ones :-).

Thanks
Sid

[Zach Tellman]

Hey Sid,

I agree the "Connections" wiki needed some cleanup.  I've made some changes, I'd appreciate any feedback you can give, in light of your newfound understanding.

Zach

[Sid Kurias]

Zach,

I have tried to outline my thought process with some drawings. The ascii art does not look too good. I have attached png for the same.

I think the current documentation is certainly much better. If I were coming to

Aleph now - for the first time - I would find it a lot easier to understand the

documentation. There is a point on which I may  still struggle to get a

clear understanding. This refers to the place where you mention that you use

only one half of a channel at a time.I can explain why I struggled with this

and maybe that will help your documentation.

In my mind the model of a channel was of a monolithic block where one end

was the receiver and the other the emitter. An incorrect model as I now

understand.

           +--------------------------+

Receiver End    --------------->        Emitter End (Channel A)

           +--------------------------+

           +--------------------------+

Emitter  End    <------------------    Receiver End (Channel B)

           +--------------------------+

I visualized this model in terms of a water pipe. Where water goes in at one

end and exits at the other. One monolithic piece of plumbing. With this model

it was difficult for me to visualize how the receiver from Channel A could talk

to the emitter from Channel B.

I was missing the bigger picture. After you explained splicing it was clear

that my model of a channel was incorrect and it had to change from one monolithic piece with

the receiver and emitter at each end, to three discrete pieces. To continue with

the plumbing analogy, there is a water tank(Receiver), pipes

(although aleph may not have this) and the tap (Emitter). So I went to a model as

shown below...

 1  Channel A          Channel B

-->+--------+           +--------+

+--|   R    |     +-----|   R    |---------+

|  +--------+     |     +--------+         |

|     | P |       |        | P |           ^ 3

|     |   |       | 4      |   |           |

|  +--------+     |     +--------+         |

|  |   E    |<----|     |   E    |---------+

|  +--------+        2  +---^----+

+---------------->----------+

When I went to this model I realized that the relationship between the emitter

and receiver is not a one-to-one but a many-to-many relationship.

That is when it all made sense and your statement of having to use only half a

channel rang true.

This also helped me understand the echo server and the statement

(siphon server server)

Now I could visualize the flow from Channel A - Receiver to Channel B -

emitter. From there upto Channel B - Receiver and then down back to Channel A -

Emitter.(1-2-3-4)

I do not know whether this model is accurate but it certainly helps me

understand the concepts. :-)

Hope that helps or let me know if you have any questions.

Sid