If remote server sends me 100k responses, will receive_data be called 100k times? No response coalescing/pooling?

[Arpan]

Hi, 

The receive_data method in EM.Connection has data buffered and given to the application as single block or is it 1 line of data at a time? What I am trying to get at is if the server at the other side (and this server is not based on EM) sends be 100k responses will I get 100k calls to my receive data or it could be lesser? 

Arpan 

[hemant]

Hi Arpan,

No you won't be getting 100K receive_data calls. There are really two things happening at EM end:

1. It tries and reads in chunks of 16 KBs. So, bunch of writes from one end may show up as one read from another end.

2. Because EM uses nonblocking IO, there is no line by line processing of data. Reading from socket happens when socket announces it is ready for read. That itself could mean -  multiple writes may arrive at the same time and read it in one go. 

Arpan 

--
You received this message because you are subscribed to the Google Groups "EventMachine" group.
To unsubscribe from this group and stop receiving emails from it, send an email to eventmachine...@googlegroups.com.
To post to this group, send email to eventm...@googlegroups.com.
Visit this group at http://groups.google.com/group/eventmachine?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

--
Let them talk of their oriental summer climes of everlasting conservatories; give me the privilege of making my own summer with my own coals.

http://gnufied.org
http://twitter.com/gnufied

[se...@sean-bollin.com]

What is a typical way of breaking up what you receive into distinct sends?

You need to implement a message protocol of sorts?  Like, end each send with "\n" ?  And then on the server side splitting the data you received by the "\n" character?

[Carsten Bormann]

On 15 Nov 2013, at 23:09, se...@sean-bollin.com wrote:

> What is a typical way of breaking up what you receive into distinct sends?

Size fields or delimiters (or both, but that is not actually necessary).

http://stackoverflow.com/questions/1046637/writing-a-stream-protocol-message-size-field-or-message-delimiter

http://en.wikipedia.org/wiki/Delimiter
"Delimiter - Wikipedia, the free encyclopedia”

> You need to implement a message protocol of sorts? Like, end each send with "\n" ? And then on the server side splitting the data you received by the "\n" character?

If newlines can’t occur in your messages, that’s fine.

Otherwise, see above. The delimiter scheme that gives data transparency as well as good efficiency is COBS:

http://en.wikipedia.org/wiki/Consistent_Overhead_Byte_Stuffing

Or start every message with its length (and then there is chunking, if you don’t know the length when starting to encode the message; HTTP has an ugly version of that).

http://en.wikipedia.org/wiki/Chunked_transfer_encoding

Often, you are best off using a self-delimiting message format such as XML, JSON or CBOR.

Grüße, Carsten