libuv attaching a custom object to a stream and reusing it.

[Rajiv Kurian]

I am evaluating libuv as a library for a C/C++ server that I am writing. The protocol is length prefixed so as soon as I can read a 32 bit integer from the stream I should be able to tell what size of buffer I should allocate. The documentation says that the uv_read_start function might be called multiple times.

UV_EXTERN int uv_read_start(uv_stream_t*, uv_alloc_cb alloc_cb, uv_read_cb read_cb);

Since I am using a length prefixed protocol, once I know the right size of the buffer I would like to allocate it and re use it for subsequent reads till I have received all my bytes. Is there an easy way to do this with libuv? Right now it seems like the uv_alloc_cb function has to take care of this. Can I associate my own buffer object with my stream object instead of putting it in a map or something? This buffer struct would contain a pointer to the actual byte array and the current position to continue writing from. It would be great if you guys can point me towards an example where 

• A buffer is initially allocated say after a new connection.
• It is re-used till it's possible to do so. Say in case of a length prefixed protocol till all expected bytes have been received.

Since I am using a length prefixed protocol, I don't want to allocate a buffer on the heap at all till I can read the first 4 bytes (32 bits). Is it possible for me to allocate on the stack a buffer of size 4 and have the uv_read_cb function actually do the heap allocation? Is the uv_read_cb function invoked synchronously as part of the uv_read_start function? If it is then seems like I should be able to allocate on the stack when I know that I don't already have a buffer attached to my stream.

What I am trying to achieve in pseudocode is:

if (new connection or new request on old connection) {  // New request on old connection happens when the previous length prefixed protocol message was received completely.

  byte arr[4];

  readBytes(byte, 4);

  Figure out expected length and allocate my buffer struct.

} else if (pendingRequest on existing connection) { // An old connection which is still receiving the bytes that it is supposed to receive.

  Fetch the buffer associated with this stream and re use the underlying byte array.

}

Thanks,

Rajiv

[Ben Noordhuis]

On Mon, Oct 7, 2013 at 4:19 AM, Rajiv Kurian <geet...@gmail.com> wrote:
> I am evaluating libuv as a library for a C/C++ server that I am writing. The
> protocol is length prefixed so as soon as I can read a 32 bit integer from
> the stream I should be able to tell what size of buffer I should allocate.
> The documentation says that the uv_read_start function might be called
> multiple times.
>
> UV_EXTERN int uv_read_start(uv_stream_t*, uv_alloc_cb alloc_cb, uv_read_cb
> read_cb);
>
> Since I am using a length prefixed protocol, once I know the right size of
> the buffer I would like to allocate it and re use it for subsequent reads
> till I have received all my bytes. Is there an easy way to do this with
> libuv? Right now it seems like the uv_alloc_cb function has to take care of
> this. Can I associate my own buffer object with my stream object instead of
> putting it in a map or something? This buffer struct would contain a pointer
> to the actual byte array and the current position to continue writing from.
> It would be great if you guys can point me towards an example where

The handle has a `void* data` field that is yours to use. You can
make it point it to an auxiliary structure where you store the length
and the buffer.

Alternatively, you can embed the uv_tcp_t in another structure, then
look up the embedding structure with container_of. It's not a
standard C macro but you can find its definition and usage examples in
the libuv/ source tree. Its benefit is that it just does some pointer
arithmetic, it saves you from another level of pointer indirection.

> A buffer is initially allocated say after a new connection.
> It is re-used till it's possible to do so. Say in case of a length prefixed
> protocol till all expected bytes have been received.
>
> Since I am using a length prefixed protocol, I don't want to allocate a
> buffer on the heap at all till I can read the first 4 bytes (32 bits). Is it
> possible for me to allocate on the stack a buffer of size 4 and have the
> uv_read_cb function actually do the heap allocation? Is the uv_read_cb

No, that's not possible. The proper way of thinking about it is that
your alloc_cb returns a buffer that libuv will fill with data sometime
in the future. The stress is on "sometime" because there are no
guarantees when that will happen; it may be immediate, it may be
seconds (or minutes) away.

[Rajiv Kurian]

Thanks for the answers Ben.

Why does alloc_cb need to be called asynchronously? Isn't uv_read-start already called asynchronously? If so then I am guessing this is only called when the libuv backend (epoll, kqueue etc) signals that there is a read interest. In that case the data to be read is already available isn't it? Why call alloc_cb asynchronously again?

If i was using epoll, I could do something like this:

if (data on fd is waiting to be read) {

  char * buffer;

  buffer = retrieve buffer from epoll_data;

  

  char buf[4];

  if (buffer is null) {  // New request on this connection.

    count = read(fd, buf, sizeof buf);

    if (count == 4) {

      // This is the length of the request.

      buffer = allocate buffer for this length;

      attach buffer to epoll_data;

      continue reading the rest of the data into this buffer;

    } else {  // Not enough bytes to read the length of the request.

      buffer = allocate buffer of 4 bytes and copy contents of buf into it;

      attach buffer to epoll_data;  // Use this buffer till the first 4 bytes can be read.

     }

  } else {  // There is already a buffer attached to the epoll_data use it.

    buffer = retrieve buffer from epoll_data;

    if (sizeof buffer == 4) {  // This buffer was just used to store the length. Use some other indicator if the actual data can only be of 4 bytes.

      new_position = retrieve current write position of buffer;

      remaining_bytes= number of free bytes in the buffer;

      count = read(fd, new_position, new_position, remaining_bytes);

      if (count == remaining_bytes) {  // Enough bytes to determine the length of a request.

        free(buffer);  // Free the temporary buffer and allocate a new one that can be used to hold all the data;

        buffer = allocate buffer of correct length;

        attach buffer to epoll_data;

        continue reading into this buffer at the correct position and size;

      } else {

        // Still haven't read 4 bytes. Just wait.

      }

    } else {  // A buffer with the expected size was allocated.

      buffer = retrieve buffer from epoll_data;

      new_position = retrieve current position of buffer;

      size = remaining size of buffer;

      continue reading into this buffer at the correct position and size;

    }    

  } 

}

Another follow up question. Where does libuv get the size_t suggested_size in the uv_alloc_c function from?

Thanks,

Rajiv

[Ben Noordhuis]

On Mon, Oct 7, 2013 at 8:14 PM, Rajiv Kurian <geet...@gmail.com> wrote:
> Thanks for the answers Ben.
>
> Why does alloc_cb need to be called asynchronously? Isn't uv_read-start
> already called asynchronously? If so then I am guessing this is only called
> when the libuv backend (epoll, kqueue etc) signals that there is a read
> interest. In that case the data to be read is already available isn't it?
> Why call alloc_cb asynchronously again?

uv_read_start() is what you as the user of libuv call to inform libuv
that from now on you want to receive incoming data. That incoming
data - when it arrives - is then read in an asynchronous fashion.

The reason you have an alloc_cb and read_cb is that libuv uses a
completion-based event model, not the readiness model of poll() and
friends. (The reason for that is that IOCP is completion-based; you
can efficiently simulate completion-based I/O with readiness-based I/O
but not the other way around.)

Completion-based means you tell the operating system "here is a
buffer, here is a socket handle, read data from one into the other and
ping me when you're done." That's why you can't make any assumptions
about the buffer you return from your alloc_cb, it may not be
processed straight away.

If you're only targeting UNIX systems, then yes, libuv basically does
read(handle, alloc_cb()) and you could optimize for that - but it's an
implementation detail that may change someday and you sacrifice
Windows portability for a minor convenience.

Right now, it's a hard-coded value of 65536. Querying the operating
system for the actual amount of data to read slashes throughput in
half and most high-performance applications use a slab allocator
anyway so it doesn't matter if there's a bit of initial wastage.

Note that it's perfectly okay to return a smaller buffer.

[Rajiv Kurian]

Thanks for the answers!

On Tuesday, October 8, 2013 1:52:15 AM UTC-7, Ben Noordhuis wrote:

On Mon, Oct 7, 2013 at 8:14 PM, Rajiv Kurian <geet...@gmail.com> wrote:
> Thanks for the answers Ben.
>
> Why does alloc_cb need to be called asynchronously? Isn't uv_read-start
> already called asynchronously? If so then I am guessing this is only called
> when the libuv backend (epoll, kqueue etc) signals that there is a read
> interest. In that case the data to be read is already available isn't it?
> Why call alloc_cb asynchronously again?

uv_read_start() is what you as the user of libuv call to inform libuv
that from now on you want to receive incoming data.  That incoming
data - when it arrives - is then read in an asynchronous fashion.

Sorry I mean't why is read_cb not called immediately after alloc_cb. 

The reason you have an alloc_cb and read_cb is that libuv uses a
completion-based event model, not the readiness model of poll() and
friends.  (The reason for that is that IOCP is completion-based; you
can efficiently simulate completion-based I/O with readiness-based I/O
but not the other way around.)

Okay this makes sense now. 

Completion-based means you tell the operating system "here is a
buffer, here is a socket handle, read data from one into the other and
ping me when you're done."  That's why you can't make any assumptions
about the buffer you return from your alloc_cb, it may not be
processed straight away.

If you're only targeting UNIX systems, then yes, libuv basically does
read(handle, alloc_cb()) and you could optimize for that - but it's an
implementation detail that may change someday and you sacrifice
Windows portability for a minor convenience.

Right, what I had in mind was a tiny optimization, but one that would apply almost every time given I expect to be able to read the first 4 bytes from a connection on the first read fairly often and hence the stack allocation. Allocating 4 bytes with jemalloc shouldn't be a big problem though and I'd rather keep portability as an option.

Yeah I was just curious, how libuv could even know a size without reading data in. I'll just allocate a 4 byte buffer to read the length initially and once I know it I'll free the small buffer and have my alloc_cb return an appropriately sized buffer.

[Bernardo Ramos]

Check this implementation:

https://github.com/litesync/libuv_message_framing

Bernardo Ramos