[erlang-questions] Non-blocking send in erlang: how to save memory when send one content to many people

[Max Lapshin]

I'm sending live stream to 4000+ users.

Stream is saved in memory (in shared ETS table, in binaries) in chunks
500 KBytes each.

Client comes and asks for next chunk.

If I write server in C (or with linked-in driver), I whould just
manage common shared circular buffer from which I will send data and
remember
client position in this buffer. If client is too slow to read from
this buffer, it is disconnected and forgotten.

Such approach give ability to save memory and not to copy it in
buffers and driver queues.

Is it possible to have something like this in erlang?

Currently, I've got limits around 3 GBit/s from one erlang node on
loopback interface and I want to raise this limit.
When traffic comes to this limit, memory begins very fast growing and
when it reaches limits of RAM, massive disconnects happen.
This is why I think, that I should save memory first without trying to
write linked-in tcp driver =)
_______________________________________________
erlang-questions mailing list
erlang-q...@erlang.org
http://erlang.org/mailman/listinfo/erlang-questions

[Lukas Larsson]

You might be interested in the new (R15B) file:sendfile function, if having the data on disc is an option.

[Max Lapshin]

On Thu, Dec 15, 2011 at 2:23 AM, Lukas Larsson
<lu...@erlang-solutions.com> wrote:
> You might be interested in the new (R15B) file:sendfile function, if having
> the data on disc is an option.
>

I think, that something like vmsplice may be an option. I see in
sources of file, that prim_inet:getfd is no here (yes, I know that it
is undocumented =)
and that is why I may use some weird tricks with optimizing.

sendfile is a bad option because:
1) I need to serve from memory
2) I need to serve several thousands of connections. Threads are not
the way to go.

However, I think that I will need to patch efile_drv for using
something like vmsplice

[Max Lapshin]

So, I think that proper behaviour for sending 1 megabyte of data is:

1) increment ref counter for binary in driver
2) replace writev with vmsplice for linux
3) release binary when data is transmitted

I need to check if this really works and helps.

[Jon Watte]

We regularly have a hundred thousand sockets open, and an Erlang process per socket. We don't do high throughput I/O to each of those clients, but Erlang processes are *not* operating system level threads.

Also, binaries on the heap are shared across processes, if they are bigger than some limitation -- I think it's 64 bytes.

Thus, you COULD have some process that pre-loads files into binaries. Then, when a  socket connects and asks for data, you ask for that binary from that process, and then write that binary to the socket. One Erlang process per socket.

This would be a super simple implementation and probably very robust. I don't know if it will scale to the I/O throughput that you request, but it certainly would have a chance to do so.

Sincerely,

jw

--
Americans might object: there is no way we would sacrifice our living standards for the benefit of people in the rest of the world. Nevertheless, whether we get there willingly or not, we shall soon have lower consumption rates, because our present rates are unsustainable.

[Rapsey]

My streaming server is in Erlang and uses this approach. Though a linked in driver is not a good idea. I started with a linked in driver but it was pretty much impossible to get it really stable. With NIFs you can use resources for sockets and stream buffers. They work beautifully.

I have a NIF thread that listens on kqueue/epoll and communicates with a gen_server. Sockets are resources. Communication from NIFs to gen_server is simple since enif_send exists, communication from gen_server to the NIF thread is done by pipes (global library read pipe and write pipe).

The Erlang process that is responsible for sending data to the sockets holds a resource to the stream buffer and a list of socket resources. So the actual loop that sends the data is in Erlang and it calls a NIF with buffer and socket resource on every iteration.


Sergej

[Zabrane Mickael]

Hi Sergej,

On Dec 15, 2011, at 8:55 AM, Rapsey wrote:

My streaming server is in Erlang and uses this approach. Though a linked in driver is not a good idea. I started with a linked in driver but it was pretty much impossible to get it really stable. With NIFs you can use resources for sockets and stream buffers. They work beautifully.

Very interesting approcah.

I have a NIF thread that listens on kqueue/epoll and communicates with a gen_server. Sockets are resources.

1 thread NIF per core?

Communication from NIFs to gen_server is simple since enif_send exists, communication from gen_server to the NIF thread is done by pipes (global library read pipe and write pipe).

Could you please elaborae further on how communication is made from Erlang to NIF (the read/write pipe)?

Regards,

Zabrane

[Rapsey]

On Thu, Dec 15, 2011 at 9:00 AM, Zabrane Mickael <zabr...@gmail.com> wrote:

I have a NIF thread that listens on kqueue/epoll and communicates with a gen_server. Sockets are resources.

1 thread NIF per core?

All that the NIF thread does is accept connections and reads from sockets. I see no need to have more than 1.
 

Communication from NIFs to gen_server is simple since enif_send exists, communication from gen_server to the NIF thread is done by pipes (global library read pipe and write pipe).

Could you please elaborae further on how communication is made from Erlang to NIF (the read/write pipe)?

 
On thread initialization the pipes get created and read pipe is placed in epoll/kqueue. Write pipe is from erlang to NIF, and read pipe is for NIF to read that data. I have a simple struct:
typedef struct kqmsg
{
    char what;
    int fd;
    ErlNifPid pid;
    void* data;
}kqmsg;
So I just fill up this struct with whatever info is required and do: write(pipe_write,&msg,sizeof(struct kqmsg)

As for sockets, I do not use prim_inet:getfd, sockets are completely separate from gen_tcp. The NIF thread keeps the socket FD until it reads the first buffer from it. Once this happens it creates a socket resource, then sends the binary and socket with enif_send to the Erlang process that is in charge of deciding what to do with it.


Sergej

[Rapsey]

On Thu, Dec 15, 2011 at 11:48 AM, Zabrane Mickael <zabr...@gmail.com> wrote:

> On thread initialization the pipes get created and read pipe is placed in epoll/kqueue. Write pipe is from erlang to NIF, and read pipe is for NIF to read that data. I have a simple struct:
> typedef struct kqmsg
> {
>     char what;
>     int fd;
>     ErlNifPid pid;
>     void* data;
> }kqmsg;
> So I just fill up this struct with whatever info is required and do: write(pipe_write,&msg,sizeof(struct kqmsg)
>
> As for sockets, I do not use prim_inet:getfd, sockets are completely separate from gen_tcp. The NIF thread keeps the socket FD until it reads the first buffer from it. Once this happens it creates a socket resource, then sends the binary and socket with enif_send to the Erlang process that is in charge of deciding what to do with it.

Does this approach "really" increase performances of your server?

In the live streaming use case absolutely. HTTP is half-duplex. Once the server receives all the headers, it does not need to listen on that socket anymore. This means there is no need to keep the socket in the NIF listen thread and all you are doing is periodic write's on the FD from the stream buffer.  This is where the real optimization is. Maintaining one circular buffer and just looping sockets over it and doing writes.
On my servers traffic goes through haproxy on port 80. Once there is a good amount of users on the server, haproxy actually uses more CPU than my streaming server.


Sergej

[Zabrane Mickael]

>
> In the live streaming use case absolutely. HTTP is half-duplex. Once the server receives all the headers, it does not need to listen on that socket anymore. This means there is no need to keep the socket in the NIF listen thread and all you are doing is periodic write's on the FD from the stream buffer. This is where the real optimization is. Maintaining one circular buffer and just looping sockets over it and doing writes.
> On my servers traffic goes through haproxy on port 80. Once there is a good amount of users on the server, haproxy actually uses more CPU than my streaming server.

Makes sense now. Thanks Sergej

Regards,
Zabrane