Server-side performance

[Brian Oh]

I initially looked at Dart as a JS replacement, and it does an absolutely brilliant job of satisfying that criteria. Then, I looked at some server-side coding, and found that it is extremely fast, so I started thinking about using server-side Dart as well. Then, I read recently that Dart compares well to Java in some benchmarks, however I don't know if that means utilizing multiple cores. While I would like to use Dart for both client-side and server-side, I am unsure as to what level it will perform that role. In terms of high-performance server with a lot of demand from the client-side, can Dart compete with other well-established and emerging languages? eg. C#, Java, Go? I have completed some tests doing batch-type RDBMS access, and Dart running a test program from multiple command-line processes, and performed extremely well. However that test does not show how the performance is shared across CPU's, because the O/S by virtue of running multiple processes would do that, and so would the RDBMS. Will Dart utilize multiple CPU cores, and how does a program achieve that. (Eg. Does a program need to create Isolates to achieve that)?

[Ladislav Thon]

> Will Dart utilize multiple CPU cores, and how does a program achieve that. (Eg. Does a program need to create Isolates to achieve that)?

As far as I know, yes, isolates are the only way to go multicore with Dart.

LT

[Gen]

I have not dared to make the effort to test myself the concurrency performance at the server side.
Where do we stand ?
Is the memory consumption per isolate still prohibitive compared to Java, Go or Erlang processes?

[Anders Johnsen]

Hi Brian,

We have improved the overall performance of dart:io over the past 6 months, and we are starting to see some very decent numbers - but we are still far from done.

The current implementation uses a separate thread to run epoll/kqueue/etc., on the different platforms to ensure we can quickly handle a lot of requests. However, in the current implementation, a Socket is bound to a single isolate. To overcome this limitation, we'd like to introduce the ability to transfer Sockets between isolates, so e.g. a HttpServer can pass requests to worker isolates. By doing so, it should be possible to scale the current throughput almost linear by the number of cores.

It should also be possible to utilize cores for e.g. the SSL workload, so it won't block the Dart thread.

I hope this answers some of your questions.

Yours,

- Anders

On Sat, May 18, 2013 at 7:27 AM, Brian Oh <brians...@gmail.com> wrote:

I initially looked at Dart as a JS replacement, and it does an absolutely brilliant job of satisfying that criteria. Then, I looked at some server-side coding, and found that it is extremely fast, so I started thinking about using server-side Dart as well. Then, I read recently that Dart compares well to Java in some benchmarks, however I don't know if that means utilizing multiple cores. While I would like to use Dart for both client-side and server-side, I am unsure as to what level it will perform that role. In terms of high-performance server with a lot of demand from the client-side, can Dart compete with other well-established and emerging languages? eg. C#, Java, Go? I have completed some tests doing batch-type RDBMS access, and Dart running a test program from multiple command-line processes, and performed extremely well. However that test does not show how the performance is shared across CPU's, because the O/S by virtue of running multiple processes would do that, and so would the RDBMS. Will Dart utilize multiple CPU cores, and how does a program achieve that. (Eg. Does a program need to create Isolates to achieve that)?

--
For other discussions, see https://groups.google.com/a/dartlang.org/
 
For HOWTO questions, visit http://stackoverflow.com/tags/dart
 
To file a bug report or feature request, go to http://www.dartbug.com/new
 
 

[Ladislav Thon]

> The current implementation uses a separate thread to run epoll/kqueue/etc.

Sounds like there might be a lot of copying involved, or are you trying to avoid that too?

> However, in the current implementation, a Socket is bound to a single isolate. To overcome this limitation, we'd like to introduce the ability to transfer Sockets between isolates

Is this going to be specific to sockets, or are we introducing "transferables"?

LT

P.S.: looks like you are doing a lot of awesome work. Hope that my questions don't sound like I don't appreciate it :-)

[Brian Oh]

@anders et al.

"To overcome this limitation, we'd like to introduce the ability to transfer Sockets between isolates, so e.g. a HttpServer can pass requests to worker isolates. By doing so, it should be possible to scale the current throughput almost linear by the number of cores."

Thanks for the feedback, and I'd like to say that I too appreciate the great work being done, and that Dart is a pleasure to work with and develop with, and it is much more than I ever expected or hoped for. If you can get it to go that extra mile, that would be great.

[Anders Johnsen]

On Sat, May 18, 2013 at 12:01 PM, Ladislav Thon <lad...@gmail.com> wrote:

> The current implementation uses a separate thread to run epoll/kqueue/etc.

Sounds like there might be a lot of copying involved, or are you trying to avoid that too?

No, it's purely event-based. The epoll will send a 'read' or 'write' event to dart, where dart will do the actual reading/writing (a fast / non-blocking operation, as it's alread ready).

> However, in the current implementation, a Socket is bound to a single isolate. To overcome this limitation, we'd like to introduce the ability to transfer Sockets between isolates

Is this going to be specific to sockets, or are we introducing "transferables"?

The main interest for me is sockets, but you are right that the implementation would be that of 'transferables'.

LT

P.S.: looks like you are doing a lot of awesome work. Hope that my questions don't sound like I don't appreciate it :-)

Thanks - not at all! :)

Yours,

- Anders 

[Anders Johnsen]

No problem, and thanks a ton! I know it's possible (from prototyping), it's just a matter of doing it "right" :)

Yours,

- Anders

[Michael Hixson]

Ah, I wish I had seen this thread earlier.  I just spent a few hours trying to get my Dart HTTP server to use all my cores. :)

I think this is a pretty important part of Dart being useful for HTTP servers.  The language and libraries are awesome -- I haven't picked up a new language and been able to write a web application this fast since... ever.  But the single-threadedness of it in this context is disappointing.  I recognize that it's a known problem and you have other (probably more serious) ones to deal with first.  Just take this as a "+1, would like that feature".

-Michael

[Søren Gjesse]

One thing to keep in mind when planning for a multithreaded Dart HTTP server where requests are handled in a number of isolates is that there is no sharing of data between isolates. Isolates can communicate through message parsing and if a Dart HTTP server with several isolates needs shared data then this data might need to be stored in a shared isolate (or perhaps several shared isolates).

Regards,

Søren

[Greg Lowe]

Another approach is to run multiple Dart HttpServer processes behind Apache or Nginx configured as a reverse proxy. I understand this is typically how node.js is deployed in production systems.

[Matthew Butler]

This is generally the much easier method to use, as you can also allow Apache/Nginx/etc to serve the static content much easier, and only forward your ajax calls for instance. This is the path I'm working towards for my apps. To get a 'comfortable' proxy configuration to allow me to write less dart server code, so I only need to worry about the logic stuff and not the static content which is much better optimized than what I want to work on for each of my projects.

[Filipe Morgado]

With only Isolates to play with, it seems quite difficult to implement a server-side multiplayer game.

In case "transferables" are implemented, what would be a typical architecture for such an application?

I would implement it in the following way.

From the main Isolate:

Create a shared Isolate that will initialize all the application data and logic.

Create an Isolate queue and initialize it with a number of worker Isolates equal to the number of available cores.

Create a ClientMessage queue to store ClientMessages awaiting to be handled.

Bind to the socket and listens for new connections and new ClientMessages.

After a client has connected (handled by the main Isolate), it starts sending ClientMessages. The main Isolate gathers the ClientMessages and stores them in the corresponding queue. When a worker Isolate is available, the main Isolate will "transfer" the next ClientMessage (along with the corresponding socket and other information) to the worker.

The worker Isolate parses the message, processes commands that do not require a shared state. If the worker Isolate requires access to the shared state, it sends a native message to the shared Isolate that will process the required operation and return an answer to the requesting worker Isolate, that will format the response for the client and send it. The worker Isolate then cleans up every memory used in the previous steps and registers itself as available to the main Isolate.

...

There's a problem with this approach. The shared Isolate needs to access all connection to broadcast updates. Maybe merge the main e the shared Isolates?

It seems there is a lot message passing with this approach. Would this be any efficient?

When a worker is started on a given message, it may execute a lot of async commands. How can I re-use the Isolate for another request while it's blocked in the current request???

I'm not sure we can get even close to the power of shared-state task/thread queues.

As a side-note:

I noticed that Platform.numberOfProcessors (dart:io) is final.

The resources available to a virtual server may change at runtime. Would this reflect inside Dart?

[Bob Nystrom]

On Thu, May 23, 2013 at 7:58 AM, Filipe Morgado <pix...@gmail.com> wrote:

When a worker is started on a given message, it may execute a lot of async commands. How can I re-use the Isolate for another request while it's blocked in the current request???

If those commands are async, then it won't be "blocked" on the current request at all. Each time it starts an async operation, it will return control back to the event loop which can then start responding to other requests when new ones come in.

Cheers,

- bob

[Filipe Morgado]

Quinta-feira, 23 de Maio de 2013 17:11:46 UTC+1, Bob Nystrom escreveu:

If those commands are async, then it won't be "blocked" on the current request at all. Each time it starts an async operation, it will return control back to the event loop which can then start responding to other requests when new ones come in.

We are in a worker Isolate. It doesn't directly receive new requests. The main Isolate gets all or most new requests and re-routes them to worker isolates when they're not currently running a previous request.

If a worker Isolate receives a new ClientMessage (from the main Isolate), it cannot register itself as available until it's finished with the current request, which could include a lot of async commands. We have no way to re-use a working Isolate when it's waiting for futures within its current request.

Dart is looking great for stateless HTTP requests (provided we use Apache or Nginx), way better than PHP, Node.js or any other single-threaded environment. Isolates are more than enough in this case.

But I think we need another approach for concurrent real-time applications. I don't think we can compete with Java or .NET on that front.

Shared-state task/thread queue are kind of a standard for high performance.

[Ladislav Thon]

If a worker Isolate receives a new ClientMessage (from the main Isolate), it cannot register itself as available until it's finished with the current request, which could include a lot of async commands.

Then the worker isolate is going to do precisely nothing most of the time, if the requests are IO heavy (sounds like it's the case, if you say that there will be a lot of async stuff going on). Your model makes sense when the requests require a lot of computation.

LT

[Don David Alegria]

is hyper threading really a big deal? afaict, if you are in the need of massive hyper threading, you'd be better of with an opencl binding / analogue of webcl.
Calls to the webserver / dartserver could utilise around 4-16 cpus. A good loadbalancing / devide and conquer algorithm would care to have a maximum of 3-5 processes running to not kill your server with thrashing respectively run very low on swap having tons of idle isolates... sequential may just be more reasonable to occupy IPS over the VM

So ... are isolates supposed to be a general paradigm to utilize a GPU? Most certainly no, but to be honest, I have no idea what the VM is thinking about that...

[Justin Fagnani]

On Thu, May 23, 2013 at 9:39 AM, Filipe Morgado <pix...@gmail.com> wrote:

Quinta-feira, 23 de Maio de 2013 17:11:46 UTC+1, Bob Nystrom escreveu:

If those commands are async, then it won't be "blocked" on the current request at all. Each time it starts an async operation, it will return control back to the event loop which can then start responding to other requests when new ones come in.

We are in a worker Isolate. It doesn't directly receive new requests. The main Isolate gets all or most new requests and re-routes them to worker isolates when they're not currently running a previous request.

If a worker Isolate receives a new ClientMessage (from the main Isolate), it cannot register itself as available until it's finished with the current request, which could include a lot of async commands. We have no way to re-use a working Isolate when it's waiting for futures within its current request.

Not necessarily, though it does get complicated. When the worker isolate is unblocked, even if it hasn't completed a request, it can process other requests. If the request handling is mostly I/O bound then the worker will be free most of the time. The main isolate just has to distribute requests among the workers intelligently, which is where things get complex.

There are many potential ways to do that, and it becomes a load-balancing problem. I think the biggest architectural questions, given the way the isolate and async APIs work currently, are:

  1. Does the main isolate push work to the workers, or do the workers ask for more work?

  2. Are workers expected to never do blocking operations, and spawn another isolate if it needs to block for a while?

The first question has implications around the communication protocol, who maintains work queues, whether you need to wrap common async APIs. Both approaches have the problem that Sockets or HttpRequest and HttpResponse are not transferrable.

I mentioned on StackOverflow that dart-isoserver has bit rotted, but it was a place where I was playing with multiple isolate servers. It might be good to get that back up and running to test various strategies. The current approach is to spawn an isolate per request, but that may be too slow until we get very light-weight isolates and transferrable sockets.

-Justin

Dart is looking great for stateless HTTP requests (provided we use Apache or Nginx), way better than PHP, Node.js or any other single-threaded environment. Isolates are more than enough in this case.

But I think we need another approach for concurrent real-time applications. I don't think we can compete with Java or .NET on that front.

Shared-state task/thread queue are kind of a standard for high performance.

--

[Filipe Morgado]

Quinta-feira, 23 de Maio de 2013 18:08:09 UTC+1, Don David Alegria escreveu:

is hyper threading really a big deal? afaict, if you are in the need of massive hyper threading, you'd be better of with an opencl binding / analogue of webcl.

Le's say you write a multiplayer game server, you'll probably setup a virtual server to host it, so it can scale independently of other apps.

If your game gets even little success (on a global scale), with a single processor, even the most powerful you can find, it won't scale.
You will need to assign more processors to your virtual server to successfully scale.

So yes, multi-threading IS a big deal, specially in server-side.

[Don David Alegria]

On Thursday, May 23, 2013 7:56:35 PM UTC+2, Filipe Morgado wrote:

Quinta-feira, 23 de Maio de 2013 18:08:09 UTC+1, Don David Alegria escreveu:

is hyper threading really a big deal? afaict, if you are in the need of massive hyper threading, you'd be better of with an opencl binding / analogue of webcl.

Le's say you write a multiplayer game server, you'll probably setup a virtual server to host it, so it can scale independently of other apps.

A virtual server usually has only one core... but if it had 2 cores :) then what?

If your game gets even little success (on a global scale), with a single processor, even the most powerful you can find, it won't scale.
You will need to assign more processors to your virtual server to successfully scale.

The most powerfull singlethread CPU would kill the dual core. This of course depends also on the algorithem. How you chose your devide and conquer paradigm. The incoming threads need to be scheduled. Thats why APUs are considered to have a normal CPU working as Co-Processor. The GPU calculates the threads. That is really a big deal. Multithreading delivers insane calculation capability. But you'd use something like OpenCL for that and for certain sophisticated tasks. 

As you pointed out, you'd need to assign more processors. A big problem is to share big data between clients. If you are not in the real time area you would go with something like hadoop, mongodb or what I find is sneaky, a mysql database with memcached plugin. 

Still, if you'd want to make huge calculations? If you assign you CPU for those tasks, you finish it. Or would you calculate a Imagetransformation on you CPU while your server app is supposed to spool/stream an insane amount of clients? I don't think so. 

The server is supposed to handle efficiently. So you calculate the amount of clients you are able to serve with one instance. And yes maybe you'd have several threads for incoming data. Still I think its better to utilize a single core instead of spreading threads all over the place. There are some cases where you'd apply massive calculation divisioning. But... as said opencl would be better. A realtime gameserver depends more on the smart pattern you use for shrinking the exchanged data. Also how you split tasks. Still then if you exchange data you need to message them.

 

So yes, multi-threading IS a big deal, specially in server-side.

With Dart its possible to have multiserver multiclient apps. The big question is how you implement cluster realtime operations.....

 

[Filipe Morgado]

Quinta-feira, 23 de Maio de 2013 18:12:54 UTC+1, mezoni escreveu:

>> But I think we need another approach for concurrent real-time applications.

In such cases, I recommend to get acquainted with the already accumulated experience.

From: http://en.wikipedia.org/wiki/Concurrent_programming

Concurrent interaction and communication.

Explicit communication can be divided into two classes:

1. Shared memory communication 

2. Message passing communication

>> Shared-state task/thread queue are kind of a standard for high performance.

Can you explain what you mean by that?

1. Shared-state

2. Thread queue

Shared memory requires the application of some form of locking (e.g., mutexes, semaphores, or monitors) to coordinate between threads.

Is that what you want?

Yes, it's what I want.

Without shared memory, I would have to use a DB server as a shared state-holder, because DB servers CAN perform true concurrency.
So we end-up assigning heavy work to apps written in another language because Dart can't.

...

There are many setups to handle concurrent connections.

Single-threaded
Only one Isolate. Works great, we can access everything we need. The only file descriptors are the sockets.
But it doesn't scale and wastes 3/4 of a quad core, so ...

Dumb Isolates
For every message that comes in (from clients that are already connected), we spawn a new Isolate. We cannot share any state, so we end-up implementing  our shared-state in a DB that is accessed by the isolates. Isolates spawning can be quite time/memory consuming, so ...

Isolates Pool
Same as dumb Isolates but whenever an Isolate completes its works, it warns the main thread so the worker can be cached and re-used.
We can get pretty ugly really fast as we still may have thousands of Isolates, which can add quite a big memory overhead.
Besides, the main thread having a message-passing mechanism to all other isolates (I'm assuming they're file descriptors), we end-up with thousands of file descriptors not counting the actual client connections, so it can impact IO quite a bit, both in time and memory consumption.

Isolates Pool with Task Queue
We only spawn the number of Isolates that our machine can actually run concurrently (4 Isolates for a quad core). Little memory overhead and few file descriptors.
If there's no available Isolates when we get a client request, we send the request to the queue. When a worker isolate completes its work, it sends a message to the main Isolate indicating that it's done and may be cached. If there's a request in the queue, it's passed to the worker, otherwise the worker is cached (in the isolate pool).
The problem is ... there's no shared state. The main Isolate would have to hold the shared state are query/update it when workers request it to. At the same time, the main Isolate needs to handle all the incoming messages. Big overload for the main.
Another problem is ... when a request is passed to a worker isolate, we don't know how much async operations it will perform, we can't modify the whole structure when we add a game plugin that performs heavy IO. Since workers cannot get more work until they're done with their current request, they could be blocked most of the time, wasting CPU cores. They could spawn sub-isolates but it could quickly go out of control and it would eliminate the gain of having a isolates pool that matches the number of cores.
So it's still a no-go. We end-up implementing our shared state in a DB server, because Dart can't handle concurrency.

Thread Pool with Task Queue (shared-memory)
This architecture is difficult to setup but, when done right, surpasses all others, both in speed and memory consumption.
The task queue would probably conflict with Dart's Futures.
The idea is to have a single thread that handles all native events, updating the corresponding tasks. We have 2 queues of Tasks, one with Tasks that are ready to be performed, one with Tasks that are waiting on async operations.
When a request arrives, the main thread creates a Task and adds it to the ready Task Queue, The next available worker is notified that a ready Task is available. The worker removes the Task from queue and processes it until an async operation is performed. The Task goes to the waiting Task queue. The Worker checks if there are available ready Tasks. If so, get one of them and repeats the cycle. Otherwise, it adds itself to the thread pool, waiting to be notified by the main thread. When an async operation completes, the main thread moves the Task from the waiting queue to the ready queue. When it's picked up by a worker, its execution is resumed where it stopped.
All queries/updates to the thread pool, ready Task queue and waiting task queue require locks.

I have never setup such an architecture so there may be a lot of errors in the way I see it, and I'm not sure how native events could be handled in a safe way.

One thing I'm sure of: If correctly setup, nothing matches its performance.

[Peter Jakobs]

With transferable arrays which may land in dart at some point (and please soon) you can send large amount of bytedata across isolates with zero copies (it just transfers the pointer to the array)

[Gen]

I am no hardware expert.

But I think that in modern computers, the bottleneck is memory (load/store) and not the CPU (data computation).

Message passing is no replacement for shared data.

Am Donnerstag, 23. Mai 2013 19:13:49 UTC schrieb mezoni:

>> So we end-up assigning heavy work to apps written in another language because Dart can't.

No problem here. Run any number of isolates as you wish.

Each isolate will be run in parallel (may be in different thread).

But there will be no shared memory.

Your isolates can communicate with each other through messages.

You may create your own message service and use it as shared object.

Inside each isolate code executed without concurrency with one common state.

[Gen]

I think this thread is mainly about server performance with the real Dart VM.

 
Am Donnerstag, 23. Mai 2013 19:22:01 UTC schrieb mezoni:

>> With transferable arrays which may land in dart at some point (and please soon) you can send large amount of bytedata across isolates with zero copies (it just transfers the pointer to the array)

In javascript in web browser also?

[Filipe Morgado]

Quinta-feira, 23 de Maio de 2013 20:13:49 UTC+1, mezoni escreveu:

>> So we end-up assigning heavy work to apps written in another language because Dart can't.

No problem here. Run any number of isolates as you wish.

Each isolate will be run in parallel (may be in different thread).

But there will be no shared memory.

Your isolates can communicate with each other through messages.

You may create your own message service and use it as shared object.

Inside each isolate code executed without concurrency with one common state.

Not a solution. Blindly spawning isolates is terrible for performance. If you pool them, they will be blocked if they perform any IO.
There is currently no way in Dart to implement a truly efficient concurrent system.

[Peter Jakobs]

Am Donnerstag, 23. Mai 2013 21:22:01 UTC+2 schrieb mezoni:

>> With transferable arrays which may land in dart at some point (and please soon) you can send large amount of bytedata across isolates with zero copies (it just transfers the pointer to the array)

In javascript in web browser also?

sure

http://updates.html5rocks.com/2011/12/Transferable-Objects-Lightning-Fast 

 

[Justin Fagnani]

On Thu, May 23, 2013 at 12:02 PM, Filipe Morgado <pix...@gmail.com> wrote:

Quinta-feira, 23 de Maio de 2013 18:12:54 UTC+1, mezoni escreveu:

>> But I think we need another approach for concurrent real-time applications.

In such cases, I recommend to get acquainted with the already accumulated experience.

From: http://en.wikipedia.org/wiki/Concurrent_programming

Concurrent interaction and communication.

Explicit communication can be divided into two classes:

1. Shared memory communication 

2. Message passing communication

>> Shared-state task/thread queue are kind of a standard for high performance.

Can you explain what you mean by that?

1. Shared-state

2. Thread queue

Shared memory requires the application of some form of locking (e.g., mutexes, semaphores, or monitors) to coordinate between threads.

Is that what you want?

Yes, it's what I want.

Without shared memory, I would have to use a DB server as a shared state-holder, because DB servers CAN perform true concurrency.
So we end-up assigning heavy work to apps written in another language because Dart can't.

It's not that Dart can't, it's that Dart has an actor-like concurrency model, but there hasn't been a lot of emphasis on writing the necessary frameworks to use it easily yet, the focus has been on web apps so far.

We will see data structures or services that live in isolates that will provide things like work queues, fork/join frameworks, key/value stores, etc. There's no need to go to an external DB just to share state across isolates, you can store the state in another isolate.

 

--

[Ladislav Thon]

It's not that Dart can't, it's that Dart has an actor-like concurrency model, but there hasn't been a lot of emphasis on writing the necessary frameworks to use it easily yet, the focus has been on web apps so far.

It's also that isolates are still pretty heavy. Erlang shows that actors can be super-awesome, if they are super-cheap. Right now, isolates in Dart are neither.

LT

[Alex Tatumizer]

http://updates.html5rocks.com/2011/12/Transferable-Objects-Lightning-Fast 

From what they say, it follows that processor can copy memory internally  at the rate of about 200MB/sec.
This doesn't make sense. Try the following experiment: create 200 MB file (note: FILE, not memory block), and read it in dart. It will take 0.3 sec on lousy hardware.


It all depends what exactly you are doing. If you copy array to isolate so that each byte gets processed there by some algo, you may not notice any difference from copying (if "copy" implemented correctly and uses most efficient instructions). Throughput of computations can well be by 100 times lower then copying.
But if you need your isolate to access, say, some huge hashtable, and most of entires will not be used, then you lose a lot by transferring the whole thing there.
There are many other variables and moving parts that can affect potential slowdown or lack thereof.

 

 

[Ladislav Thon]

>> It's also that isolates are still pretty heavy. Erlang shows that actors can be super-awesome, if they are super-cheap. Right now, isolates in Dart are neither.

Heavy on which?

1. Creation, initialization

2. Consume many memory

3. Long time for switching context

As far as I know, at least on 1 and 2. Not sure about 3 -- I believe that isolates are currently mapped to native threads in 1:1 fashion, so that can probably be considered heavy too.

 

There are two kind of isolates.

1. spawn uri

2. spawn function

Both heavy?

No difference here.

Only speaking about the standalone VM.

LT

[Greg Lowe]

One other approach which would be nice to see in Dart is the ability to bind multiple socket servers to the same port. These could be in a different isolate or a different process. This allows the kernel to make the work sharing decisions. The kernel generally has more information to all more balanced work sharing.

See SO_REUSEPORT which landed in the 3.9 linux kernel.

https://lwn.net/Articles/542629/

Shall I file a ticket for this? It's low priority and platform specific, but would be a good option for Dart on the server side.

On Fri, May 24, 2013 at 12:58 AM, Matthew Butler <butler....@gmail.com> wrote:

This is generally the much easier method to use, as you can also allow Apache/Nginx/etc to serve the static content much easier, and only forward your ajax calls for instance. This is the path I'm working towards for my apps. To get a 'comfortable' proxy configuration to allow me to write less dart server code, so I only need to worry about the logic stuff and not the static content which is much better optimized than what I want to work on for each of my projects.

On Thursday, May 23, 2013 7:17:44 AM UTC-3, Greg Lowe wrote:

Another approach is to run multiple Dart HttpServer processes behind Apache or Nginx configured as a reverse proxy. I understand this is typically how node.js is deployed in production systems.

[Peter Jakobs]

Am Donnerstag, 23. Mai 2013 21:47:26 UTC+2 schrieb Alex Tatumizer:

http://updates.html5rocks.com/2011/12/Transferable-Objects-Lightning-Fast 

From what they say, it follows that processor can copy memory internally  at the rate of about 200MB/sec.
This doesn't make sense. Try the following experiment: create 200 MB file (note: FILE, not memory block), and read it in dart. It will take 0.3 sec on lousy hardware. 

The file is simply a typed array, also keep in mind that test there was in a time when typed arrays were not saved in V8 directly. And workers are a bit different from isolates. So some if the VM guys can maybe tell how fast this transfer can be.

[James Wendel]

Any dart:io features like this would also need to work on Windows and OSX.  You could open a bug, but it may not be feasible to implement on other platforms.

Also, from a comment from Anders earlier in this very thread said the following:

"To overcome this limitation, we'd like to introduce the ability to transfer Sockets between isolates, so e.g. a HttpServer can pass requests to worker isolates."

So that would be enough to solve the problem, correct?

[Alex Tatumizer]

It's very difficult to theorize about potential bottlenecks without experiments.

By what I see, computation is a bottleneck, not data transfer of any kind (unless you are using very old networks or disk drives)

Here's something to think about:

Database can give you 10MB/sec easily (not memcached or something - just plain db).

Decode result set, encode it in JSON (no other processing). You will be lucky to to it at a rate of 10MB/sec.

A priory belief that bottleneck is always somewhere (not in your program) might be a misconception remaining from the old days.

[Gen]

The solution depends on the problem which depends on the other solution.
If your solution is having all game data as shared data (maybe object coordinates for MMO), then isolates might not be the solution even with transferable sockets.

[Filipe Morgado]

Quinta-feira, 23 de Maio de 2013 21:44:43 UTC+1, Gen escreveu:

The solution depends on the problem which depends on the other solution.
If your solution is having all game data as shared data (maybe object coordinates for MMO), then isolates might not be the solution even with transferable sockets.

Exactly.

And if I need to broadcast a change to all on the same map, the sharing isolate would need access to all connections and make sure no other isolate writes anything before we're done.

So we're stuck with one isolate with a heavy load (connections, data-sharing, broadcast), and all other isolates are only there to parse requests and spam the main isolate with messages.

Wonder if it wouldn't be better single-threaded.

[Justin Fagnani]

Remember that you can send a SendPort around, so you don't need one main isolate doing all the shared bits. The main isolate only needs to set things in motion, it can give worker isolates SendPorts to other service isolates.

Isolate-based programs become distributed systems, and you'll need to think like that if you're setting up a complex program with raw isolate APIs. You wouldn't have a single master machine handling all database, caching, network I/O, etc., and you probably won't want a single isolate doing that either. The complexity involved is why we'll need libraries that make this much simpler to set up.

Wonder if it wouldn't be better single-threaded.

--

[Søren Gjesse]

Filed https://code.google.com/p/dart/issues/detail?id=10843.

When we have the ability of distributing sockets from a server socket to several isolates this might be used to optimize this behind the scenes.

Regards,

Søren

[Ladislav Thon]

> >> It's also that isolates are still pretty heavy.
>

> Right. And always will be.

Not necessarily. Isolates in Dart will probably never be as cheap as actors in Erlang (overhead of cca 300 bytes), but they are supposed to be quite lightweight. In https://code.google.com/p/dart/issues/detail?id=5006, Lars says "It will take some time before isolates in the VM are minimal (a few KB)". If we get the overhead down to few kBs, then we are fine.

> Because for every new isolate cloned the entire global context.

You can't do anything with that. If you want to run large programs in all isolates, you can't have a lot of them. However, if you want to run a lot of small programs, each in its own isolate, then currently, the overhead of isolates will easily dominate the entire price of the system. Look at the commit message of https://code.google.com/p/dart/source/detail?r=23115 which implemented diet parsing of classes -- they have some encouraging numbers there for the initial memory usage.

> The threads are very lightweight

Surely you must be kidding here? The overhead of threads is pretty huge. Sure, modern OSes can maintain tens of thousands of threads, but if you try to use so many threads in your program, then the price of context switch will kill you. On the other hand, you can easily have a million of actors in an Erlang program without a sweat (and I think that Akka will give you similar numbers on the JVM, but it's a long time since I looked at the numbers).

Of course threads and actors are not directly comparable, they are two different worlds, but still.

LT

[Ladislav Thon]

>> If we get the overhead down to few kBs, then we are fine.

I also love to dream.

Why do you link to other people's dreams?

It's not a dream, it's a goal. From the very beginning, Dart people stated that they want isolates to be lightweight. They are not today, and they won't be in the near future, but they will be. Given that there has been very little focus on the server-side, it's not a surprise.

 

>> If you want to run large programs in all isolates, you can't have a lot of them.

You not understand. This is not my problem that I want.

This is a problem of 'isolate' model. They so constructed. They not effective in this case.

If in large program I want run a lot of 'small task' in parallel then we have great overhead.

This is because each 'isolate' clone entire context of all program.

The 'isolate' based on that principle. It always create another 'big program' in each case.

No it doesn't. You don't have to clone the entire program (spawnFunction), you can run a completely new one (spawnUri).

LT

[Ladislav Thon]

>> No it doesn't. You don't have to clone the entire program (spawnFunction), you can run a completely new one (spawnUri).

You are sure?

This example run task in parallel, It use 'spawnFunction'.

Each task (not task but isolate) create 'new big program'.

Could you please quote the full context next time? I was speaking about something completely different.

You said it always create another 'big program' in each case, and that's simply not true, because you can spawn entirely new programs (big or small, what have you) using spawnUri.

LT

[Ladislav Thon]

Please explain me about what you speaking... that not required cloning of all constants and global variables.

In the part that mentions spawnUri... which, funnily enough, you've already decided several times to ignore.

LT

[Alex Tatumizer]

Did we start petitioning for support of data: uri yet? We discussed it briefly, but I can't find open issue for this. 

. 

--

[Ladislav Thon]

You want to say if 'isolate' created in 'spawnUri' the code that executed in this isolate does not have global variables (execution context)?

No. I want to say that if you spawn an isolate using spawnUri, it can be a completely different program with a completely different set of global variables, classes, functions etc. It is completely independent of the spawning one.

LT

[Ladislav Thon]

Did we start petitioning for support of data: uri yet? We discussed it briefly, but I can't find open issue for this. 

This one looks relevant: https://code.google.com/p/dart/issues/detail?id=10733 I'm starring it right now.

LT

[Alex Tatumizer]

Starred it too. Curiously, data url suggestion was added  just a couple of days  ago.

On Sat, May 25, 2013 at 12:55 PM, mezoni <andrew...@gmail.com> wrote:

>> This one looks relevant: https://code.google.com/p/dart/issues/detail?id=10733 I'm starring it right now.

Eval?

[Filipe Morgado]

Sábado, 25 de Maio de 2013 6:38:58 UTC+1, mezoni escreveu:

>> It's also that isolates are still pretty heavy.

Right. And always will be.

Because for every new isolate cloned the entire global context.

(On Linux at least) I think the memory pages are only copied when they get dirty (written to by the parent Isolate) or when they are written to from the child Isolate.

Which means that if an isolate is run and only performs write operations on a few memory pages, only those pages are copied, which may be +retty fast.

(Of course, take this with a grain of salt, as I don't know the internal details and the GC handling might touch stuff all over the place.)

[Justin Fagnani]

On Sat, May 25, 2013 at 9:46 AM, Ladislav Thon <lad...@gmail.com> wrote:

Did we start petitioning for support of data: uri yet? We discussed it briefly, but I can't find open issue for this. 

This one looks relevant: https://code.google.com/p/dart/issues/detail?id=10733 I'm starring it right now.

There was an older almost identical one in http://dartbug.com/5187

I just closed 10733 as a dupe and updated the title of 5187. You probably want to start 5178 now.

[Justin Fagnani]

On Sat, May 25, 2013 at 2:05 PM, Justin Fagnani <justin...@google.com> wrote:

On Sat, May 25, 2013 at 9:46 AM, Ladislav Thon <lad...@gmail.com> wrote:

Did we start petitioning for support of data: uri yet? We discussed it briefly, but I can't find open issue for this. 

This one looks relevant: https://code.google.com/p/dart/issues/detail?id=10733 I'm starring it right now.

There was an older almost identical one in http://dartbug.com/5187

I just closed 10733 as a dupe and updated the title of 5187. You probably want to start 5178 now.

Never mind, the issue tracker merges starts. Neat.