BREAKING CHANGE: dart:isolate has been refactored.

[Florian Loitsch]

What changed?

We performed a major refactoring of dart:isolate (the last "dart:" library that needed a big overhaul).

The stream-based classes (IsolateSink, IsolateStream, and MessageBox) have been removed.

ReceivePort now implements Stream and as a consequence the "receive" method has been replaced by "listen".

SendPort.call has been removed.

SendPort.send only takes one argument (and no replyTo port anymore).

ReceivePort.toSendPort() has been changed to the getter ReceivePort.sendPort.

spawnFunction and spawnUri have been replaced by Isolate.spawn and Isolate.spawnUri respectively.

Both spawn versions take an initial message that is passed to the entry-point. 'spawnUri' also takes the List<String> arguments for "main".

We have added RawReceivePort which does not implement Stream and does not respect Zones. It is currently only implemented for the VM.

The global port, and stream variables have been removed.

 

Who is affected?

All users of dart:isolate.

How do I update my code?

Example:

// == old.

void echo() {

  port.receive((msg, replyTo) {

    replyTo.send(msg);

    if (msg == "bar") port.close();

  });

}

main() {

  SendPort sendPort = spawnFunction(echo);

  sendPort.call("foo").then((msg) {

    print("received: $msg");

    return sendPort.call("bar");

  }).then((msg) {

      print("received another: $msg");

  });

}

//===

is converted to (for example):

// == new.

void echo(SendPort initialReplyTo) {

  var port = new ReceivePort();

  initialReplyTo(port.sendPort);

  port.listen((msg) {

    var data = msg[0];

    SendPort replyTo = msg[1];

    replyTo.send(data);

    if (data == "bar") port.close();

  });

}

Future sendReceive(SendPort port, msg) {

  ReceivePort response = new ReceivePort();

  port.send([msg, response.sendPort]);

  return response.first;

}

main() {

  var response = new ReceivePort();

  Future<Isolate> remote = Isolate.spawn(echo, response.sendPort);

  remote.then((_) => response.first).then((sendPort) {

    sendReceive(sendPort, "foo").then((msg) {

      print("received: $msg");

      return sendReceive(sendPort, "bar");

    }).then((msg) {

      print("received another: $msg");

    });

  });

}

//===

Note: for frequent communication one should not allocate a new ReceivePort for every message but rather have a dispatching ReceivePort.

For spawnUri:

// == old file echo.dart.

void main() {

  port.receive((msg, replyTo) {

    replyTo.send(msg);

    port.close();

  });

}

// == old file start.dart

main() {

  SendPort sendPort = spawnUri("echo.dart");

  sendPort.call("foo").then((msg) {

    print("received: $msg");

 });

}

//===

can be transformed to:

// == new file echo.dart.

void main(List<String> args, SendPort replyTo) {

  replyTo.send(args[0]);

}

// == new file start.dart

main() {

  var response = new ReceivePort();

  Future<Isolate> remote =

      Isolate.spawnUri(Uri.parse("echo.dart"), ["foo"], response.sendPort);

  remote.then((_) => response.first)

    .then((msg) { print("received: $msg"); });

}

//===

Why did this change happen?

The dart:isolate library was the last "dart:" library that needed a big overhaul.

We decided to keep the core primitives (spawning, sending and receiving) simple. In future iterations we intend introduce important functionality to the Isolate class.

When will the change take effect?

The change was committed moments ago.

--
Give a man a fire and he's warm for the whole day,
but set fire to him and he's warm for the rest of his life. - Terry Pratchett

--
For more news and information, visit http://news.dartlang.org/
 
To join the conversation, visit https://groups.google.com/a/dartlang.org/

[Florian Loitsch]

Small, but important, additional comment: ReceivePorts are not automatically garbage-collected when nobody can send to them anymore (Dart does not have a cross-isolate GC). You should therefore treat ReceivePorts like resources and close them when they aren't used anymore.

In the examples from my last mail I didn't do that when the isolate creation failed. It didn't matter there, because the program would have shut down anyway, but that's just because I simplified the examples.

See updated examples inline.

  remote.then((_) => response.first)

      .whenComplete(response.close) // close when complete.

      .then((sendPort) {

    sendReceive(sendPort, "foo").then((msg) {

      print("received: $msg");

 We are completing the response if there is an error, or if there was a message from the other side. In the latter case we are closing the receivePort after it has already been canceled, but that's valid and doesn't hurt.

  remote.then((_) => response.first)

    .whenComplete(response.close)

    .then((msg) { print("received: $msg"); });

Same as before: close if there was an error, or if we got a message.

 

Why did this change happen?

The dart:isolate library was the last "dart:" library that needed a big overhaul.

We decided to keep the core primitives (spawning, sending and receiving) simple. In future iterations we intend introduce important functionality to the Isolate class.

When will the change take effect?

The change was committed moments ago.

--
Give a man a fire and he's warm for the whole day,
but set fire to him and he's warm for the rest of his life. - Terry Pratchett

--
Give a man a fire and he's warm for the whole day,
but set fire to him and he's warm for the rest of his life. - Terry Pratchett

[lihui]

maybe I am wrong. but  I really think it is hard to use.

scala actor is much better. and the active object mentioned by  Gilad Bracha is also  wonderful. 

 

在 2013年10月26日星期六UTC+8下午6时52分40秒，Florian Loitsch写道：

[Lars Bak]

Correction, please use:

  initialReplyTo.send(port.sendPort);

in the initial example Florian listed in his post.

//Lars

--
For general Dart discussion, consider posting to mi...@dartlang.org
To file a bug report or feature request, go to http://www.dartbug.com/new
---
You received this message because you are subscribed to the Google Groups "Dart Project Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to eng+uns...@dartlang.org.
For more options, visit https://groups.google.com/a/dartlang.org/groups/opt_out.

[Florian Loitsch]

We expect users to use libraries on top of the dart:isolate primitives. The dart:isolate library is as simple as possible while still allowing sophisticated libraries on top of it.

On Sat, Oct 26, 2013 at 1:15 PM, lihui <ustc....@gmail.com> wrote:

--

For general Dart discussion, consider posting to mi...@dartlang.org
To file a bug report or feature request, go to http://www.dartbug.com/new
---
You received this message because you are subscribed to the Google Groups "Dart Project Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to eng+uns...@dartlang.org.
For more options, visit https://groups.google.com/a/dartlang.org/groups/opt_out.

--
Give a man a fire and he's warm for the whole day,
but set fire to him and he's warm for the rest of his life. - Terry Pratchett

[Erik Grimes]

Hi Florian,

I see in the documentation that Isolate.spawn functions returns an "Isolate" instance that can be used to control the isolate, but I don't see any control functions, just a SendPort for the isolate.  Am I missing something?  What do you do if an isolate hangs? How do you get rid of it?

Thanks,

Erik

[tomaszkubacki]

I would love to have something like:

Isolate.spawnCode('//dart expression comes here'); 

for REPL like tools + eval.

I know eval is evil but could be useful eg. to build something like this: http://msdn.microsoft.com/en-us/library/vstudio/ms734631(v=vs.90).aspx

cheers,

Tomek

[Florian Loitsch]

On Tue, Oct 29, 2013 at 2:06 PM, Erik Grimes <erik....@gmail.com> wrote:

Hi Florian,

I see in the documentation that Isolate.spawn functions returns an "Isolate" instance that can be used to control the isolate, but I don't see any control functions, just a SendPort for the isolate.  Am I missing something?  What do you do if an isolate hangs? How do you get rid of it?

We will add more functionality over time. Currently there is no way. 

--

For general Dart discussion, consider posting to mi...@dartlang.org
To file a bug report or feature request, go to http://www.dartbug.com/new
---
You received this message because you are subscribed to the Google Groups "Dart Project Development" group.
To unsubscribe from this group and stop receiving emails from it, send an email to eng+uns...@dartlang.org.
For more options, visit https://groups.google.com/a/dartlang.org/groups/opt_out.

[Jos Hirth]

I'm really unhappy with the current state of Isolates. The simplified Hello World example requires a half a screen of hard to follow boilerplate spaghetti code. To make matters worse, there doesn't seem to be a way to hide any of that.

The kind of thing I'd like to see:

main() {
  var monkey = new MyMonkey();
  monkey.start('foo').then((result) {
    // do something with the result
  });
}

class MyMonkey extends Monkey {
  run(message) {
    // return something
  }
}

[William Hesse]

Dart Isolates are fundamentally different from Java threads, and other multithreaded OO languages, in that they do not share any objects or global state with each other.

This is important because every Isolate only runs a single thread, and all your Dart code can assume it is single-threaded, and no locking or synchronizing of objects is needed.

This makes Dart code easier to write, and also makes the Dart VM much faster than a VM that supports multiple threads working on the same object.

The current API has everything needed to implement any API you want for starting and communicating with Isolates, though, so you are encouraged to write a package that provides the simple API you want, though.  It is just that the arguments you pass to the isolate ('foo' in the example) and the return value from the isolate (result) must be

data types that can be sent over a SendPort, or can be serialized into something that can be sent over a SendPort.

--
For other discussions, see https://groups.google.com/a/dartlang.org/
 
For HOWTO questions, visit http://stackoverflow.com/tags/dart
 

To file a bug report or feature request, go to http://www.dartbug.com/new

To unsubscribe from this group and stop receiving emails from it, send an email to misc+uns...@dartlang.org.

--
William Hesse

[Kathy Walrath]

I'm having a hard time with the isolates samples, too. It'd be nice if we could provide some abstraction—if not in the API, then in the samples.

-k-

[Florian Loitsch]

On Fri, Nov 1, 2013 at 5:51 PM, Kathy Walrath <kat...@google.com> wrote:

I'm having a hard time with the isolates samples, too. It'd be nice if we could provide some abstraction—if not in the API, then in the samples.

When you say "isolates samples" which samples do you mean? 

--

[Kathy Walrath]

[Kathy Walrath]

On Fri, Nov 1, 2013 at 10:50 AM, Florian Loitsch <floi...@google.com> wrote:

On Fri, Nov 1, 2013 at 5:51 PM, Kathy Walrath <kat...@google.com> wrote:

I'm having a hard time with the isolates samples, too. It'd be nice if we could provide some abstraction—if not in the API, then in the samples.

When you say "isolates samples" which samples do you mean? 

The samples in https://www.dartlang.org/docs/dart-up-and-running/contents/ch03.html#ch03-dartisolate---concurrency-with-isolates. I'm converting them (and the rest of that chapter) to the new API.

-k- 

[Jos Hirth]

http://api.dartlang.org/docs/releases/latest/dart_isolate/Isolate.html

States: "The argument entryPoint specifies the entry point of the spawned isolate. It must be a static top-level function or a static method that takes no arguments."

However, static methods do not work.

import 'dart:isolate';
main () {
  Isolate.spawn(Foo.foo, null);
}
class Foo {
  static foo(msg){}
}

Bombs with:

    Unhandled exception:
    Illegal argument(s): Isolate.spawn expects to be passed a top-level function

Anyhow, even if that would work, it wouldn't help with adding some kind of abstraction. You can't override some static method and have that static method be called by some static method of the base class.

I'm either overlooking something (please tell me if I do), or the currently available puzzle pieces just aren't enough to create any kind of abstraction. There doesn't seem to be any way to hide all of that plumbing.

[By the way, the documentation of spawnUri mentions 4 signatures, but only lists 3.]

[Florian Loitsch]

On Sat, Nov 2, 2013 at 6:50 AM, Jos Hirth <google...@kaioa.com> wrote:

http://api.dartlang.org/docs/releases/latest/dart_isolate/Isolate.html

States: "The argument entryPoint specifies the entry point of the spawned isolate. It must be a static top-level function or a static method that takes no arguments."

However, static methods do not work.

import 'dart:isolate';
main () {
  Isolate.spawn(Foo.foo, null);
}
class Foo {
  static foo(msg){}
}

Bombs with:

    Unhandled exception:
    Illegal argument(s): Isolate.spawn expects to be passed a top-level function

That's a VM bug. 

Anyhow, even if that would work, it wouldn't help with adding some kind of abstraction. You can't override some static method and have that static method be called by some static method of the base class.

I'm either overlooking something (please tell me if I do), or the currently available puzzle pieces just aren't enough to create any kind of abstraction. There doesn't seem to be any way to hide all of that plumbing.

You won't be able to hide all the plumbing but it can be reduced to one line.

Something like:

entryPoint(args) => myFramework.entry(args, myRealEntryPoint);

myRealEntryPoint(...) {

}

Then you could write:

myFramework.spawn(entryPoint, .......) and abstract everything else away.

[By the way, the documentation of spawnUri mentions 4 signatures, but only lists 3.]

Thanks. will fix asap. 

--
For other discussions, see https://groups.google.com/a/dartlang.org/
 
For HOWTO questions, visit http://stackoverflow.com/tags/dart
 
To file a bug report or feature request, go to http://www.dartbug.com/new

To unsubscribe from this group and stop receiving emails from it, send an email to misc+uns...@dartlang.org.

--
Give a man a fire and he's warm for the whole day,
but set fire to him and he's warm for the rest of his life. - Terry Pratchett

[Jos Hirth]

Basically, what I'd like to see:

You write a top-level function or static method, which accepts one message argument and which returns some dynamic value. This function doesn't do anything with ports. You only get that message, then you do something, and then you return some result. That's it.

You can easily test thing kind of thing. X goes in, Y goes out. (Well, ideally. If you aren't doing IO.)

For running it in parallel, you'd just spawn it ("spawnWrapped", "spawnWorker", or something), pass some message, and wait for the answer.

high(numberOfFingers) => 'High $numberOfFingers!';

main () {
  Isolate.spawnWorker(high).send(5).then((result) {
    print(result);
  });
}

That's the kind of high-level API, I'd like to use.

[Florian Loitsch]

I guess this would be possible if we could send static/global functions. We will look into that (no promises).

[Justin Fagnani]

On Nov 2, 2013 8:57 PM, "Florian Loitsch" <floi...@google.com> wrote:
>
> I guess this would be possible if we could send static/global functions. We will look into that (no promises).

The inability to wrap the isolate APIs has been a problem from the beginning. Back when I was working on isolate related things I ran into this and talked with Gilad about adding an isStatic or isTopLevel getter to ClosureMirror and making closurization preserve this information.

This way the behavior of only allowing top-level functions wouldn't be an extra-lingual feature.

Justin

> To unsubscribe from this group and stop receiving emails from it, send an email to announce+u...@dartlang.org.

[Lasse R.H. Nielsen]

On Sun, Nov 3, 2013 at 4:57 AM, Florian Loitsch <floi...@google.com> wrote:

I guess this would be possible if we could send static/global functions. We will look into that (no promises).

That would definitely open up some scenarios that aren't possible now, where we can only "send" one function, the one that gets called initially. 

It would make a kind of sense if the things that could be sent were also the things that could be compile-time constants. We already have num, String, bool, null, Lists and Maps. Adding static/top-level functions would fit in fine. 

That just leaves normal const objects, or objects of a class with a const constructor. Would be kind of cool, but not nearly as simple.

/L 'one can dream :)'

 

To unsubscribe from this group and stop receiving emails from it, send an email to announce+u...@dartlang.org.

--
Lasse R.H. Nielsen - l...@google.com  

'Faith without judgement merely degrades the spirit divine'
Google Denmark ApS - Frederiksborggade 20B, 1 sal - 1360 København K - Denmark - CVR nr. 28 86 69 84

[Justin Fagnani]

-eng and announce

On Sun, Nov 3, 2013 at 12:46 PM, Lasse R.H. Nielsen <l...@google.com> wrote:

On Sun, Nov 3, 2013 at 4:57 AM, Florian Loitsch <floi...@google.com> wrote:

I guess this would be possible if we could send static/global functions. We will look into that (no promises).

That would definitely open up some scenarios that aren't possible now, where we can only "send" one function, the one that gets called initially. 

It would make a kind of sense if the things that could be sent were also the things that could be compile-time constants. We already have num, String, bool, null, Lists and Maps. Adding static/top-level functions would fit in fine.

That would be awesome. Right now we're left with trying to look up something by name with ClosureMirror.findInContext().

That just leaves normal const objects, or objects of a class with a const constructor. Would be kind of cool, but not nearly as simple.

The difficulty here is in ensuring that you have the class defined, and with a compatible definition, in the other isolate. If you can check that, or guarantee it like with spawnFunction, then const should work. It'd be nice if there were a shared heap for such objects so that no copies were needed.

And while we're dreaming :) we'd really like to be able to have transferrable objects, especially typed data, and other shared immutable objects like in River Trail. One setup that would be particularly useful for DSP is to be able to create a typed array and segment it into multiple mutable views which could be transferred to other isolates, but only one isolate could hold a reference to a mutable at a time, like Rust's unique types. This would make fork/join over large data possible without copies.

[Lasse R.H. Nielsen]

On Mon, Nov 4, 2013 at 12:40 AM, Justin Fagnani <justin...@google.com> wrote:

-eng and announce

On Sun, Nov 3, 2013 at 12:46 PM, Lasse R.H. Nielsen <l...@google.com> wrote:

On Sun, Nov 3, 2013 at 4:57 AM, Florian Loitsch <floi...@google.com> wrote:

I guess this would be possible if we could send static/global functions. We will look into that (no promises).

That would definitely open up some scenarios that aren't possible now, where we can only "send" one function, the one that gets called initially. 

It would make a kind of sense if the things that could be sent were also the things that could be compile-time constants. We already have num, String, bool, null, Lists and Maps. Adding static/top-level functions would fit in fine.

That would be awesome. Right now we're left with trying to look up something by name with ClosureMirror.findInContext().

That just leaves normal const objects, or objects of a class with a const constructor. Would be kind of cool, but not nearly as simple.

The difficulty here is in ensuring that you have the class defined, and with a compatible definition, in the other isolate. If you can check that, or guarantee it like with spawnFunction, then const should work. It'd be nice if there were a shared heap for such objects so that no copies were needed.

That problem shouldn't be any greater than the problem finding the same top-level function. It still requires the same library to be there at both ends. This is guaranteed for Isolate.spawn (nee spawnFunction), because it has all the same libraries loaded (although not necessarily with the same global state).

The real problem is that a class that has a const constructor may also have non-const constructors.

They only have final fields, and if their field values are all serializable too, then they are effectively constant, even if they are not canonicalized. The problem is with how you create the object again at the other end if there are more than one constructor. I'd really, really prefer to not create the objects without using the constructors. Consider the case where an object from a non-checked mode isolate enters a checked mode isolate, and one of the fields doesn't match - that should fail.

It might not be possible to avoid an extra-language object constructor, but it should still be tried before all other hacks.

/L

[Jos Hirth]

A one-shot Hello World example:

import 'dart:async';
import 'dart:isolate';

// boilerplate wrapper
void wrapFiverize(SendPort agency) {
  var receiver = new ReceivePort();
  agency.send(receiver.sendPort);
  receiver.first.then((portAndValue) {
    portAndValue[0].send(fiverize(portAndValue[1]));
  });
}

// boilerplate runner
Future runFiverize(value) {
  var spawnPort = new ReceivePort();
  return Isolate.spawn(wrapFiverize, spawnPort.sendPort)
      .then((_) => spawnPort.first).then((SendPort agent) {
        var receiver = new ReceivePort();
        agent.send([receiver.sendPort, value]);
        return receiver.first;
      });
}

int fiverize(int n) => n * 5;

main() {
  runFiverize(7).then((res) {
    print(res);
  });
}

It's amazingly difficult to give all those ports a somewhat meaningful name (I cheated and skipped one). I'm not really sure if I succeeded. It's still pretty hard to follow.

Well, if you pretend that the first 2 functions don't exist, it looks kinda nice. ;)

Being able to pass top-level functions around would surely help.

Got 2 more things to nag about:

1. The returned Isolate instance is completely useless. ("The isolate instance can be used to control the spawned isolate.", says the doc, but there isn't a single method.)
   
2. Closing the Isolate's ReceivePort via some secret handshake is just bizarre. What would you use there? Null? Some magical constant?

How do other languages with actor-like concurrency do this stuff?

[Mark]

Awaiting this with eager anticipation.
Loss of the replyTo argument on send() means this now merges this notification (to the isolate) into its general message data/type parsing (curious). I'm guessing there's a wider long-term technical capability reason for this, but it was a nice "free" way of handling this without having to fully pre-parse messages first. I suspect a long-lived rather than one-shot send-receive-close pattern for graceful port handling would be useful to people if it doesn't make the topic too long and scary. I'm pretty sure my attempt (on home page) is not the way its intended to be done!