Are Actor/Promise tests correct?

[Stefan Marr]

Hi:

I am a little confused by the actor tests.

Let’s take this example: (btw. I think the access modifiers are not yet updated. I would expect that #nestedResolutionWithDepth: needs to be public.)

class DeepChain = () (
public nestedResolutionWithDepth: n = (
^ n > 0 ifTrue: [ self <-: nestedResolutionWithDepth: n - 1 ]
ifFalse: [ 'done' ]
)
)

public testAsyncDeeplyChainedResolution = (
| r r2 |
r := (actors createActorFromValue: DeepChain) <-: new.
r2 := r <-: nestedResolutionWithDepth: 100000.
^ assert: r2 resolvedWith: 'done'.
)


So, the tests creates an actor for the DeepChain mixin, instantiates the class, and then does an eventual send of #new.
This eventual send returns the promise `r`, which at some point will resolve to a far reference.
But before that can happen, we do another eventual send to this promise (#nestedResolutionWithDepth:), which results in another promise `r2`. This promise is then given to #assert:resolvedWith:, which registers a #whenResolved: handler, in which the resolution of `r2` is compared to the string ‘done’.

Now, my understanding is that `r2` would resolve to a promise, which is the result of the recursive `self <-: nestedResolutionWithDepth: n - 1`.
I would expect that this chain of promises has to be traversed manually. Perhaps, I am miss reading the promise implementation or the specification, but I didn’t see anything that looks like it is forwarding the #whenResolved: handler in case promises are nested.

Should it do that? Could you point me at the part I am missing?

Thanks
Stefan

--
Stefan Marr
Johannes Kepler Universität Linz
http://stefan-marr.de/research/

[Ryan Macnak]

The intention is indeed that nested resolution are flattened. Resolving a promise to another promise is like an asynchronous tail call. This isn't specified anywhere, but this is the behavior in E, and we are largely following E's model.

In Nikolay's implementation this happens in Pastime`Resolver resolve:; the handlers themselves aren't forwarded but the promise becomes an observer of its partial resolution. In my implementation this happens in Actors2`InternalResolver doForwardingTo: and the handlers themselves are forwarded.

[Stefan Marr]

Hi Ryan:

> On 30 Jul 2015, at 03:49, Ryan Macnak <rma...@gmail.com> wrote:
>
> The intention is indeed that nested resolution are flattened. Resolving a promise to another promise is like an asynchronous tail call. This isn't specified anywhere, but this is the behavior in E, and we are largely following E's model.
>
> In Nikolay's implementation this happens in Pastime`Resolver resolve:; the handlers themselves aren't forwarded but the promise becomes an observer of its partial resolution. In my implementation this happens in Actors2`InternalResolver doForwardingTo: and the handlers themselves are forwarded.

Great, thanks for the pointer, that helped!

The next issue I got is:

public fibonacci: n = (
n = 0 ifTrue: [ ^ 0 ].
n = 1 ifTrue: [ ^ 1 ].
^ (self <-: fibonacci: n - 1) <-: + (self <-: fibonacci: n - 2)
)

What mechanism unwraps the promise of the righthand side #fibonacci: send in this case?

In the end we got something like `a Promise` <-: + `a Promise`

Is that discussed somewhere?

Thanks again

[Ryan Macnak]

It isn't unwrapped by the promise implementation. It should work because of the double dispatching that lets one mix integers, floats, fractions etc. But the implementation might not be there yet. I think this particular test is disabled.

<Promise[Integer]> + <Promise[Integer]>

<Integer> + <Promise[Integer]>

<Promise[Integer]> addFromInteger: <Integer>

<Integer> addFromInteger: <Integer>

 

[Stefan Marr]

Hi Ryan:

> On 01 Aug 2015, at 03:45, Ryan Macnak <rma...@gmail.com> wrote:
>
> It isn't unwrapped by the promise implementation. It should work because of the double dispatching that lets one mix integers, floats, fractions etc. But the implementation might not be there yet. I think this particular test is disabled.
>
> <Promise[Integer]> + <Promise[Integer]>
> <Integer> + <Promise[Integer]>
> <Promise[Integer]> addFromInteger: <Integer>
> <Integer> addFromInteger: <Integer>

Ah, I see :)

What’s the intension here generally?
Should all values implement such unwrapping semantics, or is this only meant to be convenience for number types?
And which type of operations should be considered?

If you guys already got a little bit of experience in that regard, I would appreciate to hear opinions.

Thanks

[Ryan Macnak]

On Sat, Aug 1, 2015 at 12:39 AM, Stefan Marr <goo...@stefan-marr.de> wrote:

Hi Ryan:

> On 01 Aug 2015, at 03:45, Ryan Macnak <rma...@gmail.com> wrote:
>
> It isn't unwrapped by the promise implementation. It should work because of the double dispatching that lets one mix integers, floats, fractions etc. But the implementation might not be there yet. I think this particular test is disabled.
>
> <Promise[Integer]> + <Promise[Integer]>
> <Integer> + <Promise[Integer]>
> <Promise[Integer]> addFromInteger: <Integer>
> <Integer> addFromInteger: <Integer>

Ah, I see :)

What’s the intension here generally?
Should all values implement such unwrapping semantics, or is this only meant to be convenience for number types?
And which type of operations should be considered?

If you guys already got a little bit of experience in that regard, I would appreciate to hear opinions.

We haven't used wrappers much, but it ought to be enough that the primitives are aware of wrappers in their failure code, both async promises and the lazy-evaluation wrappers used for simultaneous slots.

[rob withers]

Hi Ryan,

I am just starting to look into Newspeak and I found Actors. Yay! You mention a different implementation. How could I get a hold of Actors2? Is there a repository process I need to pursue?

I was also curious how I might run the ActorTests. 

Best,

Robert

[Ryan Macnak]

In boot images starting at 2015-08-28, a descendent of Actors2 is the only implementation. Its tests are ActorsTestingConfiguration.

[Gilad Bracha]

To expand a bit: There have been 4 Actor implementations, and the current one is the chosen one. You could find the old stuff in the repo's history, but you probably don't want it.

[Robert Withers]

That's excellence! Please bear with a bunch of questions about this Actors. I am relieved these is a dynamic language with this feature. .

Does Newspeak Actors support remote sending? I see the FarRefMirror and several message mirrors.  What is the difference between PromiseMessage, PipielineMessage and ChainMessage?

I am unfamiliar with the test harness. How can I run these tests?

Best,

Robert

[Robert Withers]

That's quite an evolution and I am excited about the code I am seeing. It looks familiar to me.

I managed to understand the differences of the various Messages, I think, at least to the first approximation. I have curiosity about the difference between PromiseMessages and EventLoopMessages, the latter being where the heavy lifting lay. PromiseMessages seem to be msg forwardewrs, while eventLoopMessages are pending or dispatching on the eventloop.  Is it so?

I managed to find MinitestUI as a launch point for testing. Looks as if I need to run Hopscotch, so I'll dive into more reading you've worked hard to supply. Thank you.

Regarding remote sending, I have some experience implementing in this area (RemotePromise/Far Handlers/Resolvers/Redirectors/OTables/PresentationSession Stack) and I would really love to help. How could I? Will I need to connect to a repository?

Does Newspeak have wrappers to an NIO library?

Best,
Robert

[Gilad Bracha]

Hi Robert,

I'll let Ryan respond wrt to the details of the Actor implementation. Maybe I can help with some of the other questions.

On Thu, Sep 17, 2015 at 5:08 PM Robert Withers <robert.w...@gmail.com> wrote:

That's quite an evolution and I am excited about the code I am seeing. It looks familiar to me.

I managed to find MinitestUI as a launch point for testing. Looks as if I need to run Hopscotch, so I'll dive into more reading you've worked hard to supply. Thank you.

So the tutorial discusses how to use Minitest, as doe sthis blog post:

http://gbracha.blogspot.com/2011/02/ministry-of-nesting-testing.html

Does Newspeak have wrappers to an NIO library?

Newspeak has an FFI called Aliens that lets you call out to C.  Take a look at the Aliens namespace (from the home page, follow the Newspeak Source link to see the list of namespaces), and at the Smalltalk class Alien.

Some discussion can be found here

http://gbracha.blogspot.com/2008/12/unidentified-foreign-objects-ufos.html

http://gbracha.blogspot.com/2008/08/foreign-functions-vm-primitives-and.html

 

[robert]

Hi Gilad,

Thank you for your guidance.

On 9/17/2015 8:35 PM, Gilad Bracha wrote:

So the tutorial discusses how to use Minitest, as doe sthis blog post:

http://gbracha.blogspot.com/2011/02/ministry-of-nesting-testing.html

I like.

Does Newspeak have wrappers to an NIO library?

Newspeak has an FFI called Aliens that lets you call out to C.  Take a look at the Aliens namespace (from the home page, follow the Newspeak Source link to see the list of namespaces), and at the Smalltalk class Alien.

Some discussion can be found here

http://gbracha.blogspot.com/2008/12/unidentified-foreign-objects-ufos.html

http://gbracha.blogspot.com/2008/08/foreign-functions-vm-primitives-and.html

NIO would need to use the callback feature, as well, I'd think. If I could call Java, that's be really ideal. Is there a Java binding?

Thank you,
Robert

[Gilad Bracha]

Hi Robert,

Alas, there is no Java binding. Talking to Java is complicated what with JNI and all. I guess it could be done, but it would be slow and probably flakey. Something that worked well would be very nice to have.  However, we have almost no resources, so we need to focus very carefully. 

Having Newspeak run on a JVM is another direction that has been brought up many times, but again, it is messy. Stefan Marr has done SOMns, a version that runs on Truffle, but on a standard JVM. And it is lacking certain key features as well. 

But tell me, what did you want to call Java for?

[Robert Withers]

On Thursday, September 17, 2015 at 9:01:59 PM UTC-4, Gilad Bracha wrote:

 

But tell me, what did you want to call Java for?

 Here's a dropbox link to the java code: https://www.dropbox.com/s/ve2d6zub8q971im/pauwau-0.3.1.zip?dl=0. It's broken.

best

Robert

[Ryan Macnak]

On Thu, Sep 17, 2015 at 4:17 PM, Robert Withers <robert.w...@gmail.com> wrote:

That's excellence! Please bear with a bunch of questions about this Actors. I am relieved these is a dynamic language with this feature. .

Does Newspeak Actors support remote sending?

Currently no. But the intent is to support this with something like E's CapTP.

 

I see the FarRefMirror and several message mirrors.  What is the difference between PromiseMessage, PipielineMessage and ChainMessage?

You're not running the latest image if you see those classes, which were part of Actors1. FWIW, PipelineMessage was an eventual send to a promise and ChainMessage was a "when".

Regarding remote sending, I have some experience implementing in this area (RemotePromise/Far Handlers/Resolvers/Redirectors/OTables/PresentationSession Stack) and I would really love to help. How could I? Will I need to connect to a repository?

You can fork https://bitbucket.org/newspeaklanguage/newspeak_bleeding_edge.

 

Does Newspeak have wrappers to an NIO library?

We are generally lacking IO libraries. We do have a file library inherited from Strongtalk, but it was not written with asynchrony, access control or ephemerons in mind. For the time being, we mostly use Squeak's.

[robert]

On 9/17/2015 11:34 PM, Ryan Macnak wrote:

OYou're not running the latest image if you see those classes, which were part of Actors1. FWIW, PipelineMessage was an eventual send to a promise and ChainMessage was a "when".

Oh, that's a challenge. How can I get the latest image as this came with the latest all-in-one. I think.

Regarding remote sending, I have some experience implementing in this area (RemotePromise/Far Handlers/Resolvers/Redirectors/OTables/PresentationSession Stack) and I would really love to help. How could I? Will I need to connect to a repository?

You can fork https://bitbucket.org/newspeaklanguage/newspeak_bleeding_edge.

Once again, a challenge. Allow me to continue the tutorial.

 

Does Newspeak have wrappers to an NIO library?

We are generally lacking IO libraries. We do have a file library inherited from Strongtalk, but it was not written with asynchrony, access control or ephemerons in mind. For the time being, we mostly use Squeak's.

I'll set that as a learning project, then. I'd love to have ns semantics: scala-like in a dynamic sense.

Thanks for ns and thanks for you.

best
Robert

[Gilad Bracha]

Robert,

Latest images are in:

https://8a616521eaba88639fe08345a88fa288e26a6b34.googledrive.com/host/0B7VkTjJg2xYDNHphX3JSNFFsd1E/bootimages/

Get the latest dated version when you decide to do this. The tutorial discusses access to the repo via MemoryHole.  Let us know if you have issues. 

[robert]

Hi Gilad,

Thank you for your help.  I've made some progress, however I am still unable use these bootimages.  I am running the 12-1-2014 Spur VM that came with the default distribution.  Do I need a more recent VM?

Using the 101 image from 12-1-14, I did manage to connect to the newspeak repository and had connected to newspeak_bleeding_edge but can no longer. I crashed the VM trying so...out of memory. Of course I was loading bot repositories, so that may be in play.

- Robert

[Gilad Bracha]

Hi Robert,

Sorry for the difficulties.  It's called bleeding_edge for a reason :-(

a. Yes, you need a newer VM. The latest VMs are at http://www.mirandabanda.org/files/Cog/VM/VM.r3427/. I myself am using 3397; I should update.

b.  The latest images are based off of Squeak 5.0. This turns out to cause a few annoying UI glitches that we haven't yet figured out how to resolve, but these are minor.

c. The very latest VM version, 3427, has a couple of gotchas that should be resolved in the next rev. I particular, you need to manually include the Squeak 5 sources in your directory.

d. I assume you are using a mac or linux.  I think there is an issue using MemoryHole on Windows (Ryan will correct me if I'm wrong on this,  or anything else listed here).

e. The tutorial is  probably out of sync with this latest version.

We should put together a nice package again; one reason we haven't is because Squeak 5 is causing some grief as I said.  I hope you can bear with us and get past these problems. We really could use more contributions.

[robert]

Hi Gilad, thanks for the pointers, most welcome and helpful. I'll be glad to contribute as I get going. I managed to get SqueakElib loaded with crypto, though I am having some file write issues. I'll need to understand porting then I could get going.

Regarding your points:
a) Which VM file do I need and how do I get it into play?
b) no worries
c) Do I grab the normal squeak distribution to get the v5 sources?
d) I am on Windows. Perhaps I could help debug the Windows MemoryHole.
e) I've gotten a lot from the tutorial, and I linked to the newspeak repo. Ns is sooo sweet!  Thanks!

- Robert

[robert]

Is there an nsvm spur MT Windows 64-bit VM I could use? To tackle NIO I'd like non-blocking FFI calls.

best

Robert

On 9/18/2015 10:31 AM, Gilad Bracha wrote:

[Gilad Bracha]

a) Since you are on Windows,  you should get a version that works well there. I'm not sure exactly how far back to go, but I'll experiment and get back to you.

c) Yes.

d) I think the problem is known, and is in the VM. 

Re 64 bits. Ryan knows the status. I think FFI is more problematic on 64 bit.

[robert]

Gilad,

I got the 3427 VM running here.  I had to use the latest image file you pointed me to. I was able to load my elib code, though there are file issues with my tracemonitor code.

I did run into the out-of-memory bug and am attaching the crash.dmp file.

best
Robert

[Gilad Bracha]

Hi Robert,

I'm not sure what the timeline for a fix for that is. I'm cc'ing Eliot. 

Eliot: I believe this is a known issue of the VM crashing in MemoryHole on Windows.

Robert: I'll try and figure out whether an older VM would be a better choice or not.

[robert]

Thank you, Gilad. My concern would be to get the latest Actors model, whichever VM+image works best.

[Gilad Bracha]

Robert,

It looks like you hit a different bug than what I thought. Hopefully, you can actually make progress with the configuration you have, which is the latest we have. I'm told you don't need the Squeak sources file anymore - it is now packaged in the VM again.  I think you can hold off using MemoryHole for a while, and in the meantime I hope the bugs involving its use will be fixed.

[robert]

I am getting some crashes, but I can work in either environment. For now I am in 14-12-1 investigating FFI. Did FFI change between 14 and 15?

Still, all I need to do is call win32API I believe.  I am attaching the ws2_32.dll and the winsock2.h files. If I can manage to initialize and load that dll, I can make some progress. Does anyone know how I could do that?

So my approach is bottom up: nio, startup state machine, encoding framework: this will be the session &transport layers. The top side of the presentation layer can target any Actor model.

I was curious - it looked like the Actors in 15 had a wrapper. That shouldn't be needed with become: and possible inferencing would be more complex.

best
Robert

[robert]

Gilad, Is there a place I can document my looks into NIO for Ns? I have some links that could be shared.

- Robert

On 9/18/2015 8:54 PM, Gilad Bracha wrote:

[Gilad Bracha]

On Fri, Sep 18, 2015 at 8:03 PM robert <robert.w...@gmail.com> wrote:

I am getting some crashes, but I can work in either environment. For now I am in 14-12-1 investigating FFI. Did FFI change between 14 and 15?

Not in any significant way. The main development in Newspeak in 2015 has been the enforcement of access control. 

The new VMs will prevent calls to protected methods from outside the object, and to private ones from outside object and the lexical scope.  

Still, all I need to do is call win32API I believe.  I am attaching the ws2_32.dll and the winsock2.h files. If I can manage to initialize and load that dll, I can make some progress. Does anyone know how I could do that?

I haven't had a chance to look, but if you go to 

http://www.newspeaklanguage.org/documents-and-slides

You will find links to The Newspeak FFI User Guide and Internals. 

 

[Gilad Bracha]

On Fri, Sep 18, 2015 at 8:09 PM robert <robert.w...@gmail.com> wrote:

Gilad, Is there a place I can document my looks into NIO for Ns? I have some links that could be shared.

 Github? 

[robert]

Yes, I found snippets at bitbucket.  I hope that's ok.

[robert]

On 9/19/2015 12:24 AM, Gilad Bracha wrote:

I haven't had a chance to look, but if you go to 

http://www.newspeaklanguage.org/documents-and-slides

You will find links to The Newspeak FFI User Guide and Internals. 

 

Thank you,
Robert

[Gilad Bracha]

Yes, that's fine too.