I implemented generator functions and fibers for writing async codes. try it out!

[Munir Hussin]

I've just released my moon-core library. It's been around for a long time, but I've finally open sourced it after removing some nda related stuff. moon-core has a number of stuff like BigInteger, random number generation algorithms, templating and routing, and other stuff. Basically the common stuff from various work projects gets dumped into moon-core.

Anyway, I've recently implemented a feature I've been longing for in haxe, which is the yield expression, so we can write readable asynchronous code. It behaves like the yield keyword in JavaScript, Python, and C#. So long as there's a @yield within a function, that function will be transformed into a resumable function. It allows you to easily write iterators.

After having generator functions, we can also now have fibers, an alternative to threads. Fibers use cooperative-multitasking, while threads use preemptive-multitasking. The yield expressions will be where the fiber takes a break and lets the next fiber resume running.

On single-threaded environments, this can be useful if you need to have long running codes and expensive computations without locking up the UI.

The async macro implementation is not perfect, and there's still some issues that needs to be solved. For example, I'm totally clueless on how make try/catch work. For other simple usage, it works okay.

Check it out here:

https://github.com/profound7/moon-core

If you're wondering how it works, this gist shows the different transformation passes for a fibonacci function:

https://gist.github.com/profound7/0408725cc82068a3019ad1a3b7beacf6

It's also on haxelib as moon-core

Contributions are welcomed, and I hope someone finds this useful!

[Juraj Kirchheim]

The scope of the project is a bit overwhelming (at least for me), but there's some really impressive stuff in there! :)

--
To post to this group haxe...@googlegroups.com
http://groups.google.com/group/haxelang?hl=en
---
You received this message because you are subscribed to the Google Groups "Haxe" group.
For more options, visit https://groups.google.com/d/optout.

[Philippe Elsass]

That's a lot of code :)
Seq looks really nice, but the generator work is crazy! A bit too crazy even, it should be added to the language.

Philippe

[underscorediscovery]

I haven't looked at the project here much but for interest sake,
caue waneck implemented a fully functional (try/catch included) yield as well in Unihx.

https://github.com/unihx/unihx/blob/development/src/unihx/pvt/macros/YieldGenerator.hx
I've used it to implement coroutines successfully before (but didn't get to finishing a usable scheduler).

[Munir Hussin]

Yeah, its a bit crazy. It would be good if haxe has native support for coroutines. I haven't learned enough OCaml to tinker with the haxe compiler.

Thanks for the heads up on Unihx. I came across that lib and thought it was unity only, and didn't realize there's a yield implementation inside it.

I think I now have a rough idea on how to make yields work within try/catch. And I'm about 40% sure that I could make switch cases with variable capture work within a generator function (I need to identify which EConst(CIdent(x)) in switch cases are variable captures, and declare them at the top).

[Chii Chan]

Awesome stuff! I really like how the fib() example shows the steps of each transformation.

[Cauê Waneck]

Hey there! Great work :)

I'm the one who did the generator implementation on unihx. How does your implementation work? Does it transform it into a state machine?

I'm very happy to see this being implemented. This is something I've wanted to do for a long time, but never actually did it. I'll check it out, for sure!

About try/catch, I've added a `handleError` to the main interface, https://github.com/unihx/unihx/blob/development/src/unihx/utils/YieldBlockBase.hx#L37 , so that it could handle asynchronous errors. You can check out in the tests https://github.com/unihx/unihx/blob/development/tests/src/unihx/tests/cases/YieldTests.hx how these were supposed to work.

As for pattern matching, I ended up using typed expressions exactly because of pattern matching support. It's too hard to replicate exactly how pattern matching works in order to make sure you know all the correct variables. There was a problem with transforming the typed expressions back into normal expressions, but this shouldn't be a problem anymore with Context.storeTypedExpr

In order to have it to compile on latest Haxe, I had to change some things. You can see the result at https://gist.github.com/waneck/c4451ce7a5d2d85bfeff68b37fe189c9 . 

As for native generators in Haxe, this is something I've been thinking about, but we need to make a strong case for it and libraries like yours really help to show the usefulness of it.

2016-06-23 7:20 GMT-03:00 Chii Chan <sangoh...@gmail.com>:

Awesome stuff! I really like how the fib() example shows the steps of each transformation.

[Munir Hussin]

Thanks. My implementation does transform a function into a state machine. It does so by making multiple expr transformation passes to "compile" the expressions into a flattened code. This is done by transforming control structures into an equivalent one by adding meta expressions @label x and @goto x. This is done only for expressions containing a yield expression.

Control structures are transformed into a block with an if/switch that contain a @goto. So a while loop would become something like:

{
  @label start_loop;
    if (!$cond) @goto end_loop;
    $body;
    @goto start_loop;
   
  @label end_loop;
    $void;
}

The void at the end is to indicate that the EBlock does not return anything, otherwise the value of last expression is returned allowing you to do weird things like x = while (cond) body;

After this step, all the EBlocks are merged into a single EBlock, and each @label becomes an integer case in a switch and each @goto changes the internal state. I didn't generate a class, and instead I generated a nested function. You can see the different transformation passes here: https://gist.github.com/profound7/0408725cc82068a3019ad1a3b7beacf6

Thanks for the heads up on typed expressions! I've never used it before, and I'll look into it. From your codes, I see that there's a TLocal(v) with a v.capture. I'll explore this more, and switch over to typed expressions.

[Cauê Waneck]

Actually v.capture is so you know if the variable was captured by a function. The important part about typed expressions is TLocal(v) - v has a unique id. This way you can deal with variable shadowing and it's an easy way to mark e.g. which variables were used across more than one state. With that, you don't have to worry about where each variable came from (e.g. from a pattern match, or a for statement, etc)

--