es7 vs core.async

[Marc Fawzi]

This is an extension to a reply to a Reagent user about running multiple async ops in parallel and executing some handler when all finish.

I am wondering if my hunch is correct that CSP is a necessary abstract model for async processes that require complex coordination and/or communication but that it is too much in simple cases like when multiple xhr requests to the server in parallel and running a handler when they all finish passing it the responses as an array. The proof is in the pudding and despite CSP being far more flexible and powerful I can't help but think that for certain cases it is not the ideal solution which brings up my previous question to the list: why call it core.async when in facf it is core.csp? Maybe I'm wrong. If I am show me how to write a simple one line of code in core.async to implement the aforementioned simple case.

Here is a copy of my reply on the Ragent list to a user wishing to implement the simple case (too lazy to type it again)

"So far, to my understanding, core.async is a CSP implementation that adheres to the Go implementation of CSP. My use of it in JS land is via async-csp (CSP implemented on top of es7 async functions) and has been mostly as a factorization tool, to make pretty and digestible those async patterns that involve coordination and/or communication between async processes.

For example, in JS, if you need to run something when N async processes complete and don't want to use the silly '.then' approach, and you don't want to use a dynamic counter approach (not fancy enough for you) you could do it in one line of code within an es7 async function with something like "await Promise.all([asyncOp1, asyncOp2, asyncOp3])" and that is all it takes, definitely more concise that using a CSP pattern. It is try-cath-able, too."

Show me the money! No seriously, what am i being an idiot about here? I feel like es7 is giving us more options with async functions and CSP is just a subset of the abstract models that can be implemented via es7's async functions, which I believe are just syntax sugar on top of native Promise.

Speaking of Promise, there is a proposal to remove the need for accessing the Promise API in the case presented above using await* (with *)

Please educate.

Thank you

Marc



Sent from my iPhone

[Johann Bestowrous]

At a high level, I think it's pretty important to note that you are comparing a language spec to a library.

[Marc Fawzi]

Well the title gives that impression and I regret having chose to do that :)

But if you read the content i am asking the question if Async functions in es7 can be used to build a performant and faithful version of CSP (github: aysnc-csp) and also be useful for common tasks like the simple server request scenario I mentioned then why wouldnt we want to think of CSP as just one pattern not the One True Pattern for async. Right now core.async is being used and or recommended for everything async and I am asking if that is ideal and if CLJS can allow itself to grow beyond this one particular pattern when it comes to async. The first thing would be renaming core.async to core.csp and promoting choice when it comes to async patterns. As it is right now, every time someone has an async design problem core.async is recommended as a solution regardless of whether or not it's the best fit solution. If you have a hammer...

That's the scope. Not es7 vs core.async and I'm sorry for the stupid title.

Sent from my iPhone

> On Sep 15, 2015, at 8:32 AM, Johann Bestowrous <johann.b...@gmail.com> wrote:
>
> At a high level, I think it's pretty important to note that you are comparing a language spec to a library.
>

> --
> Note that posts from new members are moderated - please be patient with your first post.
> ---
> You received this message because you are subscribed to the Google Groups "ClojureScript" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to clojurescrip...@googlegroups.com.
> To post to this group, send email to clojur...@googlegroups.com.
> Visit this group at http://groups.google.com/group/clojurescript.

[Shaun LeBron]

ES7 vs core.async (gist):
https://gist.github.com/shaunlebron/d231431b4d6a82d83628

[Marc Fawzi]

Thanks for that!

async function baz() {
  await* [foo(), bar()];
}

(defn baz []
  (go
    (doseq [c [(foo) (bar)]]
      (<! c))))

With the core.async case you have to define the channel c, right? 

It looks cryptic compared to the es7 version. Like "go" what does go mean, seriously? I mean in terms of its English language context. Go does not convey async. And what the heck is <! Are we using bash or something? Some kind of inverted redirection? 

I guess you can have macros that would make it look just as comprehensible as the es7 async version so people coming into CLJS won't be turned off by the crazy looking syntax and the exposed low level semantics. Maybe a bunch of core.async macros that expose common use cases in a way that anyone can understand without even having to understand CSP basics. 

In my team, everyone gets the es7 version of things but despite having been CLJS users for 6 months now, no one understands how to use core.async. I've had to play with it in different languages before I realized how powerful it is to have in your toolset to manage complex (potentially dynamic) coordination patterns between async processes but our use cases in the UI have yet to beyond the very simple use cases your gist shows which are (without use of macros) much easier to understand using es7 async functions.

If macros can solve the "comprehensibility" problem for the common use cases then maybe something that would provide es7 async like library for cljs that gives you defnasync and await 

Syntax and semantics can then be so simple while the underlying system remains so powerful and in that case you could have core.async be bundled with those macros thus allowing easy access to common async patterns without the Go syntax obfuscating things and making it seem complicated as well as too noisy syntax wise for the most common tasks 

Sent from my iPhone

[Andrey Antukh]

Hi!

I think you are comparing apples with oranges. CSP and async/await can't be compared directly. Async/await works with a promise (one value) abstraction and csp works with channel abstraction (sequence).

It seems is an anti-pattern use channels as promises because them does not has the notion of error. I remember that Timothy Baldridge have said something similar about this:

"A sort of anti-pattern I see a lot is creating a lot of one-shot channels and go blocks inside every function. The problem, as you see is that this creates a lot of garbage. A much more efficient plan is to stop using core.async as a RPC-like system, and start using it more like a dataflow language: Identity data sources and sinks, and then transform and flow the data between them via core.async. 

It's interesting to note that core.async started as something that looked a lot like C#'s Async/Await, but that was dropped in favor of CSP pretty quickly. So there's reasons why the language isn't optimized for this sort of programming style. "

Source: https://groups.google.com/d/msg/clojure/57ig0si3gUM/vRr-T1IaebUJ

Without the intention to make spam, the funcool/cats (https://github.com/funcool/cats) `mlet`  macro does something similar in semantics that async/await does. It there some examples using the ES6/7 compatible promise library: http://funcool.github.io/promesa/latest/#sugar-syntax

The advantage about this solution is that is generic and can be extended to other async related abstractions as:

- JDK8 CompletableFuture's (https://github.com/funcool/promissum/blob/master/doc/content.adoc#26-promise-chaining) 

- manifold deferred (https://github.com/funcool/cats/blob/master/doc/content.adoc#82-manifold-deferred)

- core.async channels (https://github.com/funcool/cats/blob/master/doc/content.adoc#81-channel)

Personally, I use core.async to compose different processes, but when I interacting with async apis I almost always use promise abstraction with cats sugar syntax. The promise abstraction semantics fits more properly in async rpc calls that channels because it represents a "eventually available value" and has the notion of error (unlikely core.async channels). 

Regards.

Andrey

Andrey Antukh - Андрей Антух - <ni...@niwi.nz>

http://www.niwi.nz

https://github.com/niwinz

[Marc Fawzi]

Very educational. Thank you. Re Apples and Oranges. I had a sense of that l, but it does not mean that many people are not using core.async in the way Shaun documented in the gist. On many occasions, i have seen recommended by others the use of core.async for everything async including scenarios where it is not ideal. I think calling it core.csp would make it more clear as far as what its purpose is. I am not entirely sure about error handling with core.async but  I've seen examples of async try/catch in that context. I see async/await, which is syntax sugar in es7 on top of Promise, as a more universal tool, which can be used to provide CSP functionality (see async-csp on github) or Promise like functionality (but much nicer to use than Promise) 

Sent from my iPhone

[Shaun LeBron]

Thanks for the insight, Andrey.

It looks like async/await was originally slated for Clojure, but it later became core.async using CSP instead:
http://dev.clojure.org/display/design/Async+blocks

This seems to imply that, yes, the purpose of async/await as used by C#, Python, and ES7 is fulfilled by core.async. As for the difference with Promises containing the notion of an error, it looks like David created a simple pattern for that here:
http://martintrojer.github.io/clojure/2014/03/09/working-with-coreasync-exceptions-in-go-blocks/

So, I'm still confused about the intended usages, especially after reading Baldridge's comment. It may just imply that these patterns are still being worked out.

> (<! c))))With the core.async case you have to define the channel c, right? It looks cryptic compared to the es7 version. Like "go" what does go mean, seriously? I mean in terms of its English language context. Go does not convey async. And what the heck is <! Are we using bash or something? Some kind of inverted redirection? I guess you can have macros that would make it look just as comprehensible as the es7 async version so people coming into CLJS won't be turned off by the crazy looking syntax and the exposed low level semantics. Maybe a bunch of core.async macros that expose common use cases in a way that anyone can understand without even having to understand CSP basics. In my team, everyone gets the es7 version of things but despite having been CLJS users for 6 months now, no one understands how to use core.async. I've had to play with it in different languages before I realized how powerful it is to have in your toolset to manage complex (potentially dynamic) coordination patterns between async processes but our use cases in the UI have yet to beyond the very simple use cases your gist shows which are (without use of macros) much easier to understand using es7 async functions.If macros can solve the "comprehensibility" problem for the common use cases then maybe something that would provide es7 async like library for cljs that gives you defnasync and await Syntax and semantics can then be so simple while the underlying system remains so powerful and in that case you could have core.async be bundled with those macros thus allowing easy access to common async patterns without the Go syntax obfuscating things and making it seem complicated as well as too noisy syntax wise for the most common tasks 

[Robin Heggelund Hansen]

So, core.async draws a lot of inspiration from Go (the programming language). It is a more flexible model for async programming than a simple async/await as you find in C#.

You can emulate C# async/await by thinking of channels as tasks (every function that returns a task in C# is awaitable).

(defn async-get [url]
(let [c (channel)]
(ajax/get url {:when-done #(>!! c %)}) ;; >!! puts something into a channel outside of a go-block
c))

(go
(<! (async-get "some/url"))) ;; <! receives a value inside of a go-block, just as await does inside an async function

The code above would be the exact same in C#, except in C# you have async functions in the standard library, so you rarely need to make your own "async-get" function. Channels do become more intereseting in combination with alt and timeout.

(go (alt!
(async-get "some/url") ([val] val)
(timeout 1000) ([_] "failed")))

The above code will return "failed" unless our async-get call completes within a second, if that happens it will return the result of async-get. We can have many more channels within our alt! call, and only one value is exposed from one channel.

In the end, core.async is about making async code easy to reason about. Use it as async/await if you wish, or use it like in Go.

[Robin Heggelund Hansen]

Also worth nothing, that go, <! and >! are taken from Go.

[Marc Fawzi]

I can also imagine a parallel universe where async/await was adopted for Clojure/Script and someone built a CSP library for Clojure/Script based on it. The syntax as well as the semantics for the raw async/wait (as opposed to any CSP implementation based on it) is so much more concise and comprehensible than core.async syntax and semantic for the simple use cases where we currently (in JS es6, Java, Scala and other languages) use Promises/Futures. For the CSP case, we can have CSP based on async/await (see async-csp on github) without the crazy Go inspired syntax.

And when you say that CSP covers all async scenarios you neglect to acknowledge that for some scenarios async/await would be far simpler, especially if you add async error handling to it! I can't believe that this is too hard to see that. Or am I missing something?


Sent from my iPhone

[Marc Fawzi]

An async/await example of the simple cases where we'd normally use Promise/Future

async function() { try { await* [asyncOp1(), asyncOp()2, ...] } catch (e) { .... } }

As far as I know, with CSP you have to setup a channel and then implement an async try/catch.

So given that we can build CSP model using async/await (see async-csp on github) then I would say that async/await is the more universal base. CSP can be seen as a higher level abstraction.

Sent from my iPhone

[Val Waeselynck]

Adding my grain of salt:

Le mercredi 16 septembre 2015 22:42:19 UTC+2, Andrey Antukh a écrit :
> Hi!
>
>
> I think you are comparing apples with oranges. CSP and async/await can't be compared directly. Async/await works with a promise (one value) abstraction and csp works with channel abstraction (sequence).
>
>
> It seems is an anti-pattern use channels as promises because them does not has the notion of error.

Actually, I like error management better with core.async than with Promises / Monads, because with little effort you can use the same error constructs for synchronous and asynchronous code: https://gist.github.com/vvvvalvalval/f1250cec76d3719a8343

I do agree that promises are a more natural fit for RPC systems, because of their signal-like nature. I feel I can't really avoid RPC for building web apps, I'd be glad to know about other strategies.

I remember that Timothy Baldridge have said something similar about this:
>
>
> "A sort of anti-pattern I see a lot is creating a lot of one-shot channels and go blocks inside every function. The problem, as you see is that this creates a lot of garbage. A much more efficient plan is to stop using core.async as a RPC-like system, and start using it more like a dataflow language: Identity data sources and sinks, and then transform and flow the data between them via core.async. 
> It's interesting to note that core.async started as something that looked a lot like C#'s Async/Await, but that was dropped in favor of CSP pretty quickly. So there's reasons why the language isn't optimized for this sort of programming style. "
>
>
> Source: https://groups.google.com/d/msg/clojure/57ig0si3gUM/vRr-T1IaebUJ
>
>
>
> Without the intention to make spam, the funcool/cats (https://github.com/funcool/cats) `mlet`  macro does something similar in semantics that async/await does. It there some examples using the ES6/7 compatible promise library: http://funcool.github.io/promesa/latest/#sugar-syntax
>
>
> The advantage about this solution is that is generic and can be extended to other async related abstractions as:
> - JDK8 CompletableFuture's (https://github.com/funcool/promissum/blob/master/doc/content.adoc#26-promise-chaining) 
> - manifold deferred (https://github.com/funcool/cats/blob/master/doc/content.adoc#82-manifold-deferred)
> - core.async channels (https://github.com/funcool/cats/blob/master/doc/content.adoc#81-channel)
>
>
> Personally, I use core.async to compose different processes, but when I interacting with async apis I almost always use promise abstraction with cats sugar syntax. The promise abstraction semantics fits more properly in async rpc calls that channels because it represents a "eventually available value" and has the notion of error (unlikely core.async channels). 
>
>
> Regards.
> Andrey
>
>
> On Wed, Sep 16, 2015 at 10:30 PM, Marc Fawzi <marc....@gmail.com> wrote:
>
>
>
>
> Thanks for that!
>
> async function baz() {
> await* [foo(), bar()];
> }
> (defn baz []
> (go
> (doseq [c [(foo) (bar)]]

> (<! c))))With the core.async case you have to define the channel c, right? It looks cryptic compared to the es7 version. Like "go" what does go mean, seriously? I mean in terms of its English language context. Go does not convey async. And what the heck is <! Are we using bash or something? Some kind of inverted redirection? I guess you can have macros that would make it look just as comprehensible as the es7 async version so people coming into CLJS won't be turned off by the crazy looking syntax and the exposed low level semantics. Maybe a bunch of core.async macros that expose common use cases in a way that anyone can understand without even having to understand CSP basics. In my team, everyone gets the es7 version of things but despite having been CLJS users for 6 months now, no one understands how to use core.async. I've had to play with it in different languages before I realized how powerful it is to have in your toolset to manage complex (potentially dynamic) coordination patterns between async processes but our use cases in the UI have yet to beyond the very simple use cases your gist shows which are (without use of macros) much easier to understand using es7 async functions.If macros can solve the "comprehensibility" problem for the common use cases then maybe something that would provide es7 async like library for cljs that gives you defnasync and await Syntax and semantics can then be so simple while the underlying system remains so powerful and in that case you could have core.async be bundled with those macros thus allowing easy access to common async patterns without the Go syntax obfuscating things and making it seem complicated as well as too noisy syntax wise for the most common tasks 

[Marc Fawzi]

<<

;; OPTION 3: using Promises
	;; If you're in ClojureScript, and not interested in core.async, you can just use a Promise library:
	;; - funcool/promesa is a ClojureScript wrapper of the popular Bluebird JavaScript library.
	;; - jamesmacaulay/cljs-promises is a Promise library designed to operate nicely with core.async.
	;; Promises take care of both asynchrony and error management (they're essentially a mix of Futures and Exception Monads); some may say it's convenient, others may argue it's not simple.

>>

Is there a way to use Javascript's new 'async functions' from within cljs? I read that Closure compiler can transform es6/7 syntax to es5, given some command line flags. 

[Marc Fawzi]

<<

Is there a way to use Javascript's new 'async functions' from within cljs? I read that Closure compiler can transform es6/7 syntax to es5, given some command line flags. 

>>

I meant in a way that integrates nicely with ClojureScript?

The conclusion I'd like to validate is if async/await is a more universal pattern than CSP because it is designed for the most simple and common async scenarios but scales up to the more complex CSP scenario, whereas CSP is the other way around: designed for the most complex async coordination scenarios but is often applied to the most simple scenarios.

[Andrey Antukh]

BTW, http://spion.github.io/posts/es7-async-await-step-in-the-wrong-direction.html

[Marc Fawzi]

While he has a good point to make about the given scenarios, his complaining about loss of generality with async/await compared to generators and promise api on which async/await is built is misplaced. Loss of generality is the obvious consequence of trying to hit the most common use cases with the simplest possible syntax and semantics... i.e. you end up losing true generality. 

The argument I wanted to validate here is if you can implement the very common and simple async coordination scenarios with async/await much more clearly and suitably than with CSP and if CSP itself as a model can be implemented/built using async/await (for more complicated coordination scenarios) then it seems to me that async/await would make a more universal basis for concurrency management than starting with CSP. 

[Andrey Antukh]

On Sat, Oct 3, 2015 at 11:15 AM, Marc Fawzi <marc....@gmail.com> wrote:

While he has a good point to make about the given scenarios, his complaining about loss of generality with async/await compared to generators and promise api on which async/await is built is misplaced. Loss of generality is the obvious consequence of trying to hit the most common use cases with the simplest possible syntax and semantics... i.e. you end up losing true generality. 

The argument I wanted to validate here is if you can implement the very common and simple async coordination scenarios with async/await much more clearly and suitably than with CSP and if CSP itself as a model can be implemented/built using async/await (for more complicated coordination scenarios) then it seems to me that async/await would make a more universal basis for concurrency management than starting with CSP. 

I understand and I'm pretty convinced that csp as is, is a concurrency model and can be implemented in different ways. I think that the current async/await approach is slightly coupled to the promise. 

Obviously if you decouple it from promise, a csp can be implemented using async/await. In fact, async/await is just a sugar syntax. The main error of the ES7 async/await is that is hard coupled with promises.

With the link that I have shared in previous email, I mean that generators are more versatile and also offers the same features that async await. CSP like abstraction is easily can be implemented on top of generators/coroutines: https://github.com/ubolonton/js-csp.

Furthermore, in funcool/cats[1], we have implemented an other way to do it with more clojure like syntax: `mlet` and `alet` macros. They looks very familiar because the aspect is like a normal clojure let. The difference is that them are implemented in terms of abstractions.

Imagine this example function:

async function doFoo() {

  const x = await doFirstThing()

  const y = await doSecondThing();

  return x + y;

}

With cats abstraction, you do not need any compiler support for implement async/await, just use a familiar syntax. This is equivalent code in clojure/clojurescript that does exactly the same thing:

(require '[cats.core :as m])

(defn do-foo

  []

  (m/mlet [x (do-first-thing)

           y (do-second-thing)]

    (m/return (+ x y))))

But both, the async/await, and mlet macro has one disadvantage: the code is inherently serial. But in a lot of circumstances we can optimize the execution just finding dependencies. You can observe that for calculate x we do not need any thing from previous execution, and the same for the y. With `alet` macro you can optimize this kind of cases:

(defn do-foo

  []

  (m/alet [x (do-first-thing)

           y (do-second-thing)]

    (+ x y)))

In this code example the operations of x and y are executed in paralel. And this example uses promises, but we can use core.async channels, manifold deferreds, jdk8 computable futures, rxjs observables...

And in future we can add a generalization macro that can look similar to this:

(defn do-foo''

  []

  (m/async [x (m/await (do-first-thing))

            y (m/await (do-second-thing))

            z 2]

    (+ x y z))))

Also, using the same abstractions...

This that, I intend to say that in clj/cljs we have the power to build more powerful abstractions that ES7 async/await.

Regards.

Andrey

[1]: https://github.com/funcool/cats

[Marc Fawzi]

I knew about cats from this mailing list,  and I went over examples showing the use of mlet but did not come across any examples of alet, so thank you for the whole explanation. Are the awaits try-catchable (assuming the function being called can throw)? 

Innovations start in languages like Java, Clojure, Haskel, C# and Elm et al and make their way into JS eventually but what I was suggesting is "core.async" should probably be expanded beyond CSP alone as as many new comers to CLJ/S, such as myself and a few other relative beginners, often think that core.async is the basis for all async stuff you can do in CLJ/S, which as you are explaining is not true, and it takes a while to discover libraries such as cats.

Will be definitely playing with the cats patterns you presented here and going over it in more detail.

Also, have you seen this? https://github.com/dvlsg/async-csp ... It is CSP style channels implemented with ES7 async/await ... 

  

thank you for this!

 

[Andrey Antukh]

Sorry for my late response.

On Sat, Oct 3, 2015 at 7:02 PM, Marc Fawzi <marc....@gmail.com> wrote:

I knew about cats from this mailing list,  and I went over examples showing the use of mlet but did not come across any examples of alet, so thank you for the whole explanation. Are the awaits try-catchable (assuming the function being called can throw)? 

Yes and No. At this moment if some of the called function throws an exception it just catched by the promise chain and shortcicuited returning a rejected promise. But is very easy define functions for branching that will behave in a similar way to the try/catch blocks.

Let see an example using promesa library:

(require '[promesa.core :as p])

(m/mlet [x (-> (fn-that-return-promise)

               (p/catch some-error-handler-fn))

         y (some-other-fn)]

  (do-things x y))

 

Innovations start in languages like Java, Clojure, Haskel, C# and Elm et al and make their way into JS eventually but what I was suggesting is "core.async" should probably be expanded beyond CSP alone as as many new comers to CLJ/S, such as myself and a few other relative beginners, often think that core.async is the basis for all async stuff you can do in CLJ/S, which as you are explaining is not true, and it takes a while to discover libraries such as cats.

You are completely right here! I've also made the same mistake in the past using core.async for every thing that is related to async... 

 

Will be definitely playing with the cats patterns you presented here and going over it in more detail.

Also, have you seen this? https://github.com/dvlsg/async-csp ... It is CSP style channels implemented with ES7 async/await ... 

Yes, I know about it. But as I have said previously, the main problem with async/await is that is bound directly with promise that is limiting part (promises are not cancelable by design, the execution model is bound to the promise execution model, is not switchable, I guess that it does more memory allocations that using just generators). 

I don't say that is bad but I think that it is not the best scenario for good csp implementation (obviously the usage is very comfortable and has less syntax friction..). Calling it more general th

  

thank you for this!

Thanks to you for open this interesting mailing thread ;)