When will the async/await feature in ES8 be introduced in ClojureScript?

[Philos Kim]

I wonder when the async/await feature in ES8 will be introduced in ClojureScript.

Of course, I know there is core.async in ClojureScript but I hope that the async/await feature in ES8 will be supported in ClojureScript as soon as possible.

Does anyone know when it will be supported?

[Andrew Oberstar]

Not familiar with async/await myself. What would it provides that you can't already do in core.async?

Andrew Oberstar

--
Note that posts from new members are moderated - please be patient with your first post.
---
You received this message because you are subscribed to the Google Groups "ClojureScript" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojurescrip...@googlegroups.com.
To post to this group, send email to clojur...@googlegroups.com.
Visit this group at https://groups.google.com/group/clojurescript.

[Nikita Dudnik]

Sorry for answering with a question but I have to second Andrew. What's wrong with core.async and more importantly how'd you implement async/await in clojurescript in terms of syntax?

-

Nik 

[Shaun LeBron]

thanks for posting this question.  I asked about it a few months ago on slack and apparently it has come up a few times.  There is resistance (for good reason), but here's a proposal I put together that might get discussion going:

https://beta.observablehq.com/@shaunlebron/proposal-generators-and-async-functions-in-clojurescript

in summary, core.async doesn't solve all the problems, can be harder to debug, and using the promise api directly can be unwieldy.  but there are arguments against including the extra syntax.

feedback appreciated!

[Justin Lee]

I'm not the OP, but I'll explain why I personally avoid core.async and what I do to get features that are roughly equivalent to async/await today.

My first issue with core.async is that you have to be very careful on the producer-side to handle exceptions properly.  If you drop an exception, at best you'll get a stack-trace dumped to the console that often won't have a single line that traces back to your code.  Instead, you'll get a pile of generated library code that you have no experience with.  I ran into this when JSON.parse throws an exception inside cljs-http, which in my opinion is just the kind of run-of-the-mill exception that should be easy to debug (but isn't).

The second issue is that there are still a number of bugs with core.async surrounding exception handling.  As I was trying to fix the above issues, I ran into one of them.  https://dev.clojure.org/jira/browse/ASYNC-73

In short, my basic problem with core.async is that it adds a lot of complexity (and size) to your code, which I'd rather avoid if there are better alternatives.

The three non-callback ways of dealing with async code in javascript are (1) promises, (2) generators, (3) async/await.  The first two are es6.  The last is es-2017.  Because these are now built into the language and are widely relied upon, the tooling support (e.g. browser and node) are excellent and all the corner cases have been worked out.

Async/await is sugar that you put on top of generators.  The feel of async/await is exactly like go-blocks, where "async" is "go" and "await" is "<!".  Both allow you to write code that looks synchronous but is actually event-driven.  Neither can cross function boundaries.  And I believe the underlying implementation is roughly the same idea: chop the synchronous bits up into chunks, turn the transitions into a state machine, and then keep track of where you are.  Check out this generator transpiler: http://facebook.github.io/regenerator/

So far, though, I've found that async/await is mostly useful in javascript because of the fact that you have side-effecting statements and complex syntax where it is convenient to be able to throw an "await" without refactoring your code.  In clojurescript, you tend to write code in a non-side-effecting expressions, so I don't find that I need something that general.

For simple things, I find that just relying on plain promises (using the promesa library) is enough.  You can do this: 

(-> (promise-returning-call) 

    (then (fn [response] ...)) 

    (catch (fn [error] ...))) 

This is simple, it is very close to the host language, and there are no corner cases that I know of to surprise you.

Sometimes you need to thread the results of an earlier async call to later calls, which can be clumsy.  I think async/await was largely driven by solving this problem, but I don't think you really need that generality in this language.  I have found that as long as you write "clojurescripty" code, the alet macro from the promesa library is all you need.

(alet [a (get-a)
       b (get-b a)
       c (get-c a b)]
  (do-something c))

Setting all that aside, the clear advantage of being able to target async/await (or maybe just generators) is that you get the native implementation and don't have to do all of this code manipulation in user space.  The tooling is going to be better, the code will be smaller, and the edge cases are going to be handled.  Maybe you could even write a core.async fork without all of the IOC helpers (i.e. just translate directly to async/await).

By the way, I hope I'm not coming off as being critical of the core.async authors.  This stuff is very hard to get right and not everyone is going to have the same design requirements.

--

[Justin Lee]

Shaun,

Your email came in while I was drafting mine.  Thank you so much for putting together this proposal.  With the features you propose here, we'd be able to do complex async code in javascript with even less code and fewer dependencies, and we could rely on the battle tested transpilers and polyfills for people who need earlier targets.  I, personally, would love to see something like this in the language.

Justin

--

[Thomas Heller]

I'm generally in favor of "embracing the host" but both generators and async/await would probably require substantial rewrites of core parts of the compiler. It it not just about adding a small * or async keyword somewhere. The compiler will generally emit anonymous functions at various places to ensure proper scoping and this can pretty easily break async/generator functions. Given that you'd emitting ES6+ anways however you could replace those with proper block-scoped lets though. Of course that is not a reason to not do it, just don't underestimate how much work this would be.

I certainly looks like the JS world is adopting async/await but given the rate of change in that ecosystem that might change again when the next thing comes along. Given that the React folks decided to implement a pretty substantial feature based on throwing promises I guess they are here to stay for a while though.

I do not think that this compares in any way to core.async however. It is a much more powerful abstraction which can do a lot of things async/await can't and anything that does can easily be achieved with core.async and a few helper functions/macros. Yes, core.async is not perfect either but someone could work on fixing the kinks.

I guess I agree that we should eventually support async/await + generators for the sake of full interop but not because of "issues" with core.async. 

[Shaun LeBron]

good point about the IIFEs, added some notes on what to fixl:

https://beta.observablehq.com/@shaunlebron/proposal-generators-and-async-functions-in-clojurescript#blockers

> [core.async] is a much more powerful abstraction which can do a lot of things async/await can't and anything that does can easily be achieved with core.async and a few helper functions/macros

not quite! core.async doesn't allow you to cancel a go-block (to my knowledge), which JS allows.  I added a section on this:

https://beta.observablehq.com/@shaunlebron/proposal-generators-and-async-functions-in-clojurescript#coreasync

[Justin Lee]

One other thing: although there are some things (like the bug I cited) that could be fixed, the other aspect of exception handling cannot be fixed without breaking core.async, which, by design, swallows exceptions on the producer side.  David Nolen has a proposal for the <? operator, but that requires the producer to cooperate (and still doesn't help when something like not= or JSON.parse throws an exception you weren't expecting, unless you write the code very carefully).  Promise code just doesn't have this problem because there's a built in error channel that will catch any uncaught exception--and that makes debugging much easier.

--

[Philos Kim]

I appreciate all the feedback for my question and came to know that the implementation of the async/await feature in ClojureScript is not easy as I expected.

I agree that ClojureScript doesn't need to accept every new feature in JS and I know that core.async is more powerful than the newly introduced async/await feature in JS. However I would like to quote a Korean proverb: "Don't use the knife for slautering a cow, when slautering a cock." A simple use case needs a simple solution, not a magnificent and complex one.

In JS world, the promise comes out to overcome the callback hell and the async/await feature comes out to overcome the verbosity of the promise. Of course, I know that the async/await feature is a syntactic sugar of the promise, so can be used in the promise style in ClojureScript. But What I want to avoid is that verbosity as in JS.

Furthermore the async/await feature is now heavily used in the JS world, especially in nodejs. So I want to use it in ClojureScript at least as JS interop in the similiar way to the JS world, when I use the thrid-party library written using the async/await feature.

[Shogo Ohta]

FYI.

I'm working on a ClojureScript library named kitchen-async (https://github.com/athos/kitchen-async) in my spare time, and if you would like more syntactic support for Promises than promesa provides (ie. alet macro mentioned above), you might find it useful.

I know it's not totally an elegant solution for the issue, but it works enough (at least for me) in many practical cases.

Shogo

[Thomas Heller]

not quite! core.async doesn't allow you to cancel a go-block (to my knowledge), which JS allows.  I added a section on this:

https://beta.observablehq.com/@shaunlebron/proposal-generators-and-async-functions-in-clojurescript#coreasync

This is incorrect. Closing a channel can be used to "end" a loop. In addition it is possible to use alt! or alts! with an additional "control" channel (or more) and "selecting" which channel to work on.

(defn foo []
  (let [ch (async/chan)]
    (go (loop [i 100]
          (when (pos? i)
            (<! (async/timeout 1000))
            (when (>! ch i)
              (recur (dec i)))))
        (prn :loop-terminated))
    ch))

(go (let [ch (foo)]
      (prn (<! ch))
      (prn (<! ch))
      (async/close! ch)
      (prn (<! ch))))

 

In addition the "loop" is dependent on the "ch" reference. If that gets garbage collected the loop will be as well, similar to generators or async/await.

[Didier]

I think compatibility shouldn't be an issue. Doesn't the async fn just return a promise? So you should be able to call an async fn from ClojureScript.

[Shaun LeBron]

Right!  The proposal mentions that go-blocks must check for a closed channel at every step in order to exit early. So I'll revise the title—core.async cannot stop arbitrary go-blocks.

For example, with this staggered animation code, it's not immediately clear to me how to exit after any step. In JS, it pulls the plug for free.

(go
  (dotimes [_ 3]
    (swap! game assoc :board cleared)
    (<! (timeout 170))
    (swap! game assoc :board board)
    (<! (timeout 170)))
  (swap! game assoc :board cleared)
  (<! (timeout 220))
  (swap! game assoc :board collapsed))

Anyone coming from the Unity game engine will recognize this feature as the StopCoroutine function. Core.async does not have such a feature.  the ergonomics matter!

[Thomas Heller]

Yes, core.async is not great for this but it was also not meant for this. The big abstraction there are channels or CSP. There are "zero" channels involved in your example so you could probably find a better abstraction to fit here.

There are several different concurrency models and all of them have drawbacks. As I said before I'm for adding support for async/await and generators but core.async should not even be part of the argument IMHO. I'm not trying to convince anyone that core.async is the best solution ever. I remember vividly how desperately I wanted a proper Actor-like concurrency primitive when I came over from Erlang. I actually only understood the drawbacks of Actors after I learned about CSP.

I made this example a while ago showing what Closure does when rewriting async/await to ES3.

https://gist.github.com/thheller/82ca9ef8a04b58aafa4624edbac383ac

We could emit that JS code directly using a few macros. The only reason emitting the ES6+ code makes sense is due to tool support and maybe having slightly more compact code, which is probably not a factor after :advanced is done with it. We can already interop with all of it very easily, just writing equivalent code is not ideal.

To be honest just adding support for async/await and generators alone doesn't make much sense to me. At that point we should probably look into emitting ES6+ESM directly instead and make proper use of all the features that has. Given that Closure support for ES6 is getting much better that might make sense at some point.

[Didier]

For example this blog post shows compatibility: https://blog.jeaye.com/2017/09/30/clojurescript-promesa/

And also gives a nice little macro so you can use similar syntax to async and await.

The biggest downside is that all the sugar is in a macro, and thus can't rewrite across function boundaries. Though I think that's also true in JS.

[Shaun LeBron]

thanks!  added: https://beta.observablehq.com/@shaunlebron/proposal-generators-and-async-functions-in-clojurescript#macros

[Shaun LeBron]

agreed that core.async is not relevant to this discussion.

I suppose the main question is—how should ES6+ (ES Next) features be made available to CLJS users.  Is the story going to be  "All of ES5 is accessible from CLJS—but you must use an external JS file for ES Next features"?

(I added your transpile link to macros section. Also, I'd love to see emitting ES Modules in the future, though I'm not sure of implications there)

[Thomas Heller]

A few notes about the current state of the proposal:

defn is not sufficient, fn itself would need to be support too. Example uses defn only.

Your generator examples only show yield in one direction. This is already pretty much possible with seqs and you can write seq->gen + gen->seq functions pretty easily for those. No work needs to be done for that. The part that is currently possible via seqs is "var x = yield;" since generators/yield are two-way.

Closure is changing how lanuage-in/out work. Output will be set as a feature-set specifying exactly which features should be compiled down or kept. At least that is what a recent commit [1] suggests. Input already pretty much detects the feature set when parsing, it is mostly used an optimization to skip certain work before parsing. I expect some changes here as well.

There would need to be a way for processing the CLJS generated code in :none. Since no transpilation is done for those usually (since its ES3) that would be an entirely new thing. Given that most developers will be running modern environments this might not be too critical overall though.

[1] https://github.com/google/closure-compiler/commit/7d00895022cb0704edd8beeabeabdcb065c365c5

[Shaun LeBron]

Thanks thomas.  I followed CLJS-1986 and proposed fn should attach metadata to the params, since the name is optional: (fn ^:js-generator [params] ...)

updated the proposal for anon forms, and add bidirectional yield example.

[Gary Trakhman]

Could we regain some debuggability by implementing core.async over async/await instead of callbacks at the base, or would that still be too dynamic?

--

[Antonin Hildebrand]

Even if it was conceptually possible to reimplement core.async. I believe we cannot do that without introducing subtle incompatibilities with existing code like [1].

But we could implement a new library with the same semantics and let people opt-in.

[1] https://github.com/binaryage/cljs-devtools/issues/20

[Shaun LeBron]

Updated the proposal a bit:

- added Antonin's news on chrome's recent proper handling of core.async call stacks \o/

- added Async Generators (fn ^:js-async-gen [] ...)

[Shaun LeBron]

Also, forgot to address your note on :none—

> There would need to be a way for processing the CLJS generated code in :none.Since no transpilation is done for those usually (since its ES3) that would be an entirely new thing.

I was thinking we wouldn't have to transpile async/generators outside of Closure Compiler, since I would expect :none to just output the new syntax.  

If the main use-case for :none is for development, then I think this assumption is fine and actually desirable. 

[Thomas Heller]

If the main use-case for :none is for development, then I think this assumption is fine and actually desirable. 

Yes, when running in an environment that fully supports all features you don't need transpile in :none. That would be the norm but I guess it is still possible you end up developing for a platform that doesn't support all the latest features so you'd still need to be able to transpile it.