Clojurescript async not yeilding for CPU intensive work

[Timothy Pratley]

I have an issue where work inside a 'go block' is blocking my UI.
Inserting a (<! (timeout 1)) 'fixes' it.

** Is there a more idiomatic way? **

Here is a cut down version built from a fresh template:
If you run this code, you don't see the thinking deeply countdown, and you cannot click the other button.

If you uncomment the timeout 1, it works better.

(defn hello-world []
[:div
[:h1 (:text @app-state)]
[:button
{:on-click
(fn [e]
(swap! app-state assoc :thinking true :thinking-deeply 5)
(go
(while (pos? (:thinking-deeply @app-state))
;(<! (timeout 2000))
(apply * (rest (range 10000000)))
;(<! (timeout 1))
(swap! app-state update :thinking-deeply dec))
(swap! app-state dissoc :thinking :thinking-deeply)))}
"The button!"]
(when (:thinking-deeply @app-state)
[:h2 "Thinking deeply " (:thinking-deeply @app-state)])
(when (:thinking @app-state)
[:h2 "Thinking"])
[:button "The other button"]])

Full source code is attached

Regards,
Timothy

[Daniel Compton]

Hi Timothy

We ran into a similar issue recently as well, see http://dev.clojure.org/jira/browse/ASYNC-137 for more details. I *think* in your case you would need to split your work up into chunks and yield control back to the browser after each chunk of work (with a (timeout 0)). See https://github.com/Day8/re-frame/wiki/Solve-the-CPU-hog-problem for more details on this approach.

--
Note that posts from new members are moderated - please be patient with your first post.
---
You received this message because you are subscribed to the Google Groups "ClojureScript" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojurescrip...@googlegroups.com.
To post to this group, send email to clojur...@googlegroups.com.
Visit this group at http://groups.google.com/group/clojurescript.

--

--

Daniel

[Timothy Pratley]

Thanks Daniel for the detailed info!
I'll do so.

[Thomas Heller]

This has nothing to do with core.async or the go macro. If you block the CPU you will block the UI. The browser does not support threads so neither can core.async.

What the timeout achieves is that you give some control back to the browser so instead of 1sec "blocking" you get 5x200ms blocking. Still far from 60fps but better than 0.

If you want to do CPU intensive work without blocking the UI you'll need to use one or more WebWorkers. You could layer core.async on top of that to communicate between the Workers and the UI but core.async itself does nothing in that regard.

WebWorkers are pretty easy to get going but carry their own set of drawbacks so it is very dependent on your use case whether they make sense or not.

HTH,
/thomas

[Daniel Kersten]

"The browser does not support threads so neither can core.async."

To expand on that, core.async uses cooperative multitasking, which means you have to give control back every so often so it can schedule other go blocks to be run. Calls like <! will do this while they block (which is why timeout works).

[Thomas Heller]

On Tuesday, August 25, 2015 at 10:59:53 AM UTC+2, Daniel Kersten wrote:
> "The browser does not support threads so neither can core.async."
>
>
> To expand on that, core.async uses cooperative multitasking, which means you have to give control back every so often so it can schedule other go blocks to be run. Calls like <! will do this while they block (which is why timeout works).
>

While that is correct let me emphasize that timeout is not a solution!

Do you always know how long task X will run or whether you are going to need to chunk it? Is it even possible to split up? A "task" that may complete in 10ms on your machine might take 100ms on another one or even 500ms on yours if the computer is doing something else.

If you need to do CPU intensive work in the browser use a WebWorker. It is their purpose. While not perfect it is far better than trying to be "cooperative" in your code.

My 2 cents,
/thomas

[Mike Thompson]

You have a problem, and you decide to solve it via webworkers.

Later you'll WISH you'd chosen regexs instead :-)

--
Mike

[Atamert Ölçgen]

Why the cynicism? Isn't it true that the only way to make use of multiple cores in browser is to use WebWorkers?

 

--
Mike

--
Note that posts from new members are moderated - please be patient with your first post.
---
You received this message because you are subscribed to the Google Groups "ClojureScript" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojurescrip...@googlegroups.com.
To post to this group, send email to clojur...@googlegroups.com.
Visit this group at http://groups.google.com/group/clojurescript.

--

Kind Regards,
Atamert Ölçgen

◻◼◻

◻◻◼

◼◼◼

www.muhuk.com

[Thomas Heller]

Well, while we are on the topic perhaps we should mention the downside of using Web Workers. For me it has been Data Transfer.

Since you can only transfer JSON type data between workers that usually meant you had to pr-str/read-string to roundtrip CLJS data. If you only have very small amounts of data that is no problem but read-string is not very performant and even something like 1mb of data will eat up enough CPU time to drop you below 60fps.

That might now be worked around with transit though since that is much faster. But still if you need to transfer a lot of data between workers you might be better of not using them. IE8/9 might also be an issue but even Microsoft suggest that you should not support those versions.

Oh and so far I have not seen support for workers in lein-cljsbuild and I haven't documented anything in shadow-build so building web worker javascript files might actually be something of a mystery to people. To be honest I never used them for anything other than toy projects either.

Anyways, I like them. Use them. It is not that hard. :)

Cheers,
/thomas

[Daniel Kersten]

On Wed, 26 Aug 2015 at 11:36 Thomas Heller <th.h...@gmail.com> wrote:

Well, while we are on the topic perhaps we should mention the downside of using Web Workers. For me it has been Data Transfer.

Since you can only transfer JSON type data between workers that usually meant you had to pr-str/read-string to roundtrip CLJS data. If you only have very small amounts of data that is no problem but read-string is not very performant and even something like 1mb of data will eat up enough CPU time to drop you below 60fps.

If you can use newer browser, you've got Transferrable Object to transfer binary data with zero-copy.

 

That might now be worked around with transit though since that is much faster. But still if you need to transfer a lot of data between workers you might be better of not using them. IE8/9 might also be an issue but even Microsoft suggest that you should not support those versions.

Oh and so far I have not seen support for workers in lein-cljsbuild and I haven't documented anything in shadow-build so building web worker javascript files might actually be something of a mystery to people. To be honest I never used them for anything other than toy projects either.

Agreed, its probably a bit of work. I guess that since each one has its own JS context, the simplest way is probably to have two cljs projects: your main code and your web worker code.

There is Servant: https://github.com/MarcoPolo/servant

What it seems to do is run the same compiled cljs both as the main code and the webworker code (with some setup/detection code to manage how it should run). This way both your webworker and main code can call the same functions. Communicating between them still happens through messages.

I've never actually used it, so my summary may not be great :)

 

Anyways, I like them. Use them. It is not that hard. :)

Cheers,
/thomas



>
> Why the cynicism? Isn't it true that the only way to make use of multiple cores in browser is to use WebWorkers?
>
>

[Thomas Heller]

On Wednesday, August 26, 2015 at 1:29:38 PM UTC+2, Daniel Kersten wrote:

>
> If you can use newer browser, you've got Transferrable Object to transfer binary data with zero-copy.
>  

True but I was talking about CLJS data which isn't binary.

>
> Agreed, its probably a bit of work. I guess that since each one has its own JS context, the simplest way is probably to have two cljs projects: your main code and your web worker code.
>

Actually you want to use closure modules so some code can be shared. With 2 projects the client will download 2 separate instances of cljs.core which is quite heavy.

I have an example here:
https://github.com/thheller/shadow-build/tree/master/cljs-data/workers/src
https://github.com/thheller/shadow-build/blob/master/dev/build.clj#L6-L30

It is not a good example but the idea is to have one common module (cljs) and one for each "client" (ie. the page itself, worker1 and worker). worker1 (and 2) will automatically load the cljs module which the browser already has available because the page did already load it.

But this is going too far off-topic. If anyone is actually interested in using WebWorkers I'd be happy to go into further detail on how that works in shadow-build.

Cheers,
/thomas

[Antonin Hildebrand]

I've got a fresh experience of refactoring my project to use web workers. Here is the story:

I'm building a code editor using web technologies[1]. The initial design was pretty simple and sweet. I decided to go with re-frame (thanks Mike!), which means UI driven by reagent and react with nice&clean event processing/subscription system.

The editor is doing basically three things:
1) UI rendering and responding to user actions
2) analyzing source code (for syntax highlighting, etc.)
3) layouting source code for renderer to generate markup

I observed that analyzing and layouting is blocking my rendering tasks, so I decided to offload them to another thread using webworkers. What seemed like an easy change at first, introduced a lot of complexity later.

Problems encountered so far:
1) cljs-devtools[2] does not work from web workers, this is a major problem for my development workflow
2) data must be marshalled as JSON => wasn't that hard, started using transmit
3) instead of one app-db state and one queue of events, now I have it split in two, one in the worker and one in the main thread. Keeping them in sync can be tricky due to async worker nature. imagine undo/redo operations - those have to be executed on both databases "at once".

To solve problem #1. I had to implement special development version of the app, which does not run a worker thread, but emulates it on the main thread. It does not sound that hard, except re-frame wasn't ready to provide me with two independent re-frame event queues in single javascript context. I had to rewrite it: https://github.com/Day8/re-frame/pull/107.

To make the long story short. One month later and wish I'd chosen regexs instead ;)

happy web-working! :)
Antonin

[1] https://github.com/darwin/plastic
[2] https://github.com/binaryage/cljs-devtools