Keep application running when main thread only starts go blocks

[Richard Möhn]

I'm using core.async. In my application the main thread creates a few channels, starts a few go blocks and that's it. When I run it as a stand-alone (i.e. not in the REPL), it starts those go blocks and then exits. After being surprised initially, I understand why this happens: the main thread has nothing more to do, so it terminates and with it the application.

However, I want the application to run until it receives SIGTERM, for example. My current solution is to have a channel wait-for-exit and the main thread does a

(<!! wait-for-exit)

after starting its go routines and the signal handler does a

(put! wait-for-exit :ok)

This blocks the main thread until the signal handler is called and the application shut down.

It also wastes the resources allocated to the main thread. (Which is only a little bit of RAM, as far as I know, so not too bad.) What is the usual way to solve this problem? Let the main thread do the work of one of the go routines?

Best,

Richard

[Miguel Ping]

Dunno about clojure, but in javaland you would submit jobs through an executor and then wait for all tasks on the executor to finish, blocking the main thread.

A bit of googling, and you can alter the core.async executor: http://stackoverflow.com/a/18779688/22992

And then you can await for its shutdown: http://stackoverflow.com/a/1250655/22992

[Alex Miller]

I think this is solution is fine. A single channel is not going to use any noticeable resources. You've basically created a latch - there are several latch-like things built into Java you can use as well. 

In the main thread you could do:

(let [signal (java.util.concurrent.CountDownLatch. 1)]

  ... launch your work

  (.await signal))

And in the signal handler you then:

(.countdown signal)

You could also use a Lock and Condition (the oo version of wait/notify), or a Semaphore, or a CyclicBarrier.

[Richard Möhn]

Am Samstag, 6. August 2016 16:50:35 UTC+9 schrieb Miguel Ping:

Dunno about clojure, but in javaland you would submit jobs through an executor and then wait for all tasks on the executor to finish, blocking the main thread.

A bit of googling, and you can alter the core.async executor: http://stackoverflow.com/a/18779688/22992

And then you can await for its shutdown: http://stackoverflow.com/a/1250655/22992

Thanks for the answer! It's interesting, but a bit hacky, so I would only use it if there's no alternative.

Richard

[Richard Möhn]

Am Samstag, 6. August 2016 21:30:53 UTC+9 schrieb Alex Miller:

I think this is solution is fine. A single channel is not going to use any noticeable resources. You've basically created a latch - there are several latch-like things built into Java you can use as well. 

In the main thread you could do:

(let [signal (java.util.concurrent.CountDownLatch. 1)]

  ... launch your work

  (.await signal))

And in the signal handler you then:

(.countdown signal)

You could also use a Lock and Condition (the oo version of wait/notify), or a Semaphore, or a CyclicBarrier.

 
Ah, thanks! This might make the intention clearer.

Aside from that, I was more concerned with the resources allocated to the blocked main thread than to the channel. Although I guess a "main thread doing nothing for most of the application's lifetime" just sounds wrong, but actually doesn't matter.

[Timothy Baldridge]

So I'm tempted to ask at this point, what does your program do? If you're doing nothing but CPU work, then yeah you may need to do something in the main thread. However, all IO work should be done outside of go block. Go blocks are limited in the number of concurrent threads that they use, so it's possible to deadlock the pool if you do IO inside the code called by a go block. So I'd suggest moving what ever IO work you're doing into dedicated threads and that might just solve your problem. 

--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+unsubscribe@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

“One of the main causes of the fall of the Roman Empire was that–lacking zero–they had no way to indicate successful termination of their C programs.”
(Robert Firth)

[Richard Möhn]

On Sun, Aug 07, 2016 at 10:21:44PM -0600, Timothy Baldridge wrote:
> So I'm tempted to ask at this point, what does your program do? If you're
> doing nothing but CPU work, then yeah you may need to do something in the
> main thread. However, all IO work should be done outside of go block. Go
> blocks are limited in the number of concurrent threads that they use, so
> it's possible to deadlock the pool if you do IO inside the code called by a
> go block. So I'd suggest moving what ever IO work you're doing into
> dedicated threads and that might just solve your problem.

That I shouldn't do I/O in a go block is something I found out
yesterday. This is the first time I seriously use core.async, so I hope
I will be forgiven.

The program isn't doing heavy work, though, so I think my abuse of go
blocks didn't do much harm. It just has a process that regularly (every
five minutes) downloads volcanic ash advisories and puts the data on a
published channel. The subscribers (currently two) send notifications
etc.

If you're unexpectedly interested, you can look at the code here:
https://github.com/rmoehn/sakurajima-ash-service It's pretty terrible,
because I'm was in a hurry and trying several new things at once. Also,
I haven't incorporated the "I/O no go" everywhere, yet.

Thanks for your help! I didn't have much of a problem, only a question
about best practices, but it's good to read your insights!

Richard