Using a try catch inside a future

[Tom Bodenheimer]

Hi all,

I have recently been playing with futures and have stumbled on the following situation.

(defn test-f1 [initial-v]

  (let [n (atom initial-v)]

    [n

     (future (while (not (Thread/interrupted))

               (Thread/sleep 5000)

               (swap! n inc)

               (println @n)))]))

(def t1 (test-f1 11))

This works as expected.  I see the println outputs, can @(t1 0) and (future-cancel (t1 1)) returns true.

Then I have:

(defn test-f2 [initial-v]

  (let [n (atom initial-v)]

    [n

     (future (while (not (Thread/interrupted))

               (try

                 (Thread/sleep 5000)

                 (swap! n inc)

                 (println @n)

                 (catch Exception e (println "whoops")))))]))

(def t2 (test -f2 11))

The same except now my while has a (try ... (catch Exception e ...).  However, now (future-cancel (t2 1)) returns false and the future cannot be cancelled - I also see the "whoops" at the REPL.  Seeing that "whoops" made me wonder if the interrupt that would shutdown the thread is raising an exception that is being caught by my (try .... (catch ...)) block, thus never interrupting the thread. 

I made a change to throw the exception again:

(defn test-f3 [initial-v]

  (let [n (atom initial-v)]

    [n

     (future (while (not (Thread/interrupted))

               (try

                 (Thread/sleep 5000)

                 (swap! n inc)

                 (println @n)

                 (catch Exception e (do (println "whoops") (throw e))))))])) 

and the behavior now matches that of test-f1.

I think I'm just not clear on some fundamental aspect of how future-cancel, try/catch, and the underlying java are actually working here.

It seems *very* weird to me that after future-cancel cancels the future, that the (try . . . (catch . . .)) would ever be invoked in the body of the while loop.  Yet, clearly the interrupt is being handled in the (try . . . (catch . . . )).  It also seems *very* weird that even in this case, that the next time we get back to (while (not (Thread/interrupted)) that (not(Thread/interrupted)) would be true . . . I guess the exception sets the interrupt status for the thread.  When exception gets caught in my (try . . . (catch . . .)), the interrupt status is never set?

Thanks for any clarification.

 

[Ashish Negi]

To stop any thread.. interrupts are send to it.
And threads handle this by throwing exception so that programmer can decide what to do
depending upon the kind of exception it gets. (you may get different exceptions)

Since you are catching the exception, your thread is never stopped.. and hence future-cancel returns false.

If you throw again.. exception would unwind your while loop and stop the thread.
hence.. future-cancel is able to stop the thread and returns true.

[Tom Bodenheimer]

Hi, this actually boils down to a strong case of read the manual and one surprising effect.

The executive summary: From the Java SE 8 docs description for Thread/sleep:

Throws:

IllegalArgumentException - if the value of millis is negative

InterruptedException - if any thread has interrupted the current thread. The interrupted status of the current thread is cleared when this exception is thrown.

The key to my misunderstanding is right there is the InterruptedException doc - the interrupted status is cleared by Thread/sleep.

The remaining surprise is that the future-cancel call actually returns true for all the cases (which is different from what I originally wrote).  Check out the below examples.

Example 1 - just a normal future with no try/catch

user> (defn test-interrupt-status-1 [init-v]

  (let [n (atom init-v)]

    [n

     (future (while (not (Thread/interrupted))

               (swap! n inc)))]))

#'user/test-interrupt-status-1

user> (def test-1 (test-interrupt-status-1 10))

#'user/test-1

user> (future-cancel (test-1 1))

true

user> (future-cancelled? (test-1 1))

true

user> @(test-1 0)

43202690

user> @(test-1 0)

43202690

Example 2 - a future that contains a try/catch but doesn't have any sleep included

Notice that future-cancel works fine even with the try/catch - the atom isn't updated

after the future-cancel is called on the future.

user> (defn test-interrupt-status-2 [init-v]

  (let [n (atom init-v)]

    [n

     (future (while (not (Thread/interrupted))

               (try

                 (swap! n inc)

                 (catch Exception e (println (str "Exception:\t" e "\nisInterrupted:\t" (.isInterrupted (Thread/currentThread))))))))]))

#'user/test-interrupt-status-2

user> (def test-2 (test-interrupt-status-2 10))

#'user/test-2

user> (future-cancel (test-2 1))

true

user> (future-cancelled? (test-2 1))

true

user> @(test-2 0)

44066815

user> @(test-2 0)

44066815

Example 3 - now we have a future which includes a try/catch AND A Thread/sleep

Calling future-cancel returns true (!!) and we see that the Thread/sleep

has cleared the interrupt status of the thread (e.g., the .isInterrupted is false

for the thread).  What is interesting here is that future-cancelled? returns true

but the thread is actually still happily running because the interrupt status was 

reset.  I'm not sure what I think about this - I hate surprises, but the docs are pretty

clear on what is happening here.  

user> (defn test-interrupt-status-3 [init-v]

  (let [n (atom init-v)]

    [n

     (future (while (not (Thread/interrupted))

               (try

                 (Thread/sleep 10000)

                 (swap! n inc)

                 (catch Exception e (println (str "Exception:\t" e "\nisInterrupted:\t" (.isInterrupted (Thread/currentThread))))))))]))

#'user/test-interrupt-status-3

user> (def test-3 (test-interrupt-status-3 10))

#'user/test-3

user> (future-cancel (test-3 1))

trueException: java.lang.InterruptedException: sleep interrupted

isInterrupted: false

user> (future-cancelled? (test-3 1))

true

user> @(test-3 0)

12

user> @(test-3 0)

13

user> (future-cancel (test-3 1))

false

user> (future-cancelled? (test-3 1))

true

[Dan Burton]

One technique for addressing this sort of issue has been described in Haskell-land:

https://www.schoolofhaskell.com/user/snoyberg/general-haskell/exceptions/catching-all-exceptions
http://hackage.haskell.org/package/enclosed-exceptions

I'm unaware of any comparable clojure library, but the same technique probably works.

A simpler technique in this case would be to only catch the kinds of exceptions that you expect the inner block to throw, rather than "catching all exceptions" with (catch Exception e ...).

-- Dan Burton

--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Dan Burton]

Here's something to explore.

(defmacro future-try [& body]

  `@(future (try ~@body)))

(defn test-f2 [initial-v]

  (let [n (atom initial-v)]

    [n

     (future (while (not (Thread/interrupted))

               (future-try

                 (Thread/sleep 5000)

                 (swap! n inc)

                 (println @n)

                 (catch Exception e (println "whoops")))))]))

(def t2 (test-f2 11))

test-f2 is the same as in your original email, except it uses future-try instead of try.
It seems to allow you to correctly cancel the thread. It doesn't hit the "whoops" branch.

But something is off. It doesn't stop the inner block, and the final number is printed *after* the outer future is cancelled. I would expect canceling a future to cancel all of its children, but apparently this is not the case.

----------

Want to see some really weird stuff? Here's how I tried to make future-try attempt to cancel its children:

(defmacro future-try [& body]

  `(let [thread-atom# (atom nil)]

     (try

       (reset! thread-atom# (future (try ~@body)))

       (catch Exception e#

         (if-let [thread# @thread-atom#]

           (future-cancel thread#))

         (throw e#)))))

Try running (def f2 (test-f2 11)) with that future-try, and it goes crazy. After waiting 5 seconds, it spews out numbers up to around 2031, then stops because (second f2) wasn't able to allocate any more threads. I have no explanation for this.

-- Dan Burton

On Fri, Apr 29, 2016 at 5:25 PM, Tom Bodenheimer <tom.bod...@gmail.com> wrote:

Hi Ashish,

It actually appears that there is no exception thrown by default when future-cancel is called on the thread *unless* you manage to overlook java functions that include some type of interrupt status handling.  As I managed to do.

Take a look at my below test-interrupt-status-2 that includes a try/catch loop to see what I mean.

Thanks.

[Dan Burton]

Ah, the weirdness was because I forgot to make future-try wait for the future it creates.

(defmacro future-try [& body]

  `(let [thread-atom# (atom nil)]

     (try

       (reset! thread-atom# (future (try ~@body)))

       @@thread-atom#

       (catch Exception e#

         (if-let [thread# @thread-atom#]

           (future-cancel thread#))

         (throw e#)))))

Using this does cancel the inner thread as soon as you cancel the outer one, it does run the "catch" branch of code, and it does stop the loop. You can even replace (while (not (Thread/interrupted)) ...) with (while true ...) and it still behaves this way. It might seem unexpected or undesirable that it runs the "catch" branch, but it does seem like a good idea to let a future-try/catch block clean up after itself. /shrug

-- Dan Burton

[Ashish Negi]

https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/Future.html#cancel%28boolean%29
may explain why future-cancelled? is returning true..

After this method returns, subsequent calls to isDone() will always return true. Subsequent calls to isCancelled() will always return true if this method returned true.

Thread will not stop.. but future is now cancelled.
Future represents some value in future and it should have one value after it is done.
The values that future is `still` calculating will now be not useful since its status is cancelled.
A cancelled future can also have some value but it's status means that it is not usable.

** It is true that i am trying to put logic after seeing the results. :(
Do correct me if something is wrong.

[Ashish Negi]

Also, it means that bug is not in future or threads but
in threaded-function itself which is swallowing all the exceptions..
Future is cancelled but it is just that bad-thread is not stopping.