Dealing with wait/notify and notifyall when converting to Kilim

[Ron]

Hi All,

I've come to understand how to change code that does Thread/run to
Task/execute, and how
to change synchronized blocks and methods to use the ReentrantLock
class for locking.

What I don't know how to handle, is code that is doing wait/notify or
notifyall calls. Is there
a way to get rid of these calls, and if not what effect do they have
on how Kilim works?

Regards,

Ron

[Andrew Bate]

Did you make any progress with this?

If so, what was your strategy for making minimal changes to the existing source code for dealing with wait/notify?

Many thanks,

Andrew

[Andrew Bate]

In fact, does anyone know how wait/notify was handled when implementing Berkeley DB using Kilim?

[Andrew Bate]

Does any one know of any difficulties with the strategy of obtaining a condition with kilim.ReentrantLock#newCondition() and then using that Condition objects' signal(), signalAll() and await() methods with Kilim?

This should be a drop in replacement for notify(), notifyAll() and wait() with Kilim, correct?

Best wishes,

Andrew

On Friday, 11 February 2011 19:27:46 UTC, Ron wrote:

[Ron]

I'd sure like a confirmation Andrew's suggestion of using a Condition to replace wait notity notifyall, will that work?

[Sriram Srinivasan]

> I'd sure like a confirmation Andrew's suggestion of using a Condition to replace wait notity notifyall, will that work?
>

Alas, I have just discovered that ReentrantLock as currently present is defective, so DON't use it. You can create an equivalent signaling mechanism using Cell; it'll be faster since it is purely in user-space.

The problem with ReentrantLock is this. It is thread-id based, so an unlock will have to be performed on the same thread as a lock.

Say task A and task B use the same lock. A calls lock(), then yields. B then calls lock(), which succeeds because (a) it happens to be scheduled on the same thread that A executed on, and (ii) it is reentrant. Now two logically different threads own the same lock. It gets worse with unlock. If B unlocks, the lock is released, but A does not know that.

Mixing thread and task signaling mechanisms is error-prone; which is why Go doesn't export anything task or thread identity-related (no threadlocals for example).

--sriram.

> On Tuesday, April 16, 2013 9:22:41 AM UTC-5, Andrew Bate wrote:
> Does any one know of any difficulties with the strategy of obtaining a condition with kilim.ReentrantLock#newCondition() and then using that Condition objects' signal(), signalAll() and await() methods with Kilim?
>
> This should be a drop in replacement for notify(), notifyAll() and wait() with Kilim, correct?
>
> Best wishes,
>
> Andrew
>
>
>
> On Friday, 11 February 2011 19:27:46 UTC, Ron wrote:
> Hi All,
>
> I've come to understand how to change code that does Thread/run to
> Task/execute, and how
> to change synchronized blocks and methods to use the ReentrantLock
> class for locking.
>
> What I don't know how to handle, is code that is doing wait/notify or
> notifyall calls. Is there
> a way to get rid of these calls, and if not what effect do they have
> on how Kilim works?
>
> Regards,
>
> Ron
>

> --
> You received this message because you are subscribed to the Google Groups "kilimthreads" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to kilimthreads...@googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.

[Ron]

Thanks. I can see how I might use pause(PauseReason) to replace wait, but what about notify? I was thinking of just emulating notifyAll, which would be a 'message'

to all objects of a specific type to resume. How would I do this using a cell?

[Andrew Bate]

Hi all,

Sriram's comments are interesting: I'd never realized that issue existed with the current implementation before.

There is a possible solution, but it requires quite a bit of effort. I've tried it and can confirm that it works.

The implementation of ReentrantLock and Condition are available under a public domain license (see http://gee.cs.oswego.edu/dl/classes/EDU/oswego/cs/dl/util/concurrent/intro.html). Sun/Oracle used Doug Lea's code as a basis for their implementation, and published the result under GPLv2 only. (However, I cannot see any substantial differences between Oracle's code and Doug Lea's code.)

The implementation of ReentrantLock and Condition relies on a class called AbstractQueuedSynchronizer. This is quite a complicated class, but none of it relies on Thread.wait or Thread.notify, only on LockSupport.park and LockSupport.unpark.

If one were to implement an equivalent of LockSupport.park and LockSupport.unpark in Kilim's Task class, then one could port these public domain classes over for locks and conditions. The key difficulty here is implementing something like LockSupport.park and LockSupport.unpark in Kilim's Task class, which are effectively semaphores that are local to each task. The difficulty is largely due to Kilim's design decision of checking whether the pause reason is valid at the top of the stack during an unwind of a Task, and using a PauseReason object in an implementation of a semaphore seems quite tricky.

I do have code that does what I have just described, but *not* for Kilim. Instead, it is for a concurrency library for Scala, which is yet to be released.

I would estimate that it would require about one weeks worth of development effort (assuming at least some familiarity with Kilim's internals) to implement the same functionality for Kilim.

[Ron]

I'm trying to convert existing code, not write new code, which requires getting rid of wait / notify, if I want to take a java Runnable, or Thread and turn it into a Task.

The solution that Andrew is describing would be for new designs, right? Also, Andrew take a look at this article about Hoare-style monitor library, This looks

like it would drop right in to Kilim. http://www.javaworld.com/article/2077769/core-java/better-monitors-for-java.html. Although this monitor library would probably

fit, I don't think that's within the philosophy of how Sriram envisions Kilim to be?

[Andrew Bate]

Hi Ron,

Thanks for the link to an interesting article!

I've only taken a quick look at the moment, but the main difficulty here is that one would have to implement condition objects for Kilim before you could use them (regardless of whether this is a new design or not). You could either implement them the way I described before, and then use the condition objects as a drop in replacement for wait/notify, or you could implement them the way the article you linked described.

I've had a quick read of the source code for that monitor library, and at first glance, it depends on an implementation of a Semaphore, which itself uses Java's wait/notify on the inside...

You would thus have to implement a Semaphore for tasks first, which is the tricky bit, and then adapt the code of that library to use the new Semaphore class you just created. Such a semaphore class may look somewhat similar to the implementation of the Cell mailbox, but I'm not sure what the details would look like.

As far as the philosophy of the library goes, I guess that's one for Sriram. But from my perspective, there are three parts to Kilim (or any such library): lightweight fibers (or tasks), a user-space scheduler, and actor mailboxes. So, you have Fibers + Scheduler + Mailboxes. I don't see anything wrong per se with Fibers + Scheduler + Monitors instead, even if we may prefer message passing in an ideal world.

[Ron]

Let me make sure I understand the way Kilim works: A Kilim tasks has the same issues with Object member variables as regular java, they can be corrupted by multiple instances of the same Task class running concurrently, 

and one of the Tasks modifying the object member variable, correct? Otherwise, there's no need for a Monitor feature.

[Andrew Bate]

You are correct -- you need to ensure mutual exclusion of Tasks on shared state, just as you would with Threads.

[Ron]

I think that means that it would be worthwhile to figure out how to incorporate the Monitor library functions into Kilim then. This would make it easier to take

existing java Threads and Runnables, and convert them into Kilim Tasks. If we can figure out how to replace the wait/notify call in the Semaphore class, we could 

have monitors in Kilim, right? What do you think?

[Andrew Bate]

I agree. The only difficulty is that in addition to wait/notify, you also need an implementation of lock/unlock (in order to replace the synchronized keyword in that class), since the one currently provided by Kilim is broken. Alternatively, you could just try implementing a Semaphore directly if that were easier.