spinlock which can morph into a mutex

[Miquel van Smoorenburg]

I'm trying to implement a dynamically resizable hashtable, and
I have found that after resizing the table I need to call
synchronize_rcu() and finish up before letting other writers
(inserts, deletes) access the table.

Ofcourse during the hashtable update a spinlock is held to
exclude the other writers. But I cannot hold this spinlock over
synchronize_rcu(), yet the other writers still need to be excluded.

So I probably need a mutex instead of a spinlock, but I want to
keep minimal overhead for the common case (when no resizing is in
progress). I think I need a spinlock that can morph into a mutex ..

I was thinking about using something like the code below.
It is sortof like a spinlock, but it's ofcourse less fair
than actual ticketed spinlocks.

I'm working off 2.6.27 at the moment, but I noticed that in
2.6.28 adaptive spinning was introduced for mutexes. Is the
approach below still worth it with adaptive spinning or could
I just convert the spinlocks to mutexes with minimal extra overhead ?

Example code:

int real_mutex_lock = 0; // can use int since mutex ops are barriers
struct mutex mutex;

// 1. used instead of spinlock() [common case]
while (mutex_trylock(&mutex) == 0) {
if (real_mutex_lock) {
mutex_lock(&mutex);
break;
}
}
.. have lock, do work
mutex_unlock(&mutex);

// 2. When we want to lock and be able to sleep [seldomly used]
mutex_lock(&mutex);
real_mutex_lock = 1;
smp_wmb();

.. do work ..
real_mutex_lock = 0;
mutex_unlock(&mutex);

Mike.
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majo...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

[Andi Kleen]

Miquel van Smoorenburg <miq...@cistron.nl> writes:
>
> So I probably need a mutex instead of a spinlock, but I want to
> keep minimal overhead for the common case (when no resizing is in
> progress). I think I need a spinlock that can morph into a mutex ..

The standed mutexes already do that by themselves.

-Andi

--
a...@linux.intel.com -- Speaking for myself only.

[Peter Zijlstra]

On Fri, 2009-12-18 at 15:30 +0100, Miquel van Smoorenburg wrote:

> I just convert the spinlocks to mutexes with minimal extra overhead ?

Yep.

[Thomas Gleixner]

On Fri, 18 Dec 2009, Miquel van Smoorenburg wrote:

> I'm trying to implement a dynamically resizable hashtable, and
> I have found that after resizing the table I need to call
> synchronize_rcu() and finish up before letting other writers
> (inserts, deletes) access the table.
>
> Ofcourse during the hashtable update a spinlock is held to
> exclude the other writers. But I cannot hold this spinlock over
> synchronize_rcu(), yet the other writers still need to be excluded.
>
> So I probably need a mutex instead of a spinlock, but I want to
> keep minimal overhead for the common case (when no resizing is in
> progress). I think I need a spinlock that can morph into a mutex ..

Is the writer frequency and the possible contention so high that you
need a spinlock at all ?

> I was thinking about using something like the code below.
> It is sortof like a spinlock, but it's ofcourse less fair
> than actual ticketed spinlocks.
>
> I'm working off 2.6.27 at the moment, but I noticed that in
> 2.6.28 adaptive spinning was introduced for mutexes. Is the
> approach below still worth it with adaptive spinning or could
> I just convert the spinlocks to mutexes with minimal extra overhead ?

Test it :)

If the mutex is still to heavy weight for you, then you can solve it
without implementing another weird concurrency control:

writer side:

spin_lock(&hash_lock);

if (unlikely(hash_update_active)) {
spin_unlock(&hash_lock);
wait_event_(un)interruptible(&hash_wq, !hash_update_active);
spin_lock(&hash_lock);
}

resize side:

spin_lock(&hash_lock);
hash_update_active = 1;
....
spin_unlock(&hash_lock);
synchronize_rcu();
hash_update_active = 0;
wake_up(&hash_wq);

Thanks,

tglx

[Miquel van Smoorenburg]

On Fri, 2009-12-18 at 18:14 +0100, Thomas Gleixner wrote:
> On Fri, 18 Dec 2009, Miquel van Smoorenburg wrote:
> I think I need a spinlock that can morph into a mutex ..
>
> Is the writer frequency and the possible contention so high that you
> need a spinlock at all ?

Possibly - I don't want to degrade the performance of existing code
(which uses a spinlock).

> Test it :)

Good point.

> If the mutex is still to heavy weight for you, then you can solve it
> without implementing another weird concurrency control:
>

> wait_event_(un)interruptible(&hash_wq, !hash_update_active);
>
> hash_update_active = 1;
> ....
> hash_update_active = 0;
> wake_up(&hash_wq);

Ah, ofcourse. Thanks for pointing that out!

Thanks everybody for your input. I gained quite a bit of insight.

Mike.