[erlang-questions] Parallelism, SMP, and multicore question

[Edwin Fine]

Since the cognoscenti are talking about parallelism, maybe someone can answer something that has been perplexing me.

Sometimes it has been measurably faster on a multicore system to run one VM per core, each with SMP disabled (or +S 1), than it is to run one or more SMP VMs. This is true even when running code that is innately parallel. On the other hand, I have seen benchmarks that show near-linear speedups using SMP.

I don't have code to support this; it's just something I, along with a couple of other people -- for whom I don't want to speak -- have noticed in passing.

Does anyone have any insight into the internals of SMP who can tell me when SMP might be an asset, and when a liability? I know this is a "how long is a piece of string" question, but even though it's pretty general, someone somewhere who really knows Erlang inside out (i.e. not me) must surely have encountered this phenomenon.

Are there conditions where contention between the threads of the SMP processes (e.g. mutexes) becomes significant enough that multiple separate non-SMP VMs perform better? If so, what would those conditions be?

Could it have anything to do with the mix of CPU-bound processes and inter-process messaging or network I/O? If so, please could someone explain the mechanism?

Anyone?

[Valentin Micic]

Is ETS utilizing the same locking policy for all table types (namely: public, protected or private), and if so, would it be possible to relax locking for protected and private access? 

 

We've noticed that if more than one process requires an access to the same ets table (in SMP environment), the system slows down considerably due to the locking mechanism. It is quite possible to optimize this by fronting such a table with a dedicated process for request serialization -- works better as there is always only one proccess requesting a lock.  Actually... as much as this works well for one table, not so sure how would such an "optimization" work for a large number of tables. By relaxing (or not having) a locking policy for (at least) tables with a private access, there would be no questions about it.

 

V.

 

---

_______________________________________________
erlang-questions mailing list
erlang-q...@erlang.org
http://www.erlang.org/mailman/listinfo/erlang-questions

[Gleb Peregud]

2008/9/3 Valentin Micic <vale...@pixie.co.za>:

Hello,

I've asked very similar question.

Take a look here:

http://www.erlang.org/pipermail/erlang-questions/2008-April/034514.html
http://www.erlang.org/pipermail/erlang-questions/2008-April/034530.html

BR
--
Gleb Peregud
http://gleber.pl/

Every minute is to be grasped.
Time waits for nobody.
-- Inscription on a Zen Gong

[Bjorn Gustavsson]

2008/9/3 Valentin Micic <vale...@pixie.co.za>

Is ETS utilizing the same locking policy for all table types (namely: public, protected or private), and if so, would it be possible to relax locking for protected and private access? 

We currently don't eliminate locking on private tables, mainly to support ets:info/1,2. We may try to eliminate locks

on private tables in a future release. (But we are not sure it is worthwhile. Taking a lock that is not already held by another

process is relatively cheap - it becomes expensive when several processes want to take the same lock.)

 

However, we take different locks depending on the operation to perform. An operation that only reads from the table

(such as ets:lookup/2) will take a read lock, while an operation that will update the table (such as ets:insert/2) takes

a write lock. As long as no process has a write lock, any number of processes can take read locks. If a process takes

a write lock, it will be exclusive (i.e. no other processes can have either read or write locks).

 

We've noticed that if more than one process requires an access to the same ets table (in SMP environment), the system slows down considerably due to the locking mechanism. It is quite possible to optimize this by fronting such a table with a dedicated process for request serialization -- works better as there is always only one proccess requesting a lock.  Actually... as much as this works well for one table, not so sure how would such an "optimization" work for a large number of tables. By relaxing (or not having) a locking policy for (at least) tables with a private access, there would be no questions about it.

To access an ETS table, there are actually two locks that need to be taken.

1) A lock to access the meta table, to convert the numeric table identifier to a pointer to the actual table.

2) The lock for the table itself (either a read or write lock, as described above).

In R12B-4, the locking of the meta table has been optimized. There used to be only one lock for the meta table,

but there are now different locks for different parts of the table; therefore reducing the number of lock conflicts

for the meta table.

Therefore, if you have an application that accesses many different ETS tables, performance should be slightly better

in R12B-4.

If you have an application that accesses a single ETS table (and write to it frequently), there will still be lock

conflicts on the ETS table itself, so R12B-4 will not make much difference.

/Bjorn

-- 

Björn Gustavsson, Erlang/OTP, Ericsson AB

[Edwin Fine]

Well, now I know that at least one person read my question :)

Aren't concerns about when SMP is and isn't beneficial at least thought-provoking? Or do I need to write and publish actual code showing the differences?

2008/9/3 Valentin Micic <vale...@pixie.co.za>