ETS & SMP was: Parallelism, SMP, and multicore question - Erlang Questions

The old Google Groups will be going away soon, but your browser is incompatible with the new version.
Message from discussion ETS & SMP was: Parallelism, SMP, and multicore question
Received-SPF: neutral (google.com: 193.180.168.22 is neither permitted nor denied by best guess record for domain of erlang-questions-boun...@erlang.org) client-ip=193.180.168.22;
Message-ID: <00f501c90d98$cf868720$6401a8c0@moneymaker2>
From: "Valentin Micic" <valen...@pixie.co.za>
To: <erlang-questi...@erlang.org>
References: <6c2563b20809021009n3f9619d9h4c439a3a64393c84@mail.gmail.com>
Date: Wed, 3 Sep 2008 09:43:47 +0200
MIME-Version: 1.0
Subject: [erlang-questions] ETS & SMP was: Parallelism, SMP,
	and multicore question
Precedence: list
Content-Type: multipart/mixed; boundary="===============1601737626=="
Sender: erlang-questions-boun...@erlang.org
Errors-To: erlang-questions-boun...@erlang.org

This is a multi-part message in MIME format.

--===============1601737626==
Content-Type: multipart/alternative;
	boundary="----=_NextPart_000_00F2_01C90DA9.9030EF50"

This is a multi-part message in MIME format.

------=_NextPart_000_00F2_01C90DA9.9030EF50
Content-Type: text/plain;
	charset="utf-8"
Content-Transfer-Encoding: quoted-printable

Is ETS utilizing the same locking policy for all table types (namely: =
public, protected or private), and if so, would it be possible to relax =
locking for protected and private access?=20

We've noticed that if more than one process requires an access to the =
same ets table (in SMP environment), the system slows down considerably =
due to the locking mechanism. It is quite possible to optimize this by =
fronting such a table with a dedicated process for request serialization =
-- works better as there is always only one proccess requesting a lock.  =
Actually... as much as this works well for one table, not so sure how =
would such an "optimization" work for a large number of tables. By =
relaxing (or not having) a locking policy for (at least) tables with a =
private access, there would be no questions about it.

V.

  ----- Original Message -----=20
  From: Edwin Fine=20
  To: erlang-questi...@erlang.org=20
  Sent: Tuesday, September 02, 2008 7:09 PM
  Subject: [erlang-questions] Parallelism, SMP, and multicore question


  Since the cognoscenti are talking about parallelism, maybe someone can =
answer something that has been perplexing me.

  Sometimes it has been measurably faster on a multicore system to run =
one VM per core, each with SMP disabled (or +S 1), than it is to run one =
or more SMP VMs. This is true even when running code that is innately =
parallel. On the other hand, I have seen benchmarks that show =
near-linear speedups using SMP.=20

  I don't have code to support this; it's just something I, along with a =
couple of other people -- for whom I don't want to speak -- have noticed =
in passing.

  Does anyone have any insight into the internals of SMP who can tell me =
when SMP might be an asset, and when a liability? I know this is a "how =
long is a piece of string" question, but even though it's pretty =
general, someone somewhere who really knows Erlang inside out (i.e. not =
me) must surely have encountered this phenomenon.

  Are there conditions where contention between the threads of the SMP =
processes (e.g. mutexes) becomes significant enough that multiple =
separate non-SMP VMs perform better? If so, what would those conditions =
be?

  Could it have anything to do with the mix of CPU-bound processes and =
inter-process messaging or network I/O? If so, please could someone =
explain the mechanism?

  Anyone?




-------------------------------------------------------------------------=
-----


  _______________________________________________
  erlang-questions mailing list
  erlang-questi...@erlang.org
  http://www.erlang.org/mailman/listinfo/erlang-questions
------=_NextPart_000_00F2_01C90DA9.9030EF50
Content-Type: text/html;
	charset="utf-8"
Content-Transfer-Encoding: quoted-printable

=EF=BB=BF<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META http-equiv=3DContent-Type content=3D"text/html; charset=3Dutf-8">
<META content=3D"MSHTML 6.00.6000.16705" name=3DGENERATOR>
<STYLE></STYLE>
</HEAD>
<BODY bgColor=3D#ffffff>
<DIV><FONT face=3DArial size=3D2>Is ETS utilizing the same locking =
policy for all=20
table types (namely: public, protected or private), and if so, would it =
be=20
possible to relax locking for protected and private =
access?&nbsp;</FONT></DIV>
<DIV><FONT face=3DArial size=3D2></FONT><FONT face=3DArial =
size=3D2></FONT>&nbsp;</DIV>
<DIV><FONT face=3DArial size=3D2>We've noticed that if more than one =
process=20
requires an access to the same ets table (in SMP environment), the =
system slows=20
down considerably due to the locking&nbsp;mechanism. It is quite =
possible to=20
optimize this by fronting such a table with a dedicated process for =
request=20
serialization -- works better as there is always only one proccess =
requesting a=20
lock.&nbsp;</FONT><FONT face=3DArial size=3D2> Actually... as much as =
this works=20
well for one table,&nbsp;not so sure how would such an "optimization" =
work for a=20
large number of tables. By relaxing (or not having) a locking policy for =
(at=20
least) tables with a private access, there would be no questions about=20
it.</FONT></DIV>
<DIV><FONT face=3DArial size=3D2></FONT><FONT face=3DArial =
size=3D2></FONT>&nbsp;</DIV>
<DIV><FONT face=3DArial size=3D2>V.</FONT></DIV>
<DIV><FONT face=3DArial size=3D2></FONT>&nbsp;</DIV>
<BLOCKQUOTE=20
style=3D"PADDING-RIGHT: 0px; PADDING-LEFT: 5px; MARGIN-LEFT: 5px; =
BORDER-LEFT: #000000 2px solid; MARGIN-RIGHT: 0px">
  <DIV style=3D"FONT: 10pt arial">----- Original Message ----- </DIV>
  <DIV=20
  style=3D"BACKGROUND: #e4e4e4; FONT: 10pt arial; font-color: =
black"><B>From:</B>=20
  <A title=3Derlang-questions_ef...@usa.net=20
  href=3D"mailto:erlang-questions_ef...@usa.net">Edwin Fine</A> </DIV>
  <DIV style=3D"FONT: 10pt arial"><B>To:</B> <A =
title=3Derlang-questi...@erlang.org=20
  =
href=3D"mailto:erlang-questi...@erlang.org">erlang-questi...@erlang.org</=
A>=20
  </DIV>
  <DIV style=3D"FONT: 10pt arial"><B>Sent:</B> Tuesday, September 02, =
2008 7:09=20
  PM</DIV>
  <DIV style=3D"FONT: 10pt arial"><B>Subject:</B> [erlang-questions] =
Parallelism,=20
  SMP, and multicore question</DIV>
  <DIV><BR></DIV>
  <DIV dir=3Dltr>Since the cognoscenti are talking about parallelism, =
maybe=20
  someone can answer something that has been perplexing =
me.<BR><BR>Sometimes it=20
  has been measurably faster on a multicore system to run one VM per =
core, each=20
  with SMP disabled (or +S 1), than it is to run one or more SMP VMs. =
This is=20
  true even when running code that is innately parallel. On the other =
hand, I=20
  have seen benchmarks that show near-linear speedups using SMP. =
<BR><BR>I don't=20
  have code to support this; it's just something I, along with a couple =
of other=20
  people -- for whom I don't want to speak -- have noticed in=20
  passing.<BR><BR>Does anyone have any insight into the internals of SMP =
who can=20
  tell me when SMP might be an asset, and when a liability? I know this =
is a=20
  "how long is a piece of string" question, but even though it's pretty =
general,=20
  someone somewhere who really knows Erlang inside out (i.e. not me) =
must surely=20
  have encountered this phenomenon.<BR><BR>Are there conditions where =
contention=20
  between the threads of the SMP processes (e.g. mutexes) becomes =
significant=20
  enough that multiple separate non-SMP VMs perform better? If so, what =
would=20
  those conditions be?<BR><BR>Could it have anything to do with the mix =
of=20
  CPU-bound processes and inter-process messaging or network I/O? If so, =
please=20
  could someone explain the mechanism?<BR><BR>Anyone?<BR><BR></DIV>
  <P>
  <HR>

  =
<P></P>_______________________________________________<BR>erlang-question=
s=20
  mailing=20
  =
list<BR>erlang-questi...@erlang.org<BR>http://www.erlang.org/mailman/list=
info/erlang-questions</BLOCKQUOTE></BODY></HTML>

------=_NextPart_000_00F2_01C90DA9.9030EF50--


--===============1601737626==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
erlang-questions mailing list
erlang-questi...@erlang.org
http://www.erlang.org/mailman/listinfo/erlang-questions
--===============1601737626==--
Create a group - Google Groups - Google Home - Terms of Service - Privacy Policy
Account Options
