Received: by 10.204.152.217 with SMTP id h25mr958081bkw.3.1340563065542;
Sun, 24 Jun 2012 11:37:45 -0700 (PDT)
X-BeenThere: erlang-programming@googlegroups.com
Received: by 10.204.129.208 with SMTP id p16ls3100852bks.7.gmail; Sun, 24 Jun
2012 11:37:45 -0700 (PDT)
Received: by 10.204.149.210 with SMTP id u18mr961053bkv.1.1340563065153;
Sun, 24 Jun 2012 11:37:45 -0700 (PDT)
Received: by 10.204.149.210 with SMTP id u18mr961052bkv.1.1340563065110;
Sun, 24 Jun 2012 11:37:45 -0700 (PDT)
Return-Path: <erlang-questions-boun...@erlang.org>
Received: from hades.cslab.ericsson.net (hades.cslab.ericsson.net. [192.121.151.104])
by gmr-mx.google.com with ESMTP id iv15si11123936bkc.0.2012.06.24.11.37.44;
Sun, 24 Jun 2012 11:37:45 -0700 (PDT)
Received-SPF: pass (google.com: domain of erlang-questions-boun...@erlang.org designates 192.121.151.104 as permitted sender) client-ip=192.121.151.104;
Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of erlang-questions-boun...@erlang.org designates 192.121.151.104 as permitted sender) smtp.mail=erlang-questions-boun...@erlang.org; dkim=neutral (body hash did not verify) header...@gmail.com
Received: from hades.cslab.ericsson.net (hades [192.121.151.104])
by hades.cslab.ericsson.net (Postfix) with ESMTP id 7DC9B5C02B;
Sun, 24 Jun 2012 20:37:38 +0200 (CEST)
X-Original-To: erlang-questi...@erlang.org
Delivered-To: erlang-questi...@erlang.org
Received: from mail-vc0-f181.google.com (mail-vc0-f181.google.com
[209.85.220.181])
by hades.cslab.ericsson.net (Postfix) with ESMTP id A78D95C001
for <erlang-questi...@erlang.org>; Sun, 24 Jun 2012 20:37:36 +0200 (CEST)
Received: by vcbf1 with SMTP id f1so1827684vcb.40
for <erlang-questi...@erlang.org>; Sun, 24 Jun 2012 11:37:35 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
h=references:from:in-reply-to:mime-version:date:message-id:subject:to
:cc:content-type;
bh=OwmBXwL9xXCutUxO0TQV6U0WE2KEPkrvSbUl900ggt4=;
b=gvepb7/8LC+VMFRey0Zd9mKXppgdlh7pgl4AVjbbOMZ24vIPsXEte1KneHAf87V6mi
XeZaw/xTAGnj7GZYc5GL7ro9q0fsFHpFpuDDazCXWmd/SPHvaN4DOTPR7BA4wHIErdqU
FyYS4l+wPxTe1gIr4GIdJJTFnTr8F9rixPHHpO5AXB43pUr+BaK2K+C8XbaWDrsinQE0
8iVi240WvIL6C5Pr/pluy/84c3gNtcMw+AkkmlCPTHvQXj87Q/POdlz0ccu8q20rNuAs
d8+8XoHxvyb/b4bdXhwcn89IYRNLHKsCfZWSUQvKLANumEadfoHEP5lGk5Duor8ZkIxk
0ynA==
Received: by 10.52.33.169 with SMTP id s9mr5029666vdi.0.1340563055591; Sun, 24
Jun 2012 11:37:35 -0700 (PDT)
References: <4fe42c05.7067980a.1893.6...@mx.google.com>
<CAHcC6xeLb84nzuXJg62s2_k0oT4==R9yrfAaZ=TvCdmSKFx...@mail.gmail.com>
<CAMJ=MEdpW3C3E-XU6apDVA28p0KiOawZvSkFyZ7udoT+umF...@mail.gmail.com>
<CAHcC6xcbN76nHinCo9Do=Qguj3GMYUQGjPUdwPGsyPZX8ij...@mail.gmail.com>
From: =?ISO-8859-1?Q?Bj=F6rn=2DEgil_Dahlberg?= <wallentin.dahlb...@gmail.com>
In-Reply-To: <CAHcC6xcbN76nHinCo9Do=Qguj3GMYUQGjPUdwPGsyPZX8ij...@mail.gmail.com>
Mime-Version: 1.0 (1.0)
Date: Sun, 24 Jun 2012 20:37:12 +0200
Message-ID: <746752638638550424@unknownmsgid>
To: Erisa Dervishi <erisa...@gmail.com>
Cc: "erlang-questi...@erlang.org" <erlang-questi...@erlang.org>
Subject: Re: [erlang-questions] SMP for IO-bound applications
X-BeenThere: erlang-questi...@erlang.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: General Erlang/OTP discussions <erlang-questions.erlang.org>
List-Unsubscribe: <http://erlang.org/mailman/options/erlang-questions>,
<mailto:erlang-questions-requ...@erlang.org?subject=unsubscribe>
List-Archive: <http://erlang.org/pipermail/erlang-questions>
List-Post: <mailto:erlang-questi...@erlang.org>
List-Help: <mailto:erlang-questions-requ...@erlang.org?subject=help>
List-Subscribe: <http://erlang.org/mailman/listinfo/erlang-questions>,
<mailto:erlang-questions-requ...@erlang.org?subject=subscribe>
Content-Type: multipart/mixed; boundary="===============8371471299711738597=="
Errors-To: erlang-questions-boun...@erlang.org
Sender: erlang-questions-boun...@erlang.org
--===============8371471299711738597==
Content-Type: multipart/alternative; boundary=20cf307cffacba3fda04c33c2842
--20cf307cffacba3fda04c33c2842
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
Hi Erisa,
In R15 we try to spin schedulers even more in an effort to reduce their
sleep. This is especially useful when we have jitter in runnability among
processes. If a queue is empty a scheduler is forced to sleep, the latency
in wake up is something we want avoid, hence we spin. The system is at its
best when it is fully saturated.
Another feature in R15 is delayed dealloc. Memory deallocation in the wrong
thread is instead scheduled to be deallocated. This feature lessens lock
contentions on memory allocators and improves scalability but will in all
likelihood also increase CPU utilization (I have seen this on other
measurements). In the future the schedulers will probably take this into
account among other constraints and rules.
If you want to investigate this further I recommend using a gprof-enabled
vm to examine where the time is spent.
Regards,
Bj=F6rn-Egil
24 jun 2012 kl. 20:12 skrev Erisa Dervishi <erisa...@gmail.com>:
Yes sure, even though I am using Tsung (a load generator tool written in
Erlang) to generate my load and get some results back. I have added a new
Tsung plugin for the purpose of my tests. I am a bit busy right now but If
you are interested I will share what I have done so far.
However, I would really like to have an opinion from an expert about my
comments in here
On Sun, Jun 24, 2012 at 10:47 AM, Ronny Meeus <ronny.me...@gmail.com> wrote=
:
> On Fri, Jun 22, 2012 at 1:51 PM, Erisa Dervishi <erisa...@gmail.com>
> wrote:
> > Hey, thank you for your reply, but I use the same mysql driver even for
> my
> > evaluations in Erlang OTP R12 and do not get the same CPU usage (for
> R12 is
> > up to 45%). And the tests are exactly the same (same load). I have also
> > checked the message queue of the process that supervises the connection=
s,
> > and it looks like there isn't a bottleneck in there (the queue length i=
s
> > most of the time 0 and up to 4 messages at most)
> > And even for Mnesia the CPU usage (though low), doubles from R12 to R15
> with
> > smp enabled from 6% to 12% for write tests
> >
> >
> >
> > On Fri, Jun 22, 2012 at 10:22 AM, be.dmi...@gmail.com <
> be.dmi...@gmail.com>
> > wrote:
> >>
> >> I think that much of CPU usage is created by your MySQL driver.
> >> You should have a look at driver details. There may be copying of the
> >> query you send to MySQL between your benchmarking process and the real
> >> process that owns the socket. The same with result - parsing response
> and
> >> sending result back to benchmarking process. There may be a bottleneck
> in
> >> choosing MySQL connection from connection pool if it is made through
> >> supervisor.
> >> Try to use etop to see what functions are at the bottlenecks.
> >> I mean your io-intensive test may be just the test showing driver
> >> bottlenecks and not the problems in smp.
> >>
> >>
> >>
> >>
> >> ----- Reply message -----
> >> From: "Erisa Dervishi" <erisa...@gmail.com>
> >> To: <erlang-questi...@erlang.org>
> >> Subject: [erlang-questions] SMP for IO-bound applications
> >> Date: Thu, Jun 21, 2012 23:28
> >>
> >>
> >> One additonal comment I forgot to mention:
> >>
> >> For all Mnesia tests, the CPU usage is low in general. Only for the
> >> transactional read test in R15 with SMP enabled i get a CPU usage of
> 70%.
> >> From the load generator tool, i call the read(N) function that is show=
ed
> >> below:
> >>
> >> exec_read(0) -> done_reading;
> >>
> >> exec_read(N) ->
> >>
> >> Id=3Drandom:uniform(7000000),
> >>
> >> %%io:format("~p~n",[Id]),
> >>
> >> %%[_Row] =3D mnesia:dirty_read({subscriber,Id}),
> >>
> >> Fun =3D fun() ->
> >>
> >> mnesia:read({subscriber,Id})
> >>
> >> end,
> >>
> >> {atomic,[Row]}=3Dmnesia:transaction(Fun),
> >>
> >> %io:format("~p~n",[Row]),
> >>
> >> exec_read(N-1).
> >>
> >> read(N) ->
> >>
> >> {A1,A2,A3} =3D now(),
> >>
> >> random:seed(A1, A2, A3),
> >>
> >> exec_read(N).
> >>
> >>
> >> I am using mpstat while my tests are running and beside the CPU usage,
> >> another difference of this test from the other Mnesia tests is that th=
e
> >> syscl (system calls) column of mpstat has a value which is 20 times
> higher
> >> than the other test cases.
> >>
> >>
> >>
> >> On Thu, Jun 21, 2012 at 8:37 PM, Erisa Dervishi <erisa...@gmail.com>
> >> wrote:
> >>
> >> > Hi,
> >> > As part of my studies, I have recently been doing some performance
> >> > evaluations on Erlang SMP improvements for IO-bound applications.
> >> >
> >> > The applications I considered for the evaluation were Emysql driver =
(
> >> > https://github.com/Eonblast/Emysql) and Mnesia.
> >> > I have created an Erlang module for performing reads/writes from/to
> >> > Mysql
> >> > DB through Emysql driver, and another Erlang module for communicatin=
g
> >> > with
> >> > Mnesia.
> >> > Both the modules have two methods write(nr_records) which writes
> >> > nr_records records to the database (MySQL, or Mnesia), and
> >> > read(nr_records)
> >> > which reads nr_records random records from the db. (both the reads a=
nd
> >> > writes are consecutive, not in chunks)
> >> > I have a load generator tool which simulates simultaneous requests t=
o
> >> > both
> >> > the modules and gathers performance statistics. I generate as much
> load
> >> > as
> >> > needed to stress the applications (near the saturation point)
> >> > The metrics I uses are the throughput (no. of sessions/sec), and the
> >> > session duration. A session is a call to write(nr_records)
> >> > or read(nr_records) function depending on the test case (read or
> write).
> >> > So
> >> > if I call write(500), I measure the duration of a session that does
> 500
> >> > hundred inserts into the database.
> >> >
> >> > The tests I run have these parameters:
> >> >
> >> > A)Type of operations: 1- Reads 2-Writes
> >> > I have just one table with 20 fields, and the reads and writes are
> just
> >> > select and insert operations in that table
> >> >
> >> > B) I/O applications: 1- Emysql 2-Mnesia
> >> >
> >> > C) SMP parameters:
> >> > 1- SMP enabled, no. of schedulers =3D no. of cpu cores
> >> > 2-SMP enabled, no. of schedulers =3D 1
> >> > 3- SMP disabled
> >> >
> >> > D) Erlang OTP versions:
> >> > 1- Erlang R12B (The SMP has only one run-queue and multiple
> schedulers)
> >> > 2- Erlang R15B (Improved SMP, 1 run-queue per each scheduler)
> >> >
> >> > The tests included all possible combinations from A,B,C,D. They were
> run
> >> > in Solaris 10 x86 (a 16 cores machine).
> >> > In general these were the results I got:
> >> >
> >> > *Emysql driver:*
> >>
> >> > 1- There is a a big difference in performance between SMP enabled an=
d
> >> > disabled in both the Erlang versions (R12 and R15) for both read an=
d
> >> > write
> >> > tests. So I can say that SMP rocks! However, you have to have enoug=
h
> >> > load
> >> > to achieve that (for low load I could not see any difference,
> sometimes
> >> > it
> >> > was even better to disable SMP)
> >> >
> >> > 2- I was expecting a much better performance for SMP enabled (no. of
> >> > cores
> >> > =3D no. of schedulers) in R15 than in R12, since the schedulers' log=
ic
> has
> >> > changed a lot from R12 to R15. But the results were more or less the
> >> > same.
> >> > I was thinking since I had a multithreaded db like MySql, a multipl=
e
> >> > db-connections driver as Emysql, and SMP with multiple schedulers wi=
th
> >> > their own run queues, I could get better results than in R12 where
> there
> >> > is
> >> > only one run-queue and multiple schedulers (more lock contention).
> >> > Is it maybe because the processes are just doing IO and nothing
> >> > CPU-intensive?
> >> >
> >> > 3- I realized that CPU usage in R15 SMP enabled is twice higher than
> the
> >> > CPU use in R12 SMP enable. All the cores have a CPU usage over 90%,
> and
> >> > I
> >> > can see that the beam process is using up to 75-80% of the CPU, the
> rest
> >> > is
> >> > the mysql daemon process. Is it because in R15 the scheduling
> algorithm
> >> > has
> >> > become more complex?
> >> >
> >> > *Mnesia*
> >> > *
> >>
> >> > *
> >> > I have only one table (as I said before) created with the attribute
> >> > disc_copies, and it is stored only in one node. The whole table fits
> in
> >> > RAM, and I tried to keep it simple by controlling the size of the
> table
> >> > during my tests, in order not exceed RAM capacity
> >> >
> >> > 4- For the write tests in Mnesia I see the same behavior as in Mysq=
l.
> >> > However, since the data are all loaded in RAM, the session duration =
is
> >> > shorter, and the gain in performance between SMP enabled and disable=
d
> is
> >> > not as huge as in Mysql.
> >> >
> >> > 5- For the read tests, I have two observations. First, the dirty rea=
ds
> >> > are
> >> > 10 times faster than the reads that use transactions. Second, I have
> >> > better
> >> > results for SMP disabled (twice faster, though in both cases the
> session
> >> > duration is in order of millisecods, 70 msec vs 30 msec).
> >> >
> >> > 6- Same as in Mysql, no difference in performance between R12 and R1=
5
> >> > with
> >> > SMP enabled, for both read and write tests.
> >> >
> >> > If you have read so far, and have a comment on my results, you are
> >> > welcome. Especially about the no difference btw R12 and R15 and the
> high
> >> > CPU usage in R15 when the applications are IO-bound.
> >> >
> >> > Thanks,
> >> > Erisa
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >
> >
> >
> > _______________________________________________
> > erlang-questions mailing list
> > erlang-questi...@erlang.org
> > http://erlang.org/mailman/listinfo/erlang-questions
> >
>
> Hello
>
> I'm very interested also in the behavior of Erlang applications in an
> SMP environment (so also my post "Strange observation running in an
> SMP environment." on this mailing list). Is the code that you have
> created somewhere available so that I can also play with it? In case
> not: are you willing to share it with the community?
>
> Thanks
>
> --
> Best regards,
> Ronny
>
_______________________________________________
erlang-questions mailing list
erlang-questi...@erlang.org
http://erlang.org/mailman/listinfo/erlang-questions
--20cf307cffacba3fda04c33c2842
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable
<html><head></head><body bgcolor=3D"#FFFFFF"><div>Hi Erisa,</div><div><br><=
/div><div>In R15 we try to spin schedulers even more in an effort to reduce=
their sleep. This is especially useful when we have jitter in runnability =
among processes. If a queue is empty a scheduler is forced to sleep, the la=
tency in wake up is something we want avoid, hence we spin. The system is a=
t its best when it is fully saturated.</div>
<div><br></div><div>Another feature in R15 is delayed dealloc. Memory deall=
ocation in the wrong thread is instead scheduled to be deallocated. This fe=
ature lessens lock contentions on memory allocators and improves scalabilit=
y but will in all likelihood also increase CPU utilization (I have seen thi=
s on other measurements). In the future the schedulers will probably take t=
his into account among other constraints and rules.</div>
<div><br></div><div>If you want to investigate this further I recommend usi=
ng a gprof-enabled vm to examine where the time is spent.</div><div><br></d=
iv><div>Regards,</div><div>Bj=F6rn-Egil</div><div><br></div><div><br>24 jun=
2012 kl. 20:12 skrev Erisa Dervishi <<a href=3D"mailto:erisa...@gmail.c=
om">erisa...@gmail.com</a>>:<br>
<br></div><div></div><blockquote type=3D"cite"><div>Yes sure, even though I=
am using Tsung (a load generator tool written in Erlang) to generate my lo=
ad and get some results back. I have added a new Tsung plugin for the purpo=
se of my tests. I am a bit busy right now but If you are interested I will =
share what I have done so far.<div>
<br></div><div>However, I would really like to have an opinion from an expe=
rt about my comments in here<br></div><br><div class=3D"gmail_quote">On Sun=
, Jun 24, 2012 at 10:47 AM, Ronny Meeus <span dir=3D"ltr"><<a href=3D"ma=
ilto:ronny.me...@gmail.com" target=3D"_blank">ronny.me...@gmail.com</a>>=
</span> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex"><div class=3D"HOEnZb"><div class=3D"h5">On F=
ri, Jun 22, 2012 at 1:51 PM, Erisa Dervishi <<a href=3D"mailto:erisa85d@=
gmail.com">erisa...@gmail.com</a>> wrote:<br>
> Hey, thank you for your reply, but I use the same mysql driver even fo=
r my<br>
> evaluations in=A0 Erlang OTP R12 and do not get the same CPU usage (fo=
r R12 is<br>
> up to 45%). And the tests are exactly the same (same load). I have als=
o<br>
> checked the message queue of the process that supervises the connectio=
ns,<br>
> and it looks like there isn't a bottleneck in there (the queue len=
gth is<br>
> most of the time 0 and up to 4 messages at most)<br>
> And even for Mnesia the CPU usage (though low), doubles from R12 to R1=
5 with<br>
> smp enabled from 6% to 12% for write tests<br>
><br>
><br>
><br>
> On Fri, Jun 22, 2012 at 10:22 AM, <a href=3D"mailto:be.dmi...@gmail.co=
m">be.dmi...@gmail.com</a> <<a href=3D"mailto:be.dmi...@gmail.com">be.dm=
i...@gmail.com</a>><br>
> wrote:<br>
>><br>
>> I think that much of CPU usage is created by your MySQL driver.<br=
>
>> You should have a look at driver details. There may be copying of =
the<br>
>> query you send to MySQL between your benchmarking process and the =
real<br>
>> process that owns the socket. The same with result - parsing respo=
nse and<br>
>> sending result back to benchmarking process. There may be a bottle=
neck in<br>
>> choosing MySQL connection from connection pool if it is made throu=
gh<br>
>> supervisor.<br>
>> Try to use etop to see what functions are at the bottlenecks.<br>
>> I mean your io-intensive test may be just the test showing driver<=
br>
>> bottlenecks and not the problems in smp.<br>
>><br>
>><br>
>><br>
>><br>
>> ----- Reply message -----<br>
>> From: "Erisa Dervishi" <<a href=3D"mailto:erisa85d@gm=
ail.com">erisa...@gmail.com</a>><br>
>> To: <<a href=3D"mailto:erlang-questi...@erlang.org">erlang-ques=
ti...@erlang.org</a>><br>
>> Subject: [erlang-questions] SMP for IO-bound applications<br>
>> Date: Thu, Jun 21, 2012 23:28<br>
>><br>
>><br>
>> One additonal comment I forgot to mention:<br>
>><br>
>> For all Mnesia tests, the CPU usage is low in general. Only for th=
e<br>
>> transactional read test in R15 with SMP enabled i get a CPU usage =
of 70%.<br>
>> From the load generator tool, i call the read(N) function that is =
showed<br>
>> below:<br>
>><br>
>> exec_read(0) -> done_reading;<br>
>><br>
>> exec_read(N) =A0-><br>
>><br>
>> =A0 =A0 =A0 =A0Id=3Drandom:uniform(7000000),<br>
>><br>
>> =A0 =A0 =A0 =A0%%io:format("~p~n",[Id]),<br>
>><br>
>> =A0 =A0 =A0 =A0%%[_Row] =3D mnesia:dirty_read({subscriber,Id}),<br=
>
>><br>
>> =A0 =A0 =A0 =A0Fun =3D fun() -><br>
>><br>
>> =A0 =A0 =A0 =A0 =A0 mnesia:read({subscriber,Id})<br>
>><br>
>> =A0 =A0 =A0 =A0end,<br>
>><br>
>> =A0 =A0 =A0 =A0{atomic,[Row]}=3Dmnesia:transaction(Fun),<br>
>><br>
>> =A0 =A0 =A0 =A0%io:format("~p~n",[Row]),<br>
>><br>
>> =A0 =A0 =A0 =A0exec_read(N-1).<br>
>><br>
>> read(N) -><br>
>><br>
>> =A0 =A0 =A0 =A0{A1,A2,A3} =3D now(),<br>
>><br>
>> =A0 =A0 =A0 =A0random:seed(A1, A2, A3),<br>
>><br>
>> =A0 =A0 =A0 =A0exec_read(N).<br>
>><br>
>><br>
>> I am using mpstat while my tests are running and beside the CPU us=
age,<br>
>> another difference of this test from the other Mnesia tests is tha=
t the<br>
>> syscl (system calls) column of mpstat has a value which is 20 time=
s higher<br>
>> than the other test cases.<br>
>><br>
>><br>
>><br>
>> On Thu, Jun 21, 2012 at 8:37 PM, Erisa Dervishi <<a href=3D"mai=
lto:erisa...@gmail.com">erisa...@gmail.com</a>><br>
>> wrote:<br>
>><br>
>> > Hi,<br>
>> > As part of my studies, I have recently been doing some perfor=
mance<br>
>> > evaluations on Erlang SMP improvements =A0for IO-bound applic=
ations.<br>
>> ><br>
>> > The applications I considered for the evaluation were Emysql =
driver (<br>
>> > <a href=3D"https://github.com/Eonblast/Emysql" target=3D"_bla=
nk">https://github.com/Eonblast/Emysql</a>) and Mnesia.<br>
>> > I have created an Erlang module for performing reads/writes f=
rom/to<br>
>> > Mysql<br>
>> > DB through Emysql driver, and another Erlang module for commu=
nicating<br>
>> > with<br>
>> > Mnesia.<br>
>> > Both the modules have two methods write(nr_records) which wri=
tes<br>
>> > nr_records records to the database (MySQL, or Mnesia), and<br=
>
>> > read(nr_records)<br>
>> > which reads nr_records random records from the db. (both the =
reads and<br>
>> > writes are consecutive, not in chunks)<br>
>> > I have a load generator tool which simulates simultaneous req=
uests to<br>
>> > both<br>
>> > the modules and gathers performance statistics. I generate as=
much load<br>
>> > as<br>
>> > needed to stress the applications (near the saturation point)=
<br>
>> > The metrics I uses are the throughput (no. of sessions/sec), =
and the<br>
>> > session duration. A session is a call to write(nr_records)<br=
>
>> > or read(nr_records) function depending on the test case (read=
or write).<br>
>> > So<br>
>> > if I call write(500), I measure the duration of a session tha=
t does 500<br>
>> > hundred inserts into the database.<br>
>> ><br>
>> > The tests I run have these parameters:<br>
>> ><br>
>> > A)Type of operations: 1- Reads 2-Writes<br>
>> > I have just one table with 20 fields, and the reads and write=
s are just<br>
>> > select and insert operations in that table<br>
>> ><br>
>> > B) I/O applications: 1- Emysql 2-Mnesia<br>
>> ><br>
>> > C) SMP parameters:<br>
>> > 1- SMP enabled, no. of schedulers =3D no. of cpu cores<br>
>> > 2-SMP =A0enabled, no. of schedulers =3D 1<br>
>> > 3- SMP disabled<br>
>> ><br>
>> > D) Erlang OTP versions:<br>
>> > 1- Erlang R12B (The SMP has only one run-queue and multiple s=
chedulers)<br>
>> > 2- Erlang R15B (Improved SMP, 1 run-queue per each scheduler)=
<br>
>> ><br>
>> > The tests included all possible combinations from A,B,C,D. Th=
ey were run<br>
>> > in Solaris 10 x86 (a 16 cores machine).<br>
>> > In general these were the results I got:<br>
>> ><br>
>> > *Emysql driver:*<br>
>><br>
>> > 1- There is a a big difference in performance between SMP ena=
bled and<br>
>> > disabled in both the Erlang versions (R12 and R15) =A0for bot=
h read and<br>
>> > write<br>
>> > tests. =A0So I can say that SMP rocks! However, you have to h=
ave enough<br>
>> > load<br>
>> > to achieve that (for low load I could not see any difference,=
sometimes<br>
>> > it<br>
>> > was even better to disable SMP)<br>
>> ><br>
>> > 2- I was expecting a much better performance for SMP enabled =
(no. of<br>
>> > cores<br>
>> > =3D no. of schedulers) in R15 than in R12, since the schedule=
rs' logic has<br>
>> > changed a lot from R12 to R15. But the results were more or l=
ess the<br>
>> > same.<br>
>> > =A0I was thinking since I had a multithreaded db like MySql, =
a multiple<br>
>> > db-connections driver as Emysql, and SMP with multiple schedu=
lers with<br>
>> > their own run queues, I could get better results than in R12 =
where there<br>
>> > is<br>
>> > only one run-queue and multiple schedulers (more lock content=
ion).<br>
>> > Is it maybe because the processes are just doing IO and nothi=
ng<br>
>> > CPU-intensive?<br>
>> ><br>
>> > 3- I realized that CPU usage in R15 SMP enabled is twice high=
er than the<br>
>> > CPU use in R12 SMP enable. All the cores have a CPU usage ove=
r 90%, and<br>
>> > I<br>
>> > can see that the beam process is using up to 75-80% of the CP=
U, the rest<br>
>> > is<br>
>> > the mysql daemon process. Is it because in R15 the scheduling=
algorithm<br>
>> > has<br>
>> > become more complex?<br>
>> ><br>
>> > *Mnesia*<br>
>> > *<br>
>><br>
>> > *<br>
>> > I have only one table (as I said before) created with the att=
ribute<br>
>> > disc_copies, and it is stored only in one node. The whole tab=
le fits in<br>
>> > RAM, and I tried to keep it simple by controlling the size of=
the table<br>
>> > during my tests, in order not exceed RAM capacity<br>
>> ><br>
>> > 4- =A0For the write tests in Mnesia I see the same behavior a=
s in Mysql.<br>
>> > However, since the data are all loaded in RAM, the session du=
ration is<br>
>> > shorter, and the gain in performance between SMP enabled and =
disabled is<br>
>> > not as huge as in Mysql.<br>
>> ><br>
>> > 5- For the read tests, I have two observations. First, the di=
rty reads<br>
>> > are<br>
>> > 10 times faster than the reads that use transactions. Second,=
I have<br>
>> > better<br>
>> > results for SMP disabled (twice faster, though in both cases =
the session<br>
>> > duration is in order of millisecods, 70 msec vs 30 msec).<br>
>> ><br>
>> > 6- Same as in Mysql, no difference in performance between R12=
and R15<br>
>> > with<br>
>> > SMP enabled, for both read and write tests.<br>
>> ><br>
>> > If you have read so far, and have a comment on my results, yo=
u are<br>
>> > welcome. Especially about the no difference btw R12 and R15 a=
nd the high<br>
>> > CPU usage in R15 when the applications are IO-bound.<br>
>> ><br>
>> > Thanks,<br>
>> > Erisa<br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
>> ><br>
><br>
><br>
><br>
</div></div>> _______________________________________________<br>
> erlang-questions mailing list<br>
> <a href=3D"mailto:erlang-questi...@erlang.org">erlang-questions@erlang=
.org</a><br>
> <a href=3D"http://erlang.org/mailman/listinfo/erlang-questions" target=
=3D"_blank">http://erlang.org/mailman/listinfo/erlang-questions</a><br>
><br>
<br>
Hello<br>
<br>
I'm very interested also in the behavior of Erlang applications in an<b=
r>
SMP environment (so also my post "Strange observation running in an<br=
>
SMP environment." on this mailing list). Is the code that you have<br>
created somewhere available so that I can also play with it? In case<br>
not: are you willing to share it with the community?<br>
<br>
Thanks<br>
<br>
--<br>
Best regards,<br>
Ronny<br>
</blockquote></div><br>
</div></blockquote><blockquote type=3D"cite"><div><span>___________________=
____________________________</span><br><span>erlang-questions mailing list<=
/span><br><span><a href=3D"mailto:erlang-questi...@erlang.org">erlang-quest=
i...@erlang.org</a></span><br>
<span><a href=3D"http://erlang.org/mailman/listinfo/erlang-questions">http:=
//erlang.org/mailman/listinfo/erlang-questions</a></span><br></div></blockq=
uote></body></html>
--20cf307cffacba3fda04c33c2842--
--===============8371471299711738597==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
_______________________________________________
erlang-questions mailing list
erlang-questi...@erlang.org
http://erlang.org/mailman/listinfo/erlang-questions
--===============8371471299711738597==--
Message from discussion SMP for IO-bound applications
| Create a group - Google Groups - Google Home - Terms of Service - Privacy Policy |
| ©2013 Google |