Ruby is much slower on linux when compiled with --enable-pthread?
Adam Kramer
I was seeing significant performance differences between ruby 1.8.4 on
an old distribution and a newer one. I spent some hours tracking down
the differences and it appears that --enable-pthread causes ruby to be
significantly slower. I tried searching for the reason why this is but
came up empty handed.
On Ubuntu 6.06 I built a new binary of 1.8.4 (from the package source
- nothing else was different) with --disable-pthread and compared the
execution times of the benchmark files in SVN. Here are the results:
Second column is with --enable-pthread, third column
--disable-pthread. Look at almost every test that took more than a
second, of real note is app_pentomino and loop_times.
app_answer 0.674 0.504
app_factorial 0.020 0.013
app_fib 9.377 6.623
app_mandelbrot 2.384 1.862
app_pentomino 158.618 84.739
app_raise 1.176 0.964
app_strconcat 1.197 1.215
app_tak 12.390 8.158
app_tarai 9.872 6.473
loop_generator 22.547 15.394
loop_times 11.616 4.050
loop_whileloop 9.334 9.491
loop_whileloop2 1.878 1.906
so_ackermann 5.038 9.291
so_array 10.608 6.376
so_concatenate 3.633 1.620
so_count_words 0.272 0.267
so_exception 3.012 2.221
so_lists 1.302 1.023
so_matrix 2.753 1.906
so_nested_loop 9.877 5.060
so_object 5.705 3.780
so_random 1.967 1.752
so_sieve 0.627 0.591
vm1_block 35.529 19.547
vm1_const 14.287 14.482
vm1_ensure 28.497 15.053
vm1_length 18.162 18.991
vm1_rescue 18.771 11.221
vm1_simplereturn 31.387 16.038
vm1_swap 16.581 16.850
vm2_array 4.078 4.129
vm2_case 4.041 4.076
vm2_method 21.762 9.765
vm2_mutex 31.299 17.910
vm2_poly_method 25.094 13.121
vm2_poly_method_ov 4.192 4.125
vm2_proc 8.404 5.506
vm2_regexp 3.919 3.743
vm2_send 5.280 3.780
vm2_super 7.573 4.337
vm2_unif1 4.833 3.183
vm2_zsuper 8.120 4.848
vm3_thread_create_join 0.012 0.008
vm3_thread_mutex 5.787 3.278
It appears that when compiled with --enable-pthread ruby will call
sigprocmask MANY MANY more times than without. You can verify this
with strace -c:
$ strace -c ruby -e '1.upto(100000) {|i| i.to_s}'
% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ----------------
100.00 0.000655 0 200006 sigprocmask
Yeah - it called sigprocmask over 200,000 times during that 100,000
iteration loop.
When compiled with --disable-pthread and running the same command,
rt_sigprocmask gets called 4 times.
I also tested on OS X, ruby is compiled with --enable-pthread but
sigprocmask doesn't get called an inordinate number of times, so this
may be a linux-only issue.
Has anyone else witnessed this? Is this a "feature" that's to be expected?
--
Thanks!
-Adam
Roger Pack
M. Edward (Ed) Borasky
Hash: SHA1
Yeah -- it's probably not only Linux-specific but also dependent on your
compiler and run time library versions. What I would suggest is
compiling Ruby with profiling and seeing where in the Ruby interpreter
these calls are.
Incidentally, if you use Tk and Ruby, both have to be compiled with the
same setting for pthread usage.
(cross-posted to Ruby Core) :)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFG3eNP8fKMegVjSM8RAmTYAKDKGRikNc5fYQB7aiYW1+a7nfZG/wCfQIHl
b69A7ZvwALfkBOxnCfyvRX8=
=mTkA
-----END PGP SIGNATURE-----
Jeffrey 'jf' Lim
> Hi,
>
> I was seeing significant performance differences between ruby 1.8.4 on
>
I would be curious about this - are you talking about between ubuntu
6.06, and a newer version of ubuntu?
>
> It appears that when compiled with --enable-pthread ruby will call
> sigprocmask MANY MANY more times than without. You can verify this
> with strace -c:
> $ strace -c ruby -e '1.upto(100000) {|i| i.to_s}'
> % time seconds usecs/call calls errors syscall
> ------ ----------- ----------- --------- --------- ----------------
> 100.00 0.000655 0 200006 sigprocmask
>
> Yeah - it called sigprocmask over 200,000 times during that 100,000
> iteration loop.
>
same here - ubuntu 7.04 64-bit, ruby 1.8.6 -
% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ----------------
100.00 0.002105 0 200009 rt_sigprocmask
I also see some errors (8, 3, and 7 respectively - and consistently)
on 'open', 'stat', and 'access'. Do you see those as well?
> When compiled with --disable-pthread and running the same command,
> rt_sigprocmask gets called 4 times.
>
:) for me, this is 2 times.
> I also tested on OS X, ruby is compiled with --enable-pthread but
> sigprocmask doesn't get called an inordinate number of times, so this
> may be a linux-only issue.
>
hmm.. what would be the equivalent of 'strace' here?
-jf
--
In the meantime, here is your PSA:
"It's so hard to write a graphics driver that open-sourcing it would not help."
-- Andrew Fear, Software Product Manager, NVIDIA Corporation
http://kerneltrap.org/node/7228
Jeffrey 'jf' Lim
>
> What I would suggest is
> compiling Ruby with profiling and seeing where in the Ruby interpreter
> these calls are.
>
any idea how you would do that? Don't see anything in the configure
switches for compiling ruby (1.8.6)...
M. Edward (Ed) Borasky
Hash: SHA1
Before you "configure", type
export CFLAGS='-g -pg'
Then do
/configure
make
as normal. You will have a profiling Ruby executable.
Every time you run the profiling Ruby interpreter, you will get a file
called "gmon.out" which contains a binary profile. Then run
export GPROF_PATH=/path/to/ruby/source
gprof /path/to/ruby/executable > profile.txt
gprof -A /path/to/ruby/executable > annotated-source.txt
The "profile.txt" will give a call-graph profile of Ruby executing your
script, and "annotated-source.txt" will have a source listing of Ruby
with execution counts for all the lines.
A word of warning -- this compiles with no optimization at all, and can
be much slower as Ruby compiled with optimization. The call graph should
be all you need to determine where the system call is coming from.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFG3irB8fKMegVjSM8RAj3FAKDElDD+blUEIYK5IvG3rQe9UGsARQCg2Evn
jxbv8oYmkLNr9ZC8hdWkNNo=
=A2/g
-----END PGP SIGNATURE-----
Adam Kramer
> Jeffrey 'jf' Lim wrote:
>
> Before you "configure", type
>
> export CFLAGS='-g -pg'
I tried this earlier today, gprof doesn't seem to be able to trace the
extra time that's incurred due to the system calls. As far as gprof
was concerned, the --enable-pthread version was faster, even though it
took more system time and wall time. I also tried with gdb and set a
breakpoint on setprocmask but that didn't work - it only stopped on 4
calls to setprocmask, not millions.
--
-Adam
Bill Kelly
From: "Jeffrey 'jf' Lim" <jfs....@gmail.com>
>
> > I also tested on OS X, ruby is compiled with --enable-pthread but
> > sigprocmask doesn't get called an inordinate number of times, so this
> > may be a linux-only issue.
> >
>
> hmm.. what would be the equivalent of 'strace' here?
ktrace / kdump
Regards,
Bill
Terry Poulin
Don't know if it works on OS X but truss may also be useful.
TerryP.
--
Email and shopping with the feelgood factor!
55% of income to good causes. http://www.ippimail.com
Roger Pack
Thanks.
-Roger
............
>running Rails Petstore from tw-commons and timing consecutive page hits
>with:
>ab -c 1 -n 1000
>http://localhost:3000/shop/viewProduct.shtml?product=K9-DL-01
>--disable-pthread
>78 requests/sec
--enable-pthread
62 requests/sec
25 % difference.
Eric Hodel
> Does ruby compile by default with pthreads enabled?
No.
Rick DeNatale
I seem to recall that one of the library extensions (Tk if I'm not
mistaken) either wouldn't compile or complained loudly if if found
that Ruby had been compiled without pthreads enabled.
But then again, it might be my senility cropping up. <G>
--
Rick DeNatale
My blog on Ruby
http://talklikeaduck.denhaven2.com/
M. Edward (Ed) Borasky
> On 9/25/07, Eric Hodel <drb...@segment7.net> wrote:
>> On Sep 25, 2007, at 14:59 , Roger Pack wrote:
>>
>>> Does ruby compile by default with pthreads enabled?
>> No.
>
> I seem to recall that one of the library extensions (Tk if I'm not
> mistaken) either wouldn't compile or complained loudly if if found
> that Ruby had been compiled without pthreads enabled.
>
> But then again, it might be my senility cropping up. <G>
>
system, they must all be compiled with the same setting for "pthreads"
-- either all of them have it set or all of them don't. Ruby will
complain during either configure or make, I forget which, if you tell it
to do the opposite of what it has determined you did for Tcl and Tk.
Alexey Verkhovsky
> If you have Ruby (1.8), Tcl and Tk all together on the same Linux
> system, they must all be compiled with the same setting for "pthreads"
Which is why every precompiled binary package of Ruby that I've seen
is linked with pthread.
--
Alexey Verkhovsky
CruiseControl.rb [http://cruisecontrolrb.thoughtworks.com]
RubyWorks [http://rubyworks.thoughtworks.com]
Eric Hodel
> On 9/25/07, Eric Hodel <drb...@segment7.net> wrote:
>> On Sep 25, 2007, at 14:59 , Roger Pack wrote:
>>
>>> Does ruby compile by default with pthreads enabled?
>>
>> No.
>
> I seem to recall that one of the library extensions (Tk if I'm not
> mistaken) either wouldn't compile or complained loudly if if found
> that Ruby had been compiled without pthreads enabled.
Correct.
If you check out from source and run:
autoconf && ./configure && make
ruby will not be linked with the system pthread library.
Roger Pack
Alexey Verkhovsky wrote:
> On 9/26/07, M. Edward (Ed) Borasky <zn...@cesmail.net> wrote:
>> If you have Ruby (1.8), Tcl and Tk all together on the same Linux
>> system, they must all be compiled with the same setting for "pthreads"
>
> Which is why every precompiled binary package of Ruby that I've seen
> is linked with pthread.
--
Posted via http://www.ruby-forum.com/.
Marc Heiler
Maybe try with "ldd" on the ruby binaries
Sean O'Halpin
> Hi,
>
> I was seeing significant performance differences between ruby 1.8.4 on
> an old distribution and a newer one. I spent some hours tracking down
> the differences and it appears that --enable-pthread causes ruby to be
> significantly slower.
> Has anyone else witnessed this? Is this a "feature" that's to be expected?
Yes, we've hit this - in Centos 5, the default ruby build is 1.8.5 (!)
with --enable-pthread. What I'd like to know is why[1]? Is it just
hand-waving conservatism (just-in-case-we-need-it), RedHat policy to
enable pthreads everywhere or is there a specific reason why pthreads
have to be enabled, e.g. there's an Oracle driver that requires it or
some such?
Anyone have any ideas?
Regards,
Sean
[1] Apart from why anyone thinks putting an unstable version of ruby
in their distro is a good idea
m.fel...@gmail.com
ruby ships with exteensive bindings to tk and so has to be kept
compatible to use ruby and tk together.
that's the only reason i can think of.
^ manveru