[ruby-core:27380] [Bug #2553] Fix pthreads slowness by eliminating unnecessary sigprocmask calls
Dan Peterson
http://redmine.ruby-lang.org/issues/show/2553
Author: Dan Peterson
Status: Open, Priority: Normal
ruby -v: ruby 1.8.7 (2009-12-24 patchlevel 248) [i686-linux]
This is a bug report for what's described here:
http://timetobleed.com/fix-a-bug-in-rubys-configurein-and-get-a-30-performance-boost/
Matz says here that this should already be fixed in 1.8.7:
http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/23583
But it does not appear to be. Example with 1.8.7-p248:
# apply patch from above link
% make distclean
% ./configure --enable-pthread
% make
% time ./ruby -e '1_000_000.times { x = 2 ** 256 }'
./ruby -e '1_000_000.times { x = 2 ** 256 }' 1.45s user 0.34s system 99% cpu 1.809 total
% strace ./ruby -e '1_000.times { x = 2 ** 256 }' 2>&1 | grep -c sigproc
1009
% make distclean
% ./configure --disable-ucontext --enable-pthread
% make
% time ./ruby -e '1_000_000.times { x = 2 ** 256 }'
./ruby -e '1_000_000.times { x = 2 ** 256 }' 1.16s user 0.00s system 99% cpu 1.171 total
% strace ./ruby -e '1_000.times { x = 2 ** 256 }' 2>&1 | grep -c sigproc
3
% ./ruby -v
ruby 1.8.7 (2009-12-24 patchlevel 248) [i686-linux]
----------------------------------------
http://redmine.ruby-lang.org
Dan Peterson
It's entirely possible this is a duplicate but after 10 minutes searching via both the Redmine interface and Google focused on this site I couldn't find it already reported.
----------------------------------------
http://redmine.ruby-lang.org/issues/show/2553
----------------------------------------
http://redmine.ruby-lang.org
Dan Peterson
http://redmine.ruby-lang.org/issues/show/2553
----------------------------------------
http://redmine.ruby-lang.org
You received this message because you are subscribed to the Google Groups "ruby-core-google" group.
To post to this group, send email to ruby-cor...@googlegroups.com.
To unsubscribe from this group, send email to ruby-core-goog...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/ruby-core-google?hl=en.
Roger Pack
I thought this was fixed in 1.8.7
Could you verify which branches (1.8.8, 1.8.7, 1.8.6) still show this problem?
That might help.
Patrick Mohr
> I thought this was fixed in 1.8.7
> Could you verify which branches (1.8.8, 1.8.7, 1.8.6) still show this problem?
I just ran the same tests on 1.8.6 and 1.8.7. It's still not fixed. I'm not sure where the source code for 1.8.8 is so I didn't test it.
The build information and test results are below. I didn't test 1.8.7 with --disable-ucontext because that configure flag doesn't seem to exist anymore.
Admittedly, ruby 1.8.7 is much faster than 1.8.6, but 1.8.7 still has this bug.
I would be happy to test 1.8.8 if you would tell me where to get the source code.
------------
1.8.6 with pthreads test results:
# svn co http://svn.ruby-lang.org/repos/ruby/branches/ruby_1_8_6
# autoconf && ./configure --enable-pthread && make
# ./ruby -v
ruby 1.8.6 (2010-06-12 patchlevel 415) [i686-linux]
# time ./ruby -e '1_000_000.times { x = 2 ** 256 }'
real 0m6.462s
user 0m6.128s
sys 0m0.308s
------------
1.8.6 with pthreads enabled and ucontext disabled
# svn co http://svn.ruby-lang.org/repos/ruby/branches/ruby_1_8_6
# autoconf && ./configure --disable-ucontext --enable-pthread && make
# ./ruby -v
ruby 1.8.6 (2010-06-12 patchlevel 415) [i686-linux]
# time ./ruby -e '1_000_000.times { x = 2 ** 256 }'
real 0m5.582s
user 0m5.548s
sys 0m0.000s
------------
1.8.7 with pthreads
# svn co http://svn.ruby-lang.org/repos/ruby/branches/ruby_1_8_7
# autoconf && ./configure --enable-pthread && make
# ./ruby -v
ruby 1.8.7 (2010-06-16 patchlevel 296) [i686-linux]
# time ./ruby -e '1_000_000.times { x = 2 ** 256 }'
real 0m3.241s
user 0m2.960s
sys 0m0.260s
------------
1.8.7 with --disable-pthread
# svn co http://svn.ruby-lang.org/repos/ruby/branches/ruby_1_8_7
# autoconf && ./configure --disable-pthread && make
# ./ruby -v
ruby 1.8.7 (2010-06-16 patchlevel 296) [i686-linux]
# time ./ruby -e '1_000_000.times { x = 2 ** 256 }'
real 0m2.535s
user 0m2.512s
sys 0m0.004s
Motohiro KOSAKI
ruby 1.8.8 have the same issue.
$ autoconf && ./configure --disable-pthread && make
$ time ./ruby -ve '1_000_000.times { x = 2 ** 256 }'
ruby 1.8.8dev (2010-06-15) [x86_64-linux]
real 0m1.176s
user 0m1.171s
sys 0m0.005s
$ autoconf && ./configure --enable-pthread && make
[kosaki@kosaopt ruby_1_8]$ time ./ruby -ve '1_000_000.times { x = 2 ** 256 }'
ruby 1.8.8dev (2010-06-15) [x86_64-linux]
real 0m1.513s
user 0m1.340s
sys 0m0.173s
$ strace -c ./ruby -e '1_000_000.times { x = 2 ** 256 }'
% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ----------------
99.98 0.160160 0 1000011 rt_sigprocmask
0.02 0.000025 2 14 mprotect
0.00 0.000000 0 8 read
0.00 0.000000 0 19 10 open
0.00 0.000000 0 9 close
0.00 0.000000 0 4 3 stat
0.00 0.000000 0 9 fstat
0.00 0.000000 0 26 mmap
0.00 0.000000 0 1 munmap
0.00 0.000000 0 7 brk
0.00 0.000000 0 14 rt_sigaction
0.00 0.000000 0 1 1 access
0.00 0.000000 0 1 execve
0.00 0.000000 0 6 getrlimit
0.00 0.000000 0 1 getuid
0.00 0.000000 0 1 getgid
0.00 0.000000 0 2 geteuid
0.00 0.000000 0 2 getegid
0.00 0.000000 0 1 arch_prctl
0.00 0.000000 0 2 1 futex
0.00 0.000000 0 1 set_tid_address
0.00 0.000000 0 1 set_robust_list
------ ----------- ----------- --------- --------- ----------------
100.00 0.160185 1000141 15 total
summary of ruby_1_8 code
node.h
------------------------------------------
#if defined(HAVE_GETCONTEXT) && defined(HAVE_SETCONTEXT)
#include <ucontext.h>
#define USE_CONTEXT
#endif
#include <setjmp.h>
#include "st.h"
The syscam can use getcontext(), USE_CONTEXT is always set.
eval.c
------------------------------------------
#ifdef USE_CONTEXT
(snip)
# define ruby_setjmp(just_before_setjmp, j) ((j)->status = 0, \
(just_before_setjmp), \
PRE_GETCONTEXT, \
getcontext(&(j)->context), \
POST_GETCONTEXT, \
(j)->status)
if USE_CONTEXT is defined, ruby_setjmp() mean to call getcontext().
and linux getcontext() implementation call rt_sigprocmask() internally.
Motohiro KOSAKI
note: 1.9.x tree don't have this issue beucause ruby_setjmp() don't use getcontext().
Motohiro KOSAKI
This seems to be introduced following commit.
commit 9fdbc41973971f4ef032d57c4e02ed0430227a0a
Author: nobu <nobu@b2dd03c8-39d4-4d8f-98ff-823fe69b080e>
Date: Sat Dec 13 00:01:28 2003 +0000
* configure.in: check ucontext.h.
* eval.c: use getcontext/setcontext() instead of setjmp/longjmp()
on ia64 or with native thread enabled. [ruby-core:01932]
git-svn-id: svn+ssh://ci.ruby-lang.org/ruby/trunk@5180 b2dd03c8-39d4-4d8f-98ff-823fe69b080e
but unfortunatelly, [ruby-core:01932] has alomost zero information ;)
http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/1932
Kirk Haines
> Issue #2553 has been updated by Patrick Mohr.
>
>
>> I thought this was fixed in 1.8.7
>> Could you verify which branches (1.8.8, 1.8.7, 1.8.6) still show this problem?
>
> I just ran the same tests on 1.8.6 and 1.8.7. It's still not fixed. I'm not sure where the source code for 1.8.8 is so I didn't test it.
>
> The build information and test results are below. I didn't test 1.8.7 with --disable-ucontext because that configure flag doesn't seem to exist anymore.
>
> Admittedly, ruby 1.8.7 is much faster than 1.8.6, but 1.8.7 still has this bug.
http://redmine.ruby-lang.org/issues/show/2553
I did fix this on 1.8.6: r27999
./configure --enable-pthread
# time ./ruby -ve '1_000_000.times { x = 2 ** 256 }'
ruby 1.8.6 (2010-06-12 patchlevel 415) [x86_64-linux]
real 0m3.006s
user 0m2.880s
sys 0m0.128s
./configure --enable-pthread --disable-ucontext
# time ./ruby -ve '1_000_000.times { x = 2 ** 256 }'
ruby 1.8.6 (2010-06-12 patchlevel 415) [x86_64-linux]
real 0m2.733s
user 0m2.728s
sys 0m0.004s
Kirk Haines
Roger Pack
Thanks for looking into this one.
-r
Motohiro KOSAKI
I and nobu digged previous discussion awhile. Now, we have concluded getcontext() is only necessary when following cases
1) Using linuxthreads (it has broken signal model)
2) cpu arch has register stack
Motohiro KOSAKI
Category set to core
Assigned to changed from Yukihiro Matsumoto to Shyouhei Urabe
Priority changed from Normal to Low
This issue has been fixed by commit r28404 (ruby_1_8 branch).
Andre Nathan
Any chance of backporting the fix to the ruby_1_8_7 branch?
Thanks,
Andre
Motohiro KOSAKI
There is.
But unfortunately 1.8.7-p299 was released very recently. So you need to wait about half year ;-)
Also, If anyone will find a bug in this patch, our chance will disappear....
Shyouhei Urabe
Oh, sorry for the inconvenience... I was not aware of this. It should be backported.
Andre Nathan
Hello
I was trying to update the ubuntu ruby 1.8.7 package with this patch, but I'm still seeing the bad performance. The patch has the following line:
if test -n "`(/lib/libc.so.6 2>/dev/null | fgrep 'Native POSIX Threads') 2> /dev/null`"; then
use_context=yes
fi
Isn't the use of setcontext necessary only for linuxthreads, i.e. if the system doesn't have native posix threads? In that case, shouldn't the test be "test -z", so that use_context is set to yes if "Native POSIX Threads" does *not* appear in that output?
Apologies if I have misunderstood the patch.
Best regards,
Andre
KOSAKI Motohiro
> Issue #2553 has been updated by Andre Nathan.
>
>
> Hello
>
> I was trying to update the ubuntu ruby 1.8.7 package with this patch, but I'm still seeing the bad performance. The patch has the following line:
>
> if test -n "`(/lib/libc.so.6 2>/dev/null | fgrep 'Native POSIX Threads') 2> /dev/null`"; then
> use_context=yes
> fi
>
> Isn't the use of setcontext necessary only for linuxthreads, i.e. if the system doesn't have native posix threads? In that case, shouldn't the test be "test -z", so that use_context is set to yes if "Native POSIX Threads" does *not* appear in that output?
>
> Apologies if I have misunderstood the patch.
Oops, this patch only works on 32bit machine, because some 64bit
distro don't have /lib/libc.so.6 (instead they use /lib64), and this
condition select use_context=yes.
I'll update this soon.
And, if you are using 32bit distro, please show your /lib/libc.so.6
execution result.
I'm very sorry this.
KOSAKI Motohiro
following patch works both 32bit and 64bit on my environment.
$ svn diff
Index: configure.in
===================================================================
--- configure.in (リビジョン 28594)
+++ configure.in (作業コピー)
@@ -1151,7 +1151,7 @@
if test x"$rb_with_pthread" = xyes; then
AS_CASE("$target_cpu:$target_os:$cross_compiling",
[*:linux*:no], [
- if test -n "`(/lib/libc.so.6 2>/dev/null | fgrep 'Native
POSIX Threads') 2> /dev/null`"; then
+ if test -n "`(/lib/libc.so.6 2>/dev/null | fgrep
'linuxthreads') 2> /dev/null`"; then
use_context=yes
fi
],
KOSAKI Motohiro
Committed.
Revision 28595 and 28597.
Thanks Andre, you are great.
Andre Nathan
The new patch fixed this issue for me. Thanks a lot!
Here's hoping that the patch will be merged into ruby_1_8_7. I believe it'll make it easier for distributions who package this version to have this performance bug fixed.
Thanks again!
Shyouhei Urabe
Status changed from Assigned to Closed
% Done changed from 0 to 100
This issue was solved with changeset r29854.
Dan, thank you for reporting this issue.
Your contribution to Ruby is greatly appreciated.
May Ruby be with you.