[ANN] TCS 1.9.3-p179 experimental build

146 views
Skip to first unread message

Jon

unread,
Apr 13, 2012, 4:11:56 PM4/13/12
to theco...@googlegroups.com
TCS Ruby 1.9.3 => MRI 1.9.3 performance optimizations for Windows and *nix users.

*nix users can try out TCS improvements thanks to Yura's multi-platform optimizations and his 1.9.3 backport of the COW-friendly, Bitmap Marking MRI 2.0.0 GC. Simply use your good friend RVM or rbenv to get started...and tell Michał Papis, THANK YOU.

https://github.com/thecodeshop/ruby/wiki/FAQ#wiki-rvm
https://github.com/thecodeshop/ruby/wiki/FAQ#wiki-rbenv

Windows users can download the binary archive `tcs-ruby193_require_fenix_gc_hash_20120413.7z` built from the https://github.com/thecodeshop/ruby/tree/tcs-ruby_1_9_3 branch from https://github.com/thecodeshop/ruby/wiki/Downloads

Thanks to Hiroshi's and Luis' continued efforts, Fenix (aka faster `File.expand_path` from a Window's API implementation) is fully integrated and enabled by default. You'll should see faster rails startup times and faster requires.

While the existing ruby-core test suite passes when building on both Windows and *nix, we may discover issues that require further Fenix refinements. That said, I'm burning the boats and fully committing to integrating Fenix for future TCS releases.

Finally, for those using TCS Ruby 1.9.3, let us know how its working for you. Blog it. Tweet it. Share it. If you like the optimizations and want them in a future official 1.9.3, advocate for them with ruby-core.

We'll continue the hard work of maintaining 1.9.3 pre-backports for all the optimizations ;)


=== Notable Updates from Previous Release ===

* Yura's trunk-accepted Hash opts and new spinlock protected pool allocation
* Yura's expanded_path update to his cached $LOAD_PATH optimization
* Hiroshi's fileload, Fenix tweaks, and Fenix integration optimizations
* Official ruby-core 1.9.3 backports as of ruby_1_9_3@35295
* OpenSSL 1.0.0h (Windows binary)
* RubyGems 1.8.21 (Windows binary)


=== Built and Tested On ===

* Win7 32bit/MinGW 4.6.2 (1)
* Win7 32bit/Windows SDK 7.1 (2)
* Arch 3.3.1 32bit/GCC 4.7.0 (3)
* Ubuntu 11.10 32bit
* Debian Squeeze 64bit

(1) make test && make test-all TESTS='openssl fiddle psych zlib io json mkmf pathname stringio erb fileutils ruby/test_io.rb ruby/test_io_m17n.rb ruby/test_file.rb' [1 test_utime failure, 1 test_invalid_advise(TestIO) Errno::EBADF error]
(2) nmake test
(3) make test && make test-all [2 version string failures, 2 WEBrick failures]


=== Build Recipe Summary ===

git checkout tcs-ruby_1_9_3
git reset --hard codeshop/ruby_1_9_3
git rebase winstat/ruby_1_9_3
git rebase st_opt/ruby_1_9_3
git rebase win-file/ruby_1_9_3
git rebase cached-lp/ruby_1_9_3
git rebase sorted-lf/ruby_1_9_3
git rebase backport-gc/ruby_1_9_3
rake ruby19 local=C:\Users\Jon\Documents\RubyDev\ruby-git dkver=mingw-32-4.6.2
# run tests


=== Rails 3.2.3 (Win7 32bit) Startup Time Improvement ===

# `rails new rails-empty`
C:\rails-empty>ruby --version
ruby 1.9.3p125 (2012-02-16) [i386-mingw32]
C:\rails-empty>timer ruby script\rails r "puts %Q{$: = %s\n$\" = %s} % [ $LOAD_PATH.size, $LOADED_FEATURES.size ]"
$: = 59
$" = 745
real 7.778
system 5.834
user 1.840

# `rails new rails-empty`
C:\rails-empty>ruby --version
tcs-ruby 1.9.3p179 (2012-04-11, TCS patched 2012-04-11) [i386-mingw32]
C:\rails-empty>timer ruby script\rails r "puts %Q{$: = %s\n$\" = %s} % [ $LOAD_PATH.size, $LOADED_FEATURES.size ]"
$: = 59
$" = 750
real 1.981
system 0.920
user 1.014


Jon

---
Fail fast. Fail often. Fail publicly. Learn. Adapt. Repeat.
http://thecodeshop.github.com | http://jonforums.github.com/
twitter: @jonforums

Jarmo Pertman

unread,
Apr 14, 2012, 3:55:32 AM4/14/12
to theco...@googlegroups.com
There's no need to have RUBYOPT -rfenix/replace anymore? :) That's awesome! Thanks!

Jarmo

Jarmo Pertman

unread,
Apr 14, 2012, 5:44:06 AM4/14/12
to theco...@googlegroups.com
I'm seeing segfault now when using this build with eventmachine:


C:\Users\media\Documents\Projects\Ruby>ruby -reventmachine -e "EM.run {}"
C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/gems/1.9.1/gems/eventmachine-1.0.0.beta.4.1-x86-mingw32/lib/eventmachine.rb:179: [BUG] Segmenta
tion fault
tcs-ruby 1.9.3p179 (2012-04-11, TCS patched 2012-04-11) [i386-mingw32]

-- Control frame information -----------------------------------------------
c:0005 p:---- s:0016 b:0016 l:000015 d:000015 CFUNC  :run_machine
c:0004 p:0248 s:0013 b:0013 l:000012 d:000012 METHOD C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/gems/1.9.1/gems/eventmachine-1.0.0.beta.4.
1-x86-mingw32/lib/eventmachine
c:0003 p:0015 s:0006 b:0006 l:0004cc d:001e1c EVAL   -e:1
c:0002 p:---- s:0004 b:0004 l:000003 d:000003 FINISH
c:0001 p:0000 s:0002 b:0002 l:0004cc d:0004cc TOP

-- Ruby level backtrace information ----------------------------------------
-e:1:in `<main>'
C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/gems/1.9.1/gems/eventmachine-1.0.0.beta.4.1-x86-mingw32/lib/eventmachine.rb:179:in `run'
C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/gems/1.9.1/gems/eventmachine-1.0.0.beta.4.1-x86-mingw32/lib/eventmachine.rb:179:in `run_machine
'

-- C level backtrace information -------------------------------------------
C:\Windows\SysWOW64\ntdll.dll(ZwWaitForSingleObject+0x15) [0x77bef8b1]
C:\Windows\syswow64\kernel32.dll(WaitForSingleObjectEx+0x43) [0x75991194]
C:\Windows\syswow64\kernel32.dll(WaitForSingleObject+0x12) [0x75991148]
C:\tcs-ruby193_require_fenix_gc_hash_20120413\bin\msvcrt-ruby191.dll(rb_vm_bugreport+0xf8) [0x62e49da0]
C:\tcs-ruby193_require_fenix_gc_hash_20120413\bin\msvcrt-ruby191.dll(rb_name_err_mesg_new+0x5f9) [0x62d372dd]
C:\tcs-ruby193_require_fenix_gc_hash_20120413\bin\msvcrt-ruby191.dll(rb_bug+0x2f) [0x62d37f5f]
C:\tcs-ruby193_require_fenix_gc_hash_20120413\bin\msvcrt-ruby191.dll(rb_check_safe_str+0x19c) [0x62de057c]
 [0x004011e6]
C:\Windows\syswow64\kernel32.dll(GetProfileStringW+0x12aa3) [0x759d003f]
C:\Windows\SysWOW64\ntdll.dll(RtlKnownExceptionFilter+0xb7) [0x77c474df]

-- Other runtime information -----------------------------------------------

* Loaded script: -e

* Loaded features:

    0 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/1.9.1/i386-mingw32/enc/trans/utf_16_32.so
    1 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/1.9.1/i386-mingw32/enc/iso_8859_13.so
    2 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/1.9.1/i386-mingw32/enc/encdb.so
    3 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/1.9.1/i386-mingw32/enc/trans/transdb.so
    4 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/1.9.1/dl/func.rb
    5 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/1.9.1/thread.rb
    6 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/gems/1.9.1/gems/eventmachine-1.0.0.beta.4.1-x86-mingw32/lib/em/threaded_resource.rb
    7 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/1.9.1/i386-mingw32/enc/utf_16le.so
    8 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/gems/1.9.1/gems/eventmachine-1.0.0.beta.4.1-x86-mingw32/lib/em/spawnable.rb
    9 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/gems/1.9.1/gems/eventmachine-1.0.0.beta.4.1-x86-mingw32/lib/em/deferrable.rb
   10 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/1.9.1/i386-mingw32/fiddle.so
   11 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/1.9.1/fiddle.rb
   12 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/gems/1.9.1/gems/eventmachine-1.0.0.beta.4.1-x86-mingw32/lib/eventmachine.rb
   13 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/gems/1.9.1/gems/eventmachine-1.0.0.beta.4.1-x86-mingw32/lib/1.9/rubyeventmachine.so
   14 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/gems/1.9.1/gems/eventmachine-1.0.0.beta.4.1-x86-mingw32/lib/rubyeventmachine.rb
   15 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb
   16 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/1.9.1/fiddle/closure.rb
   17 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/gems/1.9.1/gems/eventmachine-1.0.0.beta.4.1-x86-mingw32/lib/em/future.rb
   18 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/site_ruby/1.9.1/rubygems/deprecate.rb
   19 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/1.9.1/i386-mingw32/enc/trans/single_byte.so
   20 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/gems/1.9.1/gems/eventmachine-1.0.0.beta.4.1-x86-mingw32/lib/em/queue.rb
   21 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/1.9.1/dl/value.rb
   22 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/1.9.1/openssl/config.rb
   23 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/1.9.1/i386-mingw32/rbconfig.rb
   24 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/1.9.1/openssl/buffering.rb
   25 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/gems/1.9.1/gems/eventmachine-1.0.0.beta.4.1-x86-mingw32/lib/em/file_watch.rb
   26 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/gems/1.9.1/gems/eventmachine-1.0.0.beta.4.1-x86-mingw32/lib/em/process_watch.rb
   27 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/gems/1.9.1/gems/eventmachine-1.0.0.beta.4.1-x86-mingw32/lib/em/callback.rb
   28 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/1.9.1/dl/callback.rb
   29 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/1.9.1/dl/pack.rb
   30 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/1.9.1/dl/stack.rb
   31 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/gems/1.9.1/gems/eventmachine-1.0.0.beta.4.1-x86-mingw32/lib/em/buftok.rb
   32 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/1.9.1/openssl/x509-internal.rb
   33 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/1.9.1/openssl/ssl-internal.rb
   34 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/1.9.1/i386-mingw32/dl.so
   35 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/1.9.1/dl.rb
   36 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/gems/1.9.1/gems/eventmachine-1.0.0.beta.4.1-x86-mingw32/lib/em/channel.rb
   37 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/gems/1.9.1/gems/eventmachine-1.0.0.beta.4.1-x86-mingw32/lib/em/pool.rb
   38 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/1.9.1/i386-mingw32/openssl.so
   39 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/1.9.1/openssl.rb
   40 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/1.9.1/i386-mingw32/fcntl.so
   41 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/site_ruby/1.9.1/rubygems/defaults/operating_system.rb
   42 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/1.9.1/securerandom.rb
   43 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/site_ruby/1.9.1/rubygems/platform.rb
   44 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/1.9.1/openssl/bn.rb
   45 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/site_ruby/1.9.1/rubygems/version.rb
   46 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/gems/1.9.1/gems/eventmachine-1.0.0.beta.4.1-x86-mingw32/lib/em/version.rb
   47 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/site_ruby/1.9.1/rubygems/specification.rb
   48 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/gems/1.9.1/gems/eventmachine-1.0.0.beta.4.1-x86-mingw32/lib/em/connection.rb
   49 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/1.9.1/fiddle/function.rb
   50 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/gems/1.9.1/gems/eventmachine-1.0.0.beta.4.1-x86-mingw32/lib/em/completion.rb
   51 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/1.9.1/i386-mingw32/stringio.so
   52 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/gems/1.9.1/gems/eventmachine-1.0.0.beta.4.1-x86-mingw32/lib/em/tick_loop.rb
   53 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/1.9.1/openssl/cipher.rb
   54 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/gems/1.9.1/gems/eventmachine-1.0.0.beta.4.1-x86-mingw32/lib/em/streamer.rb
   55 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/1.9.1/dl/cparser.rb
   56 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/gems/1.9.1/gems/eventmachine-1.0.0.beta.4.1-x86-mingw32/lib/em/resolver.rb
   57 enumerator.so
   58 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/gems/1.9.1/gems/eventmachine-1.0.0.beta.4.1-x86-mingw32/lib/em/iterator.rb
   59 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/1.9.1/shellwords.rb
   60 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/gems/1.9.1/gems/eventmachine-1.0.0.beta.4.1-x86-mingw32/lib/em/processes.rb
   61 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/gems/1.9.1/gems/eventmachine-1.0.0.beta.4.1-x86-mingw32/lib/em/protocols.rb
   62 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/site_ruby/1.9.1/rubygems.rb
   63 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/site_ruby/1.9.1/rubygems/exceptions.rb
   64 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/gems/1.9.1/gems/eventmachine-1.0.0.beta.4.1-x86-mingw32/lib/em/timers.rb
   65 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/site_ruby/1.9.1/rubygems/defaults.rb
   66 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/1.9.1/dl/struct.rb
   67 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/1.9.1/i386-mingw32/socket.so
   68 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/1.9.1/socket.rb
   69 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/site_ruby/1.9.1/rubygems/requirement.rb
   70 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/1.9.1/dl/import.rb
   71 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/site_ruby/1.9.1/rubygems/path_support.rb
   72 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/1.9.1/i386-mingw32/digest.so
   73 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/1.9.1/digest.rb
   74 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/1.9.1/openssl/digest.rb
   75 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/1.9.1/timeout.rb
   76 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/1.9.1/win32/resolv.rb
   77 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/1.9.1/resolv.rb
   78 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/site_ruby/1.9.1/rubygems/dependency.rb
   79 C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/1.9.1/win32/registry.rb

[NOTE]
You may have encountered a bug in the Ruby interpreter or extension libraries.
Bug reports are welcome.


This application has requested the Runtime to terminate it in an unusual way.
Please contact the application's support team for more information.

C:\Users\media\Documents\Projects\Ruby>

Luis Lavena

unread,
Apr 14, 2012, 7:36:58 AM4/14/12
to theco...@googlegroups.com

Don't know if a pre built gem will work out of the box like this one, specially due FD_SETSIZE added to rubyinstaller recipe.

Sorry for top posting. Sent from mobile.

Jon

unread,
Apr 14, 2012, 9:55:04 AM4/14/12
to theco...@googlegroups.com, Jarmo Pertman
> I'm seeing segfault now when using this build with eventmachine:
> >
> >
> C:\Users\media\Documents\Projects\Ruby>ruby -reventmachine -e "EM.run {}"
> C:/tcs-ruby193_require_fenix_gc_hash_20120413/lib/ruby/gems/1.9.1/gems/eventmachine-1.0.0.beta.4.1-x86-mingw32/lib/eventmachine.rb:179:
> [BUG] Segmenta
> tion fault
> tcs-ruby 1.9.3p179 (2012-04-11, TCS patched 2012-04-11) [i386-mingw32]

On Win7 32bit, I get the same segfault when running on plain vanilla 1.9.3p179

C:\Users\Jon>ruby -reventmachine -ve "puts EM::VERSION; EM.run {}"
ruby 1.9.3p179 (2012-04-11 revision 35295) [i386-mingw32]
1.0.0.beta.4.1
C:/ruby193/lib/ruby/gems/1.9.1/gems/eventmachine-1.0.0.beta.4.1-x86-mingw32/lib/eventmachine.rb:179: [BUG] Segmentation fault
...

But I don't see a SEGV on the official released 1.9.3p125

C:\Users\Jon>ruby -reventmachine -ve "puts EM::VERSION; EM.run {}"
ruby 1.9.3p125 (2012-02-16) [i386-mingw32]
1.0.0.beta.4.1
...

Luis Lavena

unread,
Apr 14, 2012, 9:56:50 AM4/14/12
to theco...@googlegroups.com, Jarmo Pertman

Might be related to FD_SETSIZE, need to investigate since it will affect all the existing pre compiled binaries.

Sorry for top posting. Sent from mobile.

Hiroshi Shirosaki

unread,
Apr 14, 2012, 11:39:38 AM4/14/12
to theco...@googlegroups.com
> Might be related to FD_SETSIZE, need to investigate since it will affect all
> the existing pre compiled binaries.
>

https://github.com/oneclick/rubyinstaller/issues/104#issuecomment-4360131
https://github.com/eventmachine/eventmachine/pull/303#issuecomment-4401783

According to these comments, unmatched FD_SETSIZE seems to cause segfault.

I tried to build eventmachine with this patch. eventmachine started
without segfault, but crashed after Ctrl-C.


diff --git a/ext/project.h b/ext/project.h
index 893a4a8..8d55629 100644
--- a/ext/project.h
+++ b/ext/project.h
@@ -82,7 +82,7 @@ typedef int SOCKET;

#ifdef OS_WIN32
// 21Sep09: windows limits select() to 64 sockets by default, we
increase it to 1024 here (before including winsock2.h)
-#define FD_SETSIZE 1024
+// #define FD_SETSIZE 32767

#define WIN32_LEAN_AND_MEAN
#include <windows.h>


C:\Users\hiroshi\work\empty>ruby -reventmachine -ve "puts EM::VERSION;
EM.run {}"


tcs-ruby 1.9.3p179 (2012-04-11, TCS patched 2012-04-11) [i386-mingw32]

1.0.0.beta.4
C:/tcs4/lib/ruby/gems/1.9.1/gems/eventmachine-1.0.0.beta.4/lib/eventmachine.rb:1
86:in `run_machine': Interrupt
from C:/tcs4/lib/ruby/gems/1.9.1/gems/eventmachine-1.0.0.beta.4/lib/even
tmachine.rb:186:in `run'
from -e:1:in `<main>'

This application has requested the Runtime to terminate it in an unusual way.
Please contact the application's support team for more information.


--
Hiroshi Shirosaki

Luis Lavena

unread,
Apr 14, 2012, 11:43:43 AM4/14/12
to theco...@googlegroups.com
On Sat, Apr 14, 2012 at 12:39 PM, Hiroshi Shirosaki
<h.shi...@gmail.com> wrote:
>> Might be related to FD_SETSIZE, need to investigate since it will affect all
>> the existing pre compiled binaries.
>>
>
> https://github.com/oneclick/rubyinstaller/issues/104#issuecomment-4360131
> https://github.com/eventmachine/eventmachine/pull/303#issuecomment-4401783
>
> According to these comments, unmatched FD_SETSIZE seems to cause segfault.
>

I can't find any reference to these issues on the web. In theory, if
Ruby was build with FD_SETSIZE bigger than what EventMachine is using,
it will only affect the number of file descriptors from fd_set it will
iterate over using select():

http://support.microsoft.com/kb/111855

And shouldn't segfault as fd_set contains a counter with the number of FD in it.
http://marc.info/?l=openssl-users&m=109396896030470

I'm investigating this right now, FD_SETSIZE seems also affect Ruby:
http://bugs.ruby-lang.org/issues/6228

--
Luis Lavena
AREA 17
-
Perfection in design is achieved not when there is nothing more to add,
but rather when there is nothing more to take away.
Antoine de Saint-Exupéry

Luis Lavena

unread,
Apr 14, 2012, 12:03:59 PM4/14/12
to theco...@googlegroups.com
On Sat, Apr 14, 2012 at 12:43 PM, Luis Lavena <luisl...@gmail.com> wrote:
> On Sat, Apr 14, 2012 at 12:39 PM, Hiroshi Shirosaki
> <h.shi...@gmail.com> wrote:
>>> Might be related to FD_SETSIZE, need to investigate since it will affect all
>>> the existing pre compiled binaries.
>>>
>>
>> https://github.com/oneclick/rubyinstaller/issues/104#issuecomment-4360131
>> https://github.com/eventmachine/eventmachine/pull/303#issuecomment-4401783
>>
>> According to these comments, unmatched FD_SETSIZE seems to cause segfault.
>>
>
> I can't find any reference to these issues on the web. In theory, if
> Ruby was build with FD_SETSIZE bigger than what EventMachine is using,
> it will only affect the number of file descriptors from fd_set it will
> iterate over using select():
>
> http://support.microsoft.com/kb/111855
>
> And shouldn't segfault as fd_set contains a counter with the number of FD in it.
> http://marc.info/?l=openssl-users&m=109396896030470
>
> I'm investigating this right now, FD_SETSIZE seems also affect Ruby:
> http://bugs.ruby-lang.org/issues/6228
>

Worth to mention an issue that might sound similar from Python:

http://bugs.python.org/issue12287

checking if FD >= FD_SETSIZE

Will be we could do a debug build of both ruby and eventmachine and
get this through GDB.

Jon

unread,
Apr 14, 2012, 1:41:30 PM4/14/12
to theco...@googlegroups.com


On Saturday, April 14, 2012 12:03:59 PM UTC-4, Luis Lavena wrote:
On Sat, Apr 14, 2012 at 12:43 PM, Luis Lavena <luisl...@gmail.com> wrote:
> On Sat, Apr 14, 2012 at 12:39 PM, Hiroshi Shirosaki
> <h.shi...@gmail.com> wrote:
>>> Might be related to FD_SETSIZE, need to investigate since it will affect all
>>> the existing pre compiled binaries.
>>>
>>
>> https://github.com/oneclick/rubyinstaller/issues/104#issuecomment-4360131
>> https://github.com/eventmachine/eventmachine/pull/303#issuecomment-4401783
>>
>> According to these comments, unmatched FD_SETSIZE seems to cause segfault.
>>
>
> I can't find any reference to these issues on the web. In theory, if
> Ruby was build with FD_SETSIZE bigger than what EventMachine is using,
> it will only affect the number of file descriptors from fd_set it will
> iterate over using select():
>
> http://support.microsoft.com/kb/111855
>
> And shouldn't segfault as fd_set contains a counter with the number of FD in it.
> http://marc.info/?l=openssl-users&m=109396896030470
>
> I'm investigating this right now, FD_SETSIZE seems also affect Ruby:
> http://bugs.ruby-lang.org/issues/6228
>

Worth to mention an issue that might sound similar from Python:

http://bugs.python.org/issue12287

checking if FD >= FD_SETSIZE

Will be we could do a debug build of both ruby and eventmachine and
get this through GDB.


gdb (on plain vanilla 1.9.3-p179 built with RubyInstaller's -DFD_SETSIZE override) says it can't access the memory location when dereferencing a pointer in win32/win32.c:2715....

C:\Users\Jon>ruby --version

ruby 1.9.3p179 (2012-04-11 revision 35295) [i386-mingw32]

C:\Users\Jon>gdb --args c:\ruby193\bin\ruby.exe -reventmachine -e "EM.run {}"
GNU gdb (GDB) 7.4
...
Reading symbols from c:\ruby193\bin\ruby.exe...done.
(gdb) run
Starting program: c:\ruby193\bin\ruby.exe -reventmachine -e EM.run\ \{\}
[New Thread 3612.0xfcc]
[New Thread 3612.0xc7c]

Program received signal SIGSEGV, Segmentation fault.
0x62e59ce5 in rb_w32_select_with_thread (nfds=1, rd=0x22ca68, wr=0x22da6c, ex=0x22ea70,
    timeout=0x22fa74, th=0x0)
    at ../../../../Users/Jon/Documents/RubyDev/ruby-git/win32/win32.c:2715
warning: Source file is more recent than executable.
2715                    if (rd) orig_rd = *rd;
(gdb) bt 2
#0  0x62e59ce5 in rb_w32_select_with_thread (nfds=1, rd=0x22ca68, wr=0x22da6c,
    ex=0x22ea70, timeout=0x22fa74, th=0x0)
    at ../../../../Users/Jon/Documents/RubyDev/ruby-git/win32/win32.c:2715
#1  0x62e5cd95 in rb_w32_select@20 (nfds=4, rd=0x22ca68, wr=0x22da6c, ex=0x22ea70,
    timeout=0x22fa74)
    at ../../../../Users/Jon/Documents/RubyDev/ruby-git/win32/win32.c:2749
(More stack frames follow...)
(gdb) l
2710                    struct timeval *dowait = &wait;
2711
2712                    fd_set orig_rd;
2713                    fd_set orig_wr;
2714                    fd_set orig_ex;
2715                    if (rd) orig_rd = *rd;
2716                    if (wr) orig_wr = *wr;
2717                    if (ex) orig_ex = *ex;
2718                    r = do_select(nfds, rd, wr, ex, &zero); // polling
2719                    if (r != 0) break; // signaled or error
(gdb) p rd
$1 = (fd_set *) 0x22ca68
(gdb) p /x *rd
Cannot access memory at address 0x22ca68

Jon

unread,
Apr 14, 2012, 5:01:28 PM4/14/12
to theco...@googlegroups.com
Until we fix root cause, I've yanked the original Windows binary archive and replaced it with a new version (tcs-ruby193_require_fenix_gc_hash_20120413-2.7z) built *without* -DFD_SETSIZE.  The new version doesn't SEGV on my Win7 32bit machine with EM.

Download from: https://github.com/thecodeshop/ruby/wiki/Downloads

Jarmo...THANK YOU for testing so quickly...let us know how the new version works.

Jon

Jarmo Pertman

unread,
Apr 17, 2012, 1:34:27 PM4/17/12
to theco...@googlegroups.com
It seems to be working now :)

Thanks!

Jarmo

Jon

unread,
Apr 14, 2012, 1:22:23 PM4/14/12
to theco...@googlegroups.com, Jarmo Pertman
gdb says it can't access the memory location when dereferencing a pointer in win32/win32.c:2715....

C:\Users\Jon>ruby --version
ruby 1.9.3p179 (2012-04-11 revision 35295) [i386-mingw32]

Jon

Dušan D. Majkić

unread,
Aug 22, 2012, 4:31:08 AM8/22/12
to theco...@googlegroups.com
No errors on 1.9.3p231. (I stopped it manually with Ctrl+C)

C:\Users\dmajkic>ruby --version
tcs-ruby 1.9.3p231 (2012-05-25, TCS patched 2012-05-27) [i386-mingw32]

C:\Users\dmajkic>ruby -reventmachine -e "EM.run {}"
G:/Ruby193/lib/ruby/gems/1.9.1/gems/eventmachine-1.0.0.rc.4-x86-mingw32/lib/eventmachine.rb:187:in
`run_machine': Interrupt
from G:/Ruby193/lib/ruby/gems/1.9.1/gems/eventmachine-1.0.0.rc.4-x86-mingw32/lib/eventmachine.rb:187:in
`run'
from -e:1:in `<main>'

C:\Users\dmajkic>

Jarmo Pertman

unread,
Aug 22, 2012, 6:03:36 AM8/22/12
to theco...@googlegroups.com
I don't understand why this old thread came alive :) For some reason google sent me an e-mail about the update done by Jon in 14th of April. The problem is solved long time ago.

Jarmo
Reply all
Reply to author
Forward
0 new messages