shmay, Kennon, what platforms are you running REE on?
--
Phusion | The Computer Science Company
Web: http://www.phusion.nl/
E-mail: in...@phusion.nl
Chamber of commerce no: 08173483 (The Netherlands)
Hi Amos.
Do you happen to have a spare staging server around on which you can
reproduce this problem? So far we haven't been able to find the cause of
this problem, or even seen the problem ourselves. If you had saved the
gdb backtrace somewhere then we'd be happy to take a look at it.
Also, finalize_list does not have any infinite loops. :) But I don't
exclude the possibility that something very weird might be going on
which causes finalize_list to freeze anyway.
Finally, what platform are your servers running on?
Actually I was referring to Ruby Enterprise Edition.
But if you need to debug an application that has gone crazy, please try
to generate a backtrace for it and post it to this mailing list. You can
do it as follows:
1. Identify the PID of the process that has gone crazy.
2. Type:
sudo gdb
attach 12345 <--- replace "12345" with the actual PID
thread apply all bt
Please copy & paste the result.
--
Phusion | The Computer Science Company
Web: http://www.phusion.nl/
E-mail: in...@phusion.nl
Thanks. I seem to have missed that one.
> We're using:
>
> Centos 5 x86_64
> Rails 2.1.0 (although the above trace was in Rails 1.2.3)
Are other people who experience the same problem on 64-bit as well?
--
Phusion | The Computer Science Company
Web: http://www.phusion.nl/
E-mail: in...@phusion.nl
Thanks for the link. I seem to have missed that message.
> We're using:
>
> Centos 5 x86_64
> Rails 2.1.0 (although the above trace was in Rails 1.2.3)
>
> Some libraries:
> five_runs memcache-client with our crc32 extension (http://github.com/
> fiveruns/memcache-client/tree/master)
> our version of data_fabric (which had some issues with passenger
> reopening db connections in spawn_server since data_fabric does not
> define a :production db connection)
> rmagick
> hpricot
I've been spending some time stress testing a sample Rails app which
uses FiveRuns's memcache-client and RMagick with REE on 64-bit Ubuntu
8.10 server. So far I haven't been able to find any stability issues.
--
Phusion | The Computer Science Company
Web: http://www.phusion.nl/
E-mail: in...@phusion.nl
OK.
The stack trace you gave previously might not be accurate because of
compiler optimizations. Could you install REE without optimizations? You
can do that by setting the CFLAGS environment variable to an empty
string, then running the installer:
export CFLAGS=
./installer
And I'm also wondering whether your REE installation uses the system
Ruby's gems. By default, REE adds the system Ruby's gem path to its own
gem path, allowing it to use already-installed gems. But I've received a
few reports from users who say that this can cause crashes when REE
tries to load a native extension that's compiled for the system's Ruby.
So you are advised to reinstall all your gems for REE, e.g. with
'/path/to/ree/bin/gem install imagemagick'.
--
Phusion | The Computer Science Company
Web: http://www.phusion.nl/
E-mail: in...@phusion.nl
I've been spending some time reading the Ruby source code and glibc
source code for clues. I haven't found any so far, but I suspect it
might have something to do with the way we handle Unix sockets.
Could you try the following:
1. Create a script, say '/usr/bin/ruby-wrapper-script', which sets
PASSENGER_NO_ABSTRACT_NAMESPACE_SOCKETS=1 and executes Ruby, like this:
#!/bin/bash
export PASSENGER_NO_ABSTRACT_NAMESPACE_SOCKETS=1
exec ruby "$@"
2. Set the PassengerRuby config option to your wrapper script.
--
Phusion | The Computer Science Company
Web: http://www.phusion.nl/
E-mail: in...@phusion.nl
Very interesting. Can others confirm whether this money patch works and
whether the problem description is correct?
--
Phusion | The Computer Science Company
Web: http://www.phusion.nl/
E-mail: in...@phusion.nl
Great. I've talked to the Rails core team about this problem, and they
think that fixing it in Rails is probably the best solution.
I still haven't figured out *why* this problem occurs so I'm going to
investigate it a bit more.
--
Phusion | The Computer Science Company
Web: http://www.phusion.nl/
E-mail: in...@phusion.nl
Hi
I am also seeing this, and it does not seem to be intermittent, if I leave the server alone for a few minutes CPU will stay at close to 100%.
Any help would be appreciated.
Distro: Amazon Linux 64bit
uname -a
Linux 2.6.35.14-106.53.amzn1.x86_64 #1 SMP Fri Jan 6 16:20:10 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux
Apache:
Server version: Apache/2.2.22 (Unix)
#ruby -v
ruby 1.8.7 (2011-12-28 patchlevel 357) [x86_64-linux]
Rails 3.2.5
Passenger: 3.0.13
gbd output:
GNU gdb (GDB) Amazon Linux (7.2-50.11.amzn1)
Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-amazon-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
(gdb) attach 18897
Attaching to process 18897
Reading symbols from /usr/bin/ruby1.8...(no debugging symbols found)...done.
Reading symbols from /usr/lib64/libruby.so.1.8...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/libruby.so.1.8
Reading symbols from /lib64/libpthread.so.0...(no debugging symbols found)...done.
[Thread debugging using libthread_db enabled]
[New Thread 0x7f91baa83700 (LWP 18935)]
Loaded symbols for /lib64/libpthread.so.0
Reading symbols from /lib64/librt.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib64/librt.so.1
Reading symbols from /lib64/libdl.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib64/libdl.so.2
Reading symbols from /lib64/libcrypt.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib64/libcrypt.so.1
Reading symbols from /lib64/libm.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib64/libm.so.6
Reading symbols from /lib64/libc.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib64/libc.so.6
Reading symbols from /lib64/libgcc_s.so.1...(no debugging symbols found)...done.
Loaded symbols for /lib64/libgcc_s.so.1
Reading symbols from /lib64/ld-linux-x86-64.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib64/ld-linux-x86-64.so.2
Reading symbols from /lib64/libfreebl3.so...(no debugging symbols found)...done.
Loaded symbols for /lib64/libfreebl3.so
Reading symbols from /usr/lib64/ruby/1.8/x86_64-linux/socket.so...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/ruby/1.8/x86_64-linux/socket.so
Reading symbols from /usr/lib/ruby/gems/1.8/gems/passenger-3.0.12/ext/ruby/ruby-1.8.7-x86_64-linux/passenger_native_support.so...done.
Loaded symbols for /usr/lib/ruby/gems/1.8/gems/passenger-3.0.12/ext/ruby/ruby-1.8.7-x86_64-linux/passenger_native_support.so
Reading symbols from /usr/lib64/ruby/1.8/x86_64-linux/thread.so...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/ruby/1.8/x86_64-linux/thread.so
Reading symbols from /usr/lib64/ruby/1.8/x86_64-linux/etc.so...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/ruby/1.8/x86_64-linux/etc.so
Reading symbols from /usr/lib64/ruby/1.8/x86_64-linux/fcntl.so...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/ruby/1.8/x86_64-linux/fcntl.so
Reading symbols from /usr/lib64/ruby/1.8/x86_64-linux/stringio.so...(no debugging symbols found)...done.
Loaded symbols for /usr/lib64/ruby/1.8/x86_64-linux/stringio.so
0x00007f91bbd328b3 in select () from /lib64/libc.so.6
Missing separate debuginfos, use: debuginfo-install ruby-1.8.7.357-1.18.amzn1.x86_64
(gdb) thread apply all bt
Thread 2 (Thread 0x7f91baa83700 (LWP 18935)):
#0 0x00007f91bc8b675b in pthread_cond_timedwait@@GLIBC_2.3.2 () from /lib64/libpthread.so.0
#1 0x00007f91bcaf9fac in ?? () from /usr/lib64/libruby.so.1.8
#2 0x00007f91bc8b27f1 in start_thread () from /lib64/libpthread.so.0
#3 0x00007f91bbd39ccd in clone () from /lib64/libc.so.6
Thread 1 (Thread 0x7f91bcfdc700 (LWP 18897)):
#0 0x00007f91bbd328b3 in select () from /lib64/libc.so.6
#1 0x00007f91bcb05f04 in rb_thread_schedule () from /usr/lib64/libruby.so.1.8
#2 0x00007f91bcb08149 in rb_thread_select () from /usr/lib64/libruby.so.1.8
#3 0x00007f91bcb220b4 in ?? () from /usr/lib64/libruby.so.1.8
#4 0x00007f91bcb02f1e in ?? () from /usr/lib64/libruby.so.1.8
#5 0x00007f91bcb030f7 in ?? () from /usr/lib64/libruby.so.1.8
#6 0x00007f91bcaffd54 in ?? () from /usr/lib64/libruby.so.1.8
#7 0x00007f91bcafd450 in ?? () from /usr/lib64/libruby.so.1.8
#8 0x00007f91bcafdd0e in ?? () from /usr/lib64/libruby.so.1.8
#9 0x00007f91bcafe6d5 in ?? () from /usr/lib64/libruby.so.1.8
#10 0x00007f91bcb02dd0 in ?? () from /usr/lib64/libruby.so.1.8
#11 0x00007f91bcb030f7 in ?? () from /usr/lib64/libruby.so.1.8
#12 0x00007f91bcaffd54 in ?? () from /usr/lib64/libruby.so.1.8
#13 0x00007f91bcb005be in ?? () from /usr/lib64/libruby.so.1.8
#14 0x00007f91bcb00471 in ?? () from /usr/lib64/libruby.so.1.8
#15 0x00007f91bcb005be in ?? () from /usr/lib64/libruby.so.1.8
#16 0x00007f91bcb02dd0 in ?? () from /usr/lib64/libruby.so.1.8
#17 0x00007f91bcb030f7 in ?? () from /usr/lib64/libruby.so.1.8
#18 0x00007f91bcaffe2d in ?? () from /usr/lib64/libruby.so.1.8
#19 0x00007f91bcb00471 in ?? () from /usr/lib64/libruby.so.1.8
#20 0x00007f91bcb0fb45 in ?? () from /usr/lib64/libruby.so.1.8
#21 0x00007f91bcb0fb85 in ruby_exec () from /usr/lib64/libruby.so.1.8
#22 0x00007f91bcb0fbb5 in ruby_run () from /usr/lib64/libruby.so.1.8
#23 0x0000000000400831 in main ()
(gdb)
--
You received this message because you are subscribed to the Google Groups "Phusion Passenger Discussions" group.
To view this discussion on the web visit https://groups.google.com/d/msg/phusion-passenger/-/r-pvlpq3vyYJ.
To post to this group, send email to phusion-...@googlegroups.com.
To unsubscribe from this group, send email to phusion-passen...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/phusion-passenger?hl=en.
--
You received this message because you are subscribed to the Google Groups "Phusion Passenger Discussions" group.
To post to this group, send email to phusion-...@googlegroups.com.
To unsubscribe from this group, send email to phusion-passen...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/phusion-passenger?hl=en.