Exception Errno::EBADF in spawn manager (Bad file descriptor)

[Justin Ramos]

I have Passenger 2.2.11 running with Ruby 1.9.1 on an Ubuntu 6.06
server. I am unable to spawn a passenger server, either from the
command line or through nginx.

#:~$ passenger-status
ERROR: Phusion Passenger doesn't seem to be running.

#:~$ passenger-spawn-server
*** Exception Errno::EBADF in spawn manager (Bad file descriptor)
(process 18774):
from /usr/local/lib/ruby191/gems/1.9.1/gems/passenger-2.2.11/bin/
passenger-spawn-server:37:in `initialize'
from /usr/local/lib/ruby191/gems/1.9.1/gems/passenger-2.2.11/bin/
passenger-spawn-server:37:in `new'
from /usr/local/lib/ruby191/gems/1.9.1/gems/passenger-2.2.11/bin/
passenger-spawn-server:37:in `<top (required)>'
from /usr/local/bin/passenger-spawn-server:19:in `load'
from /usr/local/bin/passenger-spawn-server:19:in `<main>'

#:~$ sudo /etc/init.d/nginx start
Starting nginx daemon: nginx.

#:~$ ps ax | grep nginx
18815 ? Ss 0:00 nginx: master process /usr/local/nginx/sbin/
nginx -c /usr/local/nginx/conf/nginx.conf
18816 ? S 0:00 nginx: worker
process
18818 pts/1 S+ 0:00 grep nginx

#:~$ ps ax | grep passenger
18820 pts/1 S+ 0:00 grep passenger

#:~$ tail -f /usr/local/nginx/logs/error.log
2010/03/31 12:45:37 [error] 18816#0: *1 connect() to unix:/tmp/
passenger.18801/master/helper_server.sock failed (111: Connection
refused) while connecting to upstream, client: 192.168.9.108, server:
www.foo.com, request: "GET / HTTP/1.1", upstream: "passenger://unix:/
tmp/passenger.18801/master/helper_server.sock:", host: "www.foo.com"

#:~$ sudo passenger-status
*** ERROR: Cannot query status for Passenger instance 18815:
No such file or directory - /tmp/passenger.18801/info/status.socket

Any ideas? I'm completely stuck.

Thanks.

-- Justin

[Hongli Lai]

The spawn server isn't supposed to be started from the command line.
It will fail if you try.

Is there nothing else in the logs? Nothing that indicates the reason
why the helper server has crashed?

--
Phusion | The Computer Science Company

Web: http://www.phusion.nl/
E-mail: in...@phusion.nl
Chamber of commerce no: 08173483 (The Netherlands)

[Justin Ramos]

No, I couldn't find any other useful info in the logs. I suspect it's
related to the server being so old and unmaintained, but I'm not sure.

I rolled back to 2.2.5, added "require 'stringio'" to lib/
phusion_passenger/utils.rb, and I'm back in business. We're using
2.2.5 on another app, so it made sense to keep the versions consistent
for now.

I'll spend more time with 2.2.11 when I have it and report back
anything else I can find.

-- Justin

On Mar 31, 5:17 pm, Hongli Lai <hon...@phusion.nl> wrote:

> E-mail: i...@phusion.nl

[Justin Ramos]

I found this in the logs on start-up:

[ pid=4741 file=ext/nginx/HelperServer.cpp:837 time=2010-04-07
15:38:00.762 ]:
Cannot set permissions on file /tmp/passenger.4739/.guard: Operation
not permitted (1)
in 'int main(int, char**)' (HelperServer.cpp:805)

#:~$ sudo ls -la /tmp/passenger.2080/
total 28
d-ws--x--x 6 root root 4096 2010-04-07 15:22 .
drwxrwxrwt 10 root root 4096 2010-04-07 15:22 ..
d-ws------ 2 webrun webrun 4096 2010-04-07 15:22 backends
-rw-r--r-- 1 webrun webrun 4 2010-04-07 15:22 control_process.pid
p--------- 1 root root 0 2010-04-07 15:22 .guard
drws------ 2 webrun webrun 4096 2010-04-07 15:22 info
d-ws--x--x 2 webrun webrun 4096 2010-04-07 15:22 master
d-ws------ 2 webrun webrun 4096 2010-04-07 15:22 webserver_private

So passenger is creating some files as root and some as webrun.

#:$ cat /usr/local/nginx/conf/nginx.conf | grep webrun
user webrun webrun;
passenger_default_user webrun; # force us to run as webrun

When I change webrun to root, everything works fine -- so it appears
to be related to trying to run with a different default user than the
one spawning the nginx process.

-- Justin

On Mar 31, 5:17 pm, Hongli Lai <hon...@phusion.nl> wrote:

> E-mail: i...@phusion.nl

[Hongli Lai]

Is /tmp mounted with nodev? If so try removing the nodev flag.

> --
> You received this message because you are subscribed to the Google Groups "Phusion Passenger Discussions" group.
> To post to this group, send email to phusion-...@googlegroups.com.
> To unsubscribe from this group, send email to phusion-passen...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/phusion-passenger?hl=en.
>
>

--
Phusion | The Computer Science Company

Web: http://www.phusion.nl/
E-mail: in...@phusion.nl

[Justin Ramos]

No, /tmp is mounted on /:

#:$ mount
/dev/sda1 on / type ext3 (rw,errors=remount-ro)
proc on /proc type proc (rw)
/sys on /sys type sysfs (rw)
varrun on /var/run type tmpfs (rw)
varlock on /var/lock type tmpfs (rw)
udev on /dev type tmpfs (rw)
devpts on /dev/pts type devpts (rw,gid=5,mode=620)
devshm on /dev/shm type tmpfs (rw)

-- Justin

> > For more options, visit this group athttp://groups.google.com/group/phusion-passenger?hl=en.

>
> --
> Phusion | The Computer Science Company
>
> Web:http://www.phusion.nl/

[Justin Ramos]

I also get these warnings when compiling the nginx extension:

#:$ sudo rake nginx
(in /usr/local/ruby191/lib/ruby/gems/1.9.1/gems/passenger-2.2.11)
g++ ext/nginx/HelperServer.cpp -o ext/nginx/HelperServer -Iext -Iext/
common -D_REENTRANT -I/usr/local/include -Wall -g -DPASSENGER_DEBUG -
DBOOST_DISABLE_ASSERTS ext/nginx/libpassenger_common.a ext/nginx/
libboost_oxt.a -lpthread
ext/nginx/HelperServer.cpp: In constructor ‘Server::Server(const
std::string&, const std::string&, const std::string&, int, int,
unsigned int, unsigned int, unsigned int, bool, const std::string&,
uid_t, gid_t)’:
ext/nginx/HelperServer.cpp:720: warning: converting negative value
‘-0x00000000000000001’ to ‘uid_t’
ext/nginx/HelperServer.cpp:720: warning: converting negative value
‘-0x00000000000000001’ to ‘gid_t’

-- Justin

[Justin Ramos]

I figured it out! This little line in nginx.conf was causing all the
trouble:

passenger_user_switching off; # tricky stuff where we inherit PID
from environment.rb (or get nobody); nasty

I guess someone didn't like user_switching and turned it off in our
config. Odd that it worked fine in 2.2.5 with this config, but I guess
it doesn't matter now.

-- Justin

[Ash]

It's nice that turning on user switching solved your problem but what
if you don't want the running user to be the owner of the
environment.rb file? I found this problem myself and posted a fix for
it, which basically sets the owner of the guard file to the running
user rather than root. However, it was never picked up and put into
the code.

http://code.google.com/p/phusion-passenger/issues/detail?id=458&colspec=ID%20Type%20Status%20Priority%20Milestone%20Stars%20Summary

[Hongli Lai]

I've given the issue a higher priority, especially now that the cause
is known. Thanks for finding this out!

--
Phusion | The Computer Science Company

Web: http://www.phusion.nl/
E-mail: in...@phusion.nl