Passenger noob... can't get it working under Apache. Any help much appreciated!

[Nick Weavers]

So I have a production web server running Apache for a Joomla CMS website with and extensive bespoke Joomla extension and there is a section in the apache.config (below) to set that up for port 80 which runs just fine.

<VirtualHost *:80>

        ServerName mydomain

        DocumentRoot /var/www/vhosts/mydomain/httpdocs

        <Directory /var/www/vhosts/mydomain/httpdocs>

                Options Indexes FollowSymLinks MultiViews

                AllowOverride All

        </Directory>

        CustomLog /var/log/apache2/mydomain-access.log combined

        ErrorLog /var/log/apache2/mydomain-error.log

        LogLevel warn

</VirtualHost>

I want to start developing some back-office functionality for this website in ruby/rails so I cloned the production machine to a test server (RackSpace VM) and set up ruby using ruby-install and chruby, and installed Passenger using the instructions described in "Installing Passenger + Apache on a Linux/Unix production server for Ruby apps + Ubuntu 14.04 LTS " (https://www.phusionpassenger.com/library/walkthroughs/deploy/ruby/ownserver/apache/oss/trusty/install_passenger.html).

Since the clone needs to be addressed by ip address the original is updated to replace mydomain with myipaddress

<VirtualHost *:80>

        ServerName myipaddress

        DocumentRoot /var/www/vhosts/mydomain/httpdocs

        <Directory /var/www/vhosts/mydomain/httpdocs>

                Options Indexes FollowSymLinks MultiViews

                AllowOverride All

        </Directory>

        CustomLog /var/log/apache2/mydomain-access.log combined

        ErrorLog /var/log/apache2/mydomain-error.log

        LogLevel warn

</VirtualHost> 

And I added a new section to configure the rails app as follows:

<VirtualHost *:3000>

        ServerName myipaddress

        DocumentRoot /var/www/vhosts/mydomain.co.uk/rails/hello-world

        <Directory /var/www/vhosts/mydomain.co.uk/rails/hello-world>

                Options Indexes FollowSymLinks MultiViews

                AllowOverride All

        </Directory>

        PassengerEnabled on

        PassengerRuby /opt/rubies/ruby-2.2.3/bin/ruby

        PassengerLogFile /var/log/apache2/passenger.log

</VirtualHost>

I then created a new rails app by running "rails new hello-world" under /var/www/vhosts/mydomain/rails as the user owning the app

After this I ran passenger-config --detect-apache2 to check it looked okay:

$passenger-config --detect-apache2

Looking for possible Apache installations...

 --> Looking for /opt/rubies/ruby-2.2.3/bin/apxs2: not found

 --> Looking for /usr/local/sbin/apxs2: not found

 --> Looking for /usr/local/bin/apxs2: not found

 --> Looking for /usr/sbin/apxs2: not found

 --> Looking for /usr/bin/apxs2: found

 --> Looking for /sbin/apxs2: not found

 --> Looking for /bin/apxs2: not found

 --> Looking for /opt/ohai-solo/bin/apxs2: not found

 --> Looking for /opt/rubies/ruby-2.2.3/bin/apxs: not found

 --> Looking for /usr/local/sbin/apxs: not found

 --> Looking for /usr/local/bin/apxs: not found

 --> Looking for /usr/sbin/apxs: not found

 --> Looking for /usr/bin/apxs: found

 --> Looking for /sbin/apxs: not found

 --> Looking for /bin/apxs: not found

 --> Looking for /opt/ohai-solo/bin/apxs: not found

1 symlink duplicate(s) detected; ignoring them.

Analyzing /usr/bin/apxs...

Detecting main Apache executable...

 --> Looking for /usr/bin/apache2: not found

 --> Looking for /usr/sbin/apache2: found

Detecting version...

 --> 2.4.7

Detecting control command...

 --> Looking for /usr/bin/apache2ctl: not found

 --> Looking for /usr/bin/apachectl2: not found

 --> Looking for /usr/bin/apachectl: not found

 --> Looking for /usr/sbin/apache2ctl: found

Detecting configuration file location...

 --> Looking for /usr/bin/apache2ctl: not found

 --> Looking for /usr/bin/apachectl2: not found

 --> Looking for /usr/bin/apachectl: not found

 --> Looking for /usr/sbin/apache2ctl: found

 --> Looking for /usr/bin/apache2ctl: not found

 --> Looking for /usr/bin/apachectl2: not found

 --> Looking for /usr/bin/apachectl: not found

 --> Looking for /usr/sbin/apache2ctl: found

 --> /etc/apache2/apache2.conf

Detecting error log file...

 --> Looking for /usr/bin/apache2ctl: not found

 --> Looking for /usr/bin/apachectl2: not found

 --> Looking for /usr/bin/apachectl: not found

 --> Looking for /usr/sbin/apache2ctl: found

 --> Looking for /usr/bin/apache2ctl: not found

 --> Looking for /usr/bin/apachectl2: not found

 --> Looking for /usr/bin/apachectl: not found

 --> Looking for /usr/sbin/apache2ctl: found

 --> Attempting to substitute environment variables in Apache config value "${APACHE_LOG_DIR}/error.log"... 

 --> Substituted "${APACHE_LOG_DIR}" -> "/var/log/apache2"

 --> /var/log/apache2/error.log

Detecting a2enmod and a2dismod...

 --> Looking for /usr/bin/a2enmod: not found

 --> Looking for /usr/sbin/a2enmod: found

 --> Looking for /usr/bin/a2dismod: not found

 --> Looking for /usr/sbin/a2dismod: found

 --> Looking for /usr/bin/apache2ctl: not found

 --> Looking for /usr/bin/apachectl2: not found

 --> Looking for /usr/bin/apachectl: not found

 --> Looking for /usr/sbin/apache2ctl: found

Found a usable Apache installation using /usr/bin/apxs.

Final autodetection results

 * Found Apache 2.4.7!

   Information:

      apxs2          : /usr/bin/apxs

      Main executable: /usr/sbin/apache2

      Control command: /usr/sbin/apache2ctl

      Config file    : /etc/apache2/apache2.conf

      Error log file : /var/log/apache2/error.log

   To install Phusion Passenger against this specific Apache version:

      /opt/rubies/ruby-2.2.3/bin/ruby /usr/bin/passenger-install-apache2-module --apxs2-path='/usr/bin/apxs'

   To start, stop or restart this specific Apache version:

      /usr/sbin/apache2ctl start

      /usr/sbin/apache2ctl stop

      /usr/sbin/apache2ctl restart

   To troubleshoot, please read the logs in this file:

      /var/log/apache2/error.log

$

I was a bit puzzled by the message:

To install Phusion Passenger against this specific Apache version:

      /opt/rubies/ruby-2.2.3/bin/ruby /usr/bin/passenger-install-apache2-module --apxs2-path='/usr/bin/apxs'

So I ran the specified command for good measure:

$sudo  /opt/rubies/ruby-2.2.3/bin/ruby /usr/bin/passenger-install-apache2-module --apxs2-path='/usr/bin/apxs'

The Phusion Passenger Apache module is correctly installed :-)

Press Enter to learn how to deploy a web app on Apache.

--------------------------------------------

Deploying a web application

To learn how to deploy a web app on Passenger, please follow the deployment

guide:

  https://www.phusionpassenger.com/library/deploy/apache/deploy/

Enjoy Phusion Passenger, a product of Phusion (www.phusion.nl) :-)

https://www.phusionpassenger.com

Phusion Passenger is a registered trademark of Hongli Lai & Ninh Bui.

$

I also ran passenger-config validate-install passenger:

$sudo passenger-config validate-install

What would you like to validate?

Use <space> to select.

If the menu doesn't display correctly, press '!'

 ‣ ⬢  Passenger itself

   ⬡  Apache

-------------------------------------------------------------------------

 * Checking whether this Passenger install is in PATH... ✓

 * Checking whether there are no other Passenger installations... ✓

Everything looks good. :-)

$

And for Apache:

$sudo passenger-config validate-install

What would you like to validate?

Use <space> to select.

If the menu doesn't display correctly, press '!'

   ⬢  Passenger itself

 ‣ ⬡  Apache

-------------------------------------------------------------------------

 * Checking whether this Passenger install is in PATH... ✓

 * Checking whether there are no other Passenger installations... ✓

Everything looks good. :-)

$

However,  when I try to access my server from Chrome on my PC via the URL <my-ip-address>:3000 I see a page with:

This web page is not available

ERR_CONNECTION_REFUSED

I wondered if it could be the firewall so I also did a "sudo ufw insert 1 allow 3000" but still no joy.

Can anyone help?

[Nick Weavers]

I don't see anything listening on port 3000 despite having configured it for apache:

>sudo netstat -tulpn | grep LISTEN

tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      1226/sshd       

tcp        0      0 0.0.0.0:25              0.0.0.0:*               LISTEN      1568/master     

tcp        0      0 127.0.0.1:6010          0.0.0.0:*               LISTEN      2205/sshd: nhw@nott

tcp        0      0 127.0.0.1:9306          0.0.0.0:*               LISTEN      1610/searchd    

tcp        0      0 127.0.0.1:6011          0.0.0.0:*               LISTEN      19545/sshd: ddfs@no

tcp        0      0 0.0.0.0:9312            0.0.0.0:*               LISTEN      1610/searchd    

tcp        0      0 0.0.0.0:3306            0.0.0.0:*               LISTEN      1344/mysqld     

tcp6       0      0 :::22                   :::*                    LISTEN      1226/sshd       

tcp6       0      0 :::25                   :::*                    LISTEN      1568/master     

tcp6       0      0 ::1:6010                :::*                    LISTEN      2205/sshd: nhw@nott

tcp6       0      0 :::443                  :::*                    LISTEN      17186/apache2   

tcp6       0      0 ::1:6011                :::*                    LISTEN      19545/sshd: ddfs@no

tcp6       0      0 :::80                   :::*                    LISTEN      17186/apache2   

[Nick Weavers]

Adding a "listen 3000" to my apache2 config means I now see Apache2 show me a directory listing for my Rails app when I visit my_ip_address:3000 in my browser:

Index of /

[ICO] Name Last modified Size Description

[   ] Gemfile 2015-11-16 14:16 1.5K  

[   ] Gemfile.lock 2015-11-16 14:17 3.7K  

[   ] README.rdoc 2015-11-16 14:16 478  

[   ] Rakefile 2015-11-16 14:16 249  

[DIR] app/ 2015-11-16 14:16 -  

[DIR] bin/ 2015-11-16 14:17 -  

[   ] config.ru 2015-11-16 14:16 153  

[DIR] config/ 2015-11-16 14:16 -  

[DIR] db/ 2015-11-16 14:16 -  

[DIR] lib/ 2015-11-16 14:16 -  

[DIR] log/ 2015-11-17 09:04 -  

[DIR] public/ 2015-11-16 14:16 -  

[DIR] test/ 2015-11-16 14:16 -  

[DIR] tmp/ 2015-11-16 14:16 -  

[DIR] vendor/ 2015-11-16 14:16 -  

Apache/2.4.7 (Ubuntu) Server at my_ip_address Port 3000

In my passenger log I don't see anything indicating that it is doing anything when the http request is made:

>tail passenger.log

[ 2015-11-17 09:07:56.2645 3280/7ff0061b9700 Ser/Server.h:444 ]: [ServerThr.2] Shutdown finished

[ 2015-11-17 09:07:56.2645 3280/7ff0069ba700 Ser/Server.h:759 ]: [ServerThr.1] Freed 128 spare client objects

[ 2015-11-17 09:07:56.2646 3280/7ff0069ba700 Ser/Server.h:444 ]: [ServerThr.1] Shutdown finished

[ 2015-11-17 09:07:56.2672 3280/7ff00d18b7c0 age/Cor/CoreMain.cpp:942 ]: Passenger core shutdown finished

[ 2015-11-17 09:07:56.3347 3311/7fd07eb387c0 age/Wat/WatchdogMain.cpp:1276 ]: Starting Passenger watchdog...

[ 2015-11-17 09:07:56.3573 3314/7f5f753117c0 age/Cor/CoreMain.cpp:957 ]: Starting Passenger core...

[ 2015-11-17 09:07:56.3578 3314/7f5f753117c0 age/Cor/CoreMain.cpp:234 ]: Passenger core running in multi-application mode.

[ 2015-11-17 09:07:56.3619 3314/7f5f753117c0 age/Cor/CoreMain.cpp:707 ]: Passenger core online, PID 3314

[ 2015-11-17 09:07:56.3833 3326/7fb413e647c0 age/Ust/UstRouterMain.cpp:504 ]: Starting Passenger UstRouter...

[ 2015-11-17 09:07:56.3854 3326/7fb413e647c0 age/Ust/UstRouterMain.cpp:317 ]: Passenger UstRouter online, PID 3326

 

[Nick Weavers]

Updated my apache2 config for the rails app so that DocumentRoot and Directory refer to the path to the rails app's public folder:

<VirtualHost *:3000>

        ServerName myipaddress

        DocumentRoot /var/www/vhosts/mydomain.co.uk/rails/hello-world/public

        <Directory /var/www/vhosts/mydomain.co.uk/rails/hello-world/public>

                Options Indexes FollowSymLinks MultiViews

                AllowOverride All

        </Directory>

        PassengerEnabled on

        PassengerRuby /opt/rubies/ruby-2.2.3/bin/ruby

        PassengerLogFile /var/log/apache2/passenger.log

</VirtualHost>

Now the browser shows me a page with:

<h2>Incomplete response received from application</h2>

And the passenger log says:

>tail passenger.log

App 5214 stderr:        from /opt/rubies/ruby-2.2.3/lib/ruby/gems/2.2.0/gems/railties-4.2.5/lib/rails/application.rb:534:in `validate_secret_key_config!'

App 5214 stderr:        from /opt/rubies/ruby-2.2.3/lib/ruby/gems/2.2.0/gems/railties-4.2.5/lib/rails/application.rb:246:in `env_config'

App 5214 stderr:        from /opt/rubies/ruby-2.2.3/lib/ruby/gems/2.2.0/gems/railties-4.2.5/lib/rails/engine.rb:514:in `call'

App 5214 stderr:        from /opt/rubies/ruby-2.2.3/lib/ruby/gems/2.2.0/gems/railties-4.2.5/lib/rails/application.rb:165:in `call'

App 5214 stderr:        from /usr/lib/ruby/vendor_ruby/phusion_passenger/rack/thread_handler_extension.rb:97:in `process_request'

App 5214 stderr:        from /usr/lib/ruby/vendor_ruby/phusion_passenger/request_handler/thread_handler.rb:152:in `accept_and_process_next_request'

App 5214 stderr:        from /usr/lib/ruby/vendor_ruby/phusion_passenger/request_handler/thread_handler.rb:113:in `main_loop'

App 5214 stderr:        from /usr/lib/ruby/vendor_ruby/phusion_passenger/request_handler.rb:416:in `block (3 levels) in start_threads'

App 5214 stderr:        from /usr/lib/ruby/vendor_ruby/phusion_passenger/utils.rb:113:in `block in create_thread_and_abort_on_exception'

[ 2015-11-17 09:34:36.9228 5180/7fb3eaa2e700 age/Cor/Req/Utils.cpp:96 ]: [Client 1-1] Sending 502 response: application did not send a complete response

[Hongli Lai]

You should scroll up in the log file. The backtrace there tells you
exactly why the application is misbehaving. I'm only seeing a portion
of the backtrace, but from what I see so far it indicates that you are
missing some configuration option in the app.

> --
> You received this message because you are subscribed to the Google Groups
> "Phusion Passenger Discussions" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to phusion-passen...@googlegroups.com.
> To post to this group, send email to phusion-...@googlegroups.com.
> Visit this group at http://groups.google.com/group/phusion-passenger.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/phusion-passenger/00b3d582-230c-4fef-99e9-969c1d23cd49%40googlegroups.com.
>
> For more options, visit https://groups.google.com/d/optout.



--
Phusion B.V. | Web Application deployment, scaling, and monitoring solutions

Web: http://www.phusion.nl/
E-mail: in...@phusion.nl
Chamber of commerce no: 63501007 (The Netherlands)

[Nick Weavers]

Thanks.

Now I cannot seem to get my GEM_PATH set correctly.

I'm using chruby so I set up a wrapper  script in /usr/local/

>ls -la /usr/local

drwxr-xr-x 10 root root 4096 Nov 18 01:17 .

drwxr-xr-x 10 root root 4096 Apr 17  2014 ..

drwxr-xr-x  2 root root 4096 Nov 17 09:52 bin

drwxr-xr-x  2 root root 4096 Apr 17  2014 etc

drwxr-xr-x  2 root root 4096 Apr 17  2014 games

drwxr-xr-x  2 root root 4096 Apr 17  2014 include

drwxr-xr-x  5 root root 4096 Nov 15 14:26 lib

lrwxrwxrwx  1 root root    9 Apr 17  2014 man -> share/man

-rwxr-xr-x  1 root root  122 Nov 18 01:17 my_ruby_wrapper_script

drwxr-xr-x  2 root root 4096 Apr 17  2014 sbin

drwxr-xr-x 10 root root 4096 Nov 13 18:44 share

drwxr-xr-x  4 root root 4096 Nov 13 21:28 src

Containing:

>cat /usr/local/my_ruby_wrapper_script

#!/bin/bash

source /usr/local/share/chruby/chruby.sh

source /usr/local/share/chruby/auto.sh

chruby 2.2.3

exec "ruby" "$@"

And in my apache conf:

<VirtualHost *:3000>

...

PassengerRuby /usr/local/my_ruby_wrapper_script

...

</VirtualHost>

If I start an ssh session for my rails app user (ddfs) and issue the env command I see:

$ env

...

GEM_HOME=/home/ddfs/.gem/ruby/2.2.3

...

PATH=/home/ddfs/.gem/ruby/2.2.3/bin:/opt/rubies/ruby-2.2.3/lib/ruby/gems/2.2.0/bin:/opt/rubies/ruby-2.2.3/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games

...

RUBY_ENGINE=ruby

...

HOME=/home/ddfs

...

GEM_ROOT=/opt/rubies/ruby-2.2.3/lib/ruby/gems/2.2.0

...

GEM_PATH=/home/ddfs/.gem/ruby/2.2.3:/opt/rubies/ruby-2.2.3/lib/ruby/gems/2.2.0

...

RUBY_ROOT=/opt/rubies/ruby-2.2.3

...

RUBYOPT=

RUBY_VERSION=2.2.3

...

$

I have set "PassengerFriendlyErrorPages on" in apache conf.

When I view my app in a browser I see:

Could not find minitest-5.8.3 in any of the sources

...

APACHE_RUN_USER = ddfs

USER = ddfs 

But the GEM_PATH below don't match what was shown by env

GEM_PATH = /opt/rubies/ruby-2.2.3/lib/ruby/gems/2.2.0:/home/ddfs/.gem/ruby/2.2.0

It has the paths in the wrong sequence, and the user's gem path is incorrect. It shows as /home/ddfs/.gem/ruby/2.2.0 when the env shows it as the correct value of /home/ddfs/.gem/ruby/2.2.3

The reason the minitest gem isn't found is because it's in /home/ddfs/.gem/ruby/2.2.3

How could this be happening?

[Tinco Andringa]

Hi Nick,

You'll need to set the GEM_PATH in your wrapper script as well. Passenger does not execute with the same env as your ssh bash session.

Kind regards,

Tinco

But the GEM_PATH below don't match what was shown by env

GEM_PATH = /opt/rubies/ruby-2.2.3/lib/ruby/gems/2.2.0:/home/ddfs/.gem/ruby/2.2.0

It has the paths in the wrong sequence, and the user's gem path is incorrect. It shows as /home/ddfs/.gem/ruby/2.2.0 when the env shows it as the correct value of /home/ddfs/.gem/ruby/2.2.3

The reason the minitest gem isn't found is because it's in /home/ddfs/.gem/ruby/2.2.3

How could this be happening?

--
You received this message because you are subscribed to the Google Groups "Phusion Passenger Discussions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to phusion-passen...@googlegroups.com.
To post to this group, send email to phusion-...@googlegroups.com.
Visit this group at http://groups.google.com/group/phusion-passenger.

To view this discussion on the web visit https://groups.google.com/d/msgid/phusion-passenger/a3f6f237-5b85-4729-9bbe-acf3542c2be2%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Nick Weavers]

Thanks Tinco,

I tried doing as you suggested as follows:

>cat /usr/local/my_ruby_wrapper_script

#!/bin/bash

source /usr/local/share/chruby/chruby.sh

source /usr/local/share/chruby/auto.sh

chruby 2.2.3

export GEM_PATH=/home/ddfs/.gem/ruby/2.2.3:/opt/rubies/ruby-2.2.3/lib/ruby/gems/2.2.0

exec "ruby" "$@"

but strangely the export in my wrapper script isn't updating the GEM_PATH as shown in the PassengerFriendlyErrorPages web page. That page remains unchanged and still shows

GEM_PATH = /opt/rubies/ruby-2.2.3/lib/ruby/gems/2.2.0:/home/ddfs/.gem

However, if I updated my apache config file to add this:

SetEnv GEM_PATH /home/ddfs/.gem:/opt/rubies/ruby-2.2.3/lib/ruby/gems/2.2.0

Then in the Environment variables section of the passenger error page error page I see this:

GEM_PATH = /opt/rubies/ruby-2.2.3/lib/ruby/gems/2.2.0:/home/ddfs/.gem
_ORIGINAL_GEM_PATH = /home/ddfs/.gem:/opt/rubies/ruby-2.2.3/lib/ruby/gems/2.2.0
GEM_HOME = /opt/rubies/ruby-2.2.3/lib/ruby/gems/2.2.0

Why would passenger do that (change my env variable to _ORIGINAL_GEM_PATH) , and not honour the environment variable setting I gave?

[Nick Weavers]

I have also tried using 

PassengerLoadShellEnvvars=on

in my apache config and setting a test dummy variable in .my .bashrc file:

export ZZZ=1

but this doesn't show up the the Environment variables section of the passenger error page error page so is this feature actually working?

[Hongli Lai]

PassengerLoadShellEnvvars works fine. If an `export` statement in your
bashrc doesn't work, then that's likely because of something in your
bashrc. Passenger executes the app through `bash -l -c`. This means
that bash will try to load .profile or .bash_profile, which may or may
not load .bashrc. On some systems, .bashrc may also contain a
statement to not evaluate the rest of the file if there is no TTY
(which there isn't when running Passenger). You should double check
all of those things.

If this sounds too complicated to you, then you should use the SetEnv
directive in Apache.

[Nick Weavers]

That was it. Moved the export to .profile and it worked. 

Thanks Hongli Lai