Umlaut and Passenger: Internal Server Error

56 views
Skip to first unread message

Lauren Magnuson

unread,
Sep 20, 2015, 7:41:04 PM9/20/15
to Umlaut
Hello,

I'm working on setting up Umlaut on my commercial host (DreamHost) and I've run into some problems with Passenger configuration.  Because I'm using a VPS through DreamHost, I don't have access to all of the settings that my host uses, but I'm hoping my description of the problem might provide a clue to the problem.

Essentially, the app is set up, configured, and runs perfectly when I start up Passenger manually using the following command:

passenger start -d

Everything works great on port 3000 (i.e., mywebsite.com:3000).  However, I want the app to work at my root domain on port 80 (i.e., mywebsite.com).

If I stop passenger and try to start up the app on port 80 using this command:

passenger start -p 80 -d

I get this error:

nginx: [emerg] bind() to 0.0.0.0:80 failed (13: Permission denied)


I've contacted my host and they've told me that essentially if the app is configured correctly and in the correct directory (which it is) the app should just start up automatically without me automatically starting Passenger (in other words, I should be able to, without starting passenger, just go to mywebsite.com and it will 'just work').  This is apparently how DreamHost works with Passenger:

--------------------

The following are the basic actions that take place once a file is requested from a domain running Passenger and Ruby on Rails:

  1. When a request is made to a domain/subdomain, the Apache HTTP Server passes the request to Passenger.
  2. Passenger first looks for an appropriately-named HTML or CGI file in the domain/subdomain's /public subdirectory.
  3. If no matching file is found, the request is passed to Passenger's Rack interface.
  4. Note that this use of the /public subdirectory meshes precisely with the way that Ruby on Rails makes use of the same subdirectory.
  5. In order to generate a response, Rack looks for a file named "config.ru" in the domain/subdomain's root directory (i.e., the parent of the domain's /public subdirectory).
  6. Rack requires that you place the appropriate Ruby code into "config.ru" to invoke your desired web framework or application to handle the request.
    (See Rack for more information about how information is passed through the Rack interface.)

Under normal circumstances, Ruby on Rails (RoR) will automatically create and initialize all of the files and directories needed to interface with Passenger/Rack. When running a RoR application, the only Rack-related files you are likely to modify are possibly adding GEM_PATH information to "config.ru" and "touching" the "tmp/restart.txt" file.

--------------------------

However, if I attempt to go to my root domain I get a 500 Internal Server Error, and this is the extremely unhelpful message in my error.log:

Premature end of script headers: internal_error.html


I asked if maybe this had something to do with the fact that I'm running the app in development rather than production, but DreamHost tells me that rails apps should run OK in production or development.  Is there something about Umlaut config that requires the use of port 3000, or some setting I am missing somewhere?


Any thoughts or things I could try next to diagnose this problem?  Obviously I can keep just running Passenger manually on 3000 during development, but eventually in production I'd love the app to run on port 80 for a variety of reasons.  I'm just a lowly developer (and a n00b to rails at that) and my server admin skillz are apparently not up to snuff, so I really appreciate any pointers from anyone who's successfully set up Umlaut with Passenger.


Thanks in advance!


Lauren



Jonathan Rochkind

unread,
Sep 21, 2015, 8:40:30 AM9/21/15
to umlaut-...@googlegroups.com

Hm, this is mysterious. 


I, and many others, have succesfully set up an Umlaut app with Passenger. We don't run into the problems you are running into, which seem more about Dreamhost's specific set up. If we wanted it to run on port 80, we'd just do the thing you tried first, which doesn't work on dreamhost (or, myself, I actually use passenger-apache instead). 


Can you try making a bare bones hello world Rails app, and see if that runs properly on port 80 according to Dreamhost instructions? That is, confirm if this problem occurs with any Rails app at all, and has nothing to do with Umlaut, or is really specific to something involving Umlaut. 


I have not heard good things about running Rails apps on Dreamhost, but maybe they've gotten better since I last heard. 


I googled for "dreamhost rails Premature end of script headers: internal_error.html​" -- and got a bunch of hits about getting this error using Passenger on Dreamhost... but with Django (Python), rather than Rails.  But it does seem like you get this error on Dreamhost for some odd reasons, that are hard to diagnose. It may be that Dreamhost is still not great for app deployment like this. 


I'm happy to try to keep helping though, if I can. I am also always interested to learn about new people experimenting with Umlaut (I'm the Umlaut lead developer), do you want to share what institution you are from and what you are considering Umlaut for? 


Jonathan



From: umlaut-...@googlegroups.com <umlaut-...@googlegroups.com> on behalf of Lauren Magnuson <lauren.l...@gmail.com>
Sent: Sunday, September 20, 2015 7:41 PM
To: Umlaut
Subject: [umlaut] Umlaut and Passenger: Internal Server Error
 
--
You received this message because you are subscribed to the Google Groups "Umlaut" group.
To unsubscribe from this group and stop receiving emails from it, send an email to umlaut-softwa...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Jonathan Rochkind

unread,
Sep 21, 2015, 8:42:41 AM9/21/15
to umlaut-...@googlegroups.com

​Ah, I also see the error.log you are checking is for the web server. 


Try checking the actual Rails app error log for more information. You should find it inside your app directory, at logs/production.log (since you are trying to start up in production mode).  


Can you find that log, and see if there are more informative error messages in there? 



From: umlaut-...@googlegroups.com <umlaut-...@googlegroups.com> on behalf of Jonathan Rochkind <roch...@jhu.edu>
Sent: Monday, September 21, 2015 8:40 AM
To: umlaut-...@googlegroups.com
Subject: Re: [umlaut] Umlaut and Passenger: Internal Server Error
 

Lauren Magnuson

unread,
Sep 21, 2015, 11:55:34 AM9/21/15
to Umlaut
Hi Jonathan,

Thank you so much for the advice - you're totally right, it's a problem with DreamHost (but at least I'm reassured that it's not something that I screwed up!).  I took your suggestion to install a generic rails app and the same error continues (the same 500 Internal Server error).  So this is not an Umlaut problem (yay).  Looking at the Rails logs, nothing is actually *being* logged when I attempt to access the domain that should be loading the Rails app - when I run Passenger manually on port 3000, log entries appear, but just trying to load the app in the browser without manually starting passenger doesn't log anything (so, basically, Rails doesn't even know it's trying to be loaded).  I'll continue working with DreamHost support about this and will post back to this group if we come up with a solution, in case anyone else encounters this issue.

I work part-time for the PALNI consortium (Private Academic Library Network of Indiana) and we're setting up Umlaut to try to use with our OCLC WorldShare Management System / WorldCat Discovery system.  Essentially we'd like to set up logic in Umlaut to serve as a single button that would allow for the patron to place holds on items that are available, to place resource sharing requests for items that are available in the consortium, and to place traditional ILL requests for items that are not available in the consortium.  So when a user clicks the Umlaut "get it" link, the following logic takes place:

-if the item is held by the local institution, either show options to place hold and/or show stack map (map of where the item is shelved in the library)
-if the item is not held by local institution, but is held by a consortium library (found by using the OCLC WMS Collection Management / Availability API), show options to place a network resource sharing request
-if the item is not held by the local institution or a consortium library, show options to place a tradition ILL request

We've been calling it the "One Button" approach.  Right now, in WorldCat Discovery, for a single resource record, the user can be presented with up to 24 different links to 'get' an item (check availability links for 23 consortium libraries (including their own institution) + a traditional ILL link).  This seems overwhelming to the user, who probably doesn't care where something comes from - they just want the thing.  So we're trying to handle some of the logic for the user so they're automatically presented with the best option.

So Umlaut is amazing and we're really excited to get it going, and definitely plan on sharing out any progress made with anyone who is interested.  I am sure I will have more questions for you as things go along - as I mentioned, I'm a rails noob but luckily the Umlaut documentation so far has been really helpful, even for a noob like me. :)

Thanks again!

Lauren Magnuson
PALNI Development Coordinator

Jonathan Rochkind

unread,
Sep 21, 2015, 12:13:19 PM9/21/15
to umlaut-...@googlegroups.com
Hi Laura,

So _I'm_ relieved you could duplicate the error with a plain vanilla 'hello world' Rails app, because it means it's not anything Umlaut specific. But that's little consolation for you. :(

You could try troubleshooting and asking for support on that basic Hello World application. In my own googling, it looks like if the Rails app can't find the database at all, that is something that causes it to die without being able to write to the error logs, at least on Dreamhost (I feel like on other platforms I've still seen that error in logs); so that'd be one thing to look into maybe.

But when I google around for Rails and Dreamhost, I'm afraid I find un-ending tails of woe. I think you will thank yourself later if you can choose a different deployment platform. I've heard good things about Digital Ocean, although I've never used it myself. We deploy on local on-campus hardware here, and I think that's probably true of all existing Umlaut installations. But I don't know of any reason a 'cloud' host wouldn't work with Umlaut, if you choose a good non-broken one.

Also, simply for prototyping or basic proof of concept, there's no reason you can't simply use an OSX workstation on your desk, and get to figuring out deployment options later. But I understand wanting to have proof that your deployment scenario will work right at the front. I think Dreamhost is going to give you challenges.

Thanks for sharing your plans; that's exciting to hear about. That does sound like a very appropriate use case for Umlaut. You will have to write some custom code in the form of Umlaut plug-in adapters to make it so.  Please feel free to ask for advice or help on that when you get to that point -- and please consider sharing your code with others, if you get to that point, it sounds like other WorldCat Discovery users might be interested in the solution too.

And I'm glad you've been finding the Umlaut documentation useful; I think good docs are important and have put a lot of time into trying to make and keep the Umlaut docs good.

Jonathan

Ross Singer

unread,
Sep 21, 2015, 11:41:01 PM9/21/15
to umlaut-...@googlegroups.com
I agree the problem here is most likely on the DreamHost side, but I think that this should actually work, if you can get the app/Passenger all configured properly.

I'm a little confused about the fact that you don't have access to all your settings: I thought that was the whole purpose of the VPS?  I say this because I have gotten Rack and Rails running on DH *shared* hosting (the Code4Lib diebold-o-tron ran on it for years, until it moved to Oregon State last year) which is far more restrictive than what you can do on a VPS (in fact, the solution to most shared hosting/Ruby problems is "upgrade to a VPS").

Looking back through the steps you've taken to get it started, it works fine on port 3000?  Do you have sudo access?  What happens when you run:

sudo passenger start -p 80 -d

?

-Ross.

Lauren Magnuson

unread,
Sep 22, 2015, 1:22:12 AM9/22/15
to umlaut-...@googlegroups.com
Hi Ross,

If I try to start passenger on port 80 with sudo, this is what I get:

nginx: [emerg] bind() to 0.0.0.0:80 failed (98: Address already in use)

I think when I'm manually starting up passenger the server is actually using nginx to run on port 3000, so I'm avoiding the mess that is apparently the Apache settings on my VPS.  But port 80 is taken up by Apache.  I could kill apache processes with my sudoer, but I have some other PHP apps on my VPS that I don't want to totally nuke at the moment.  Plus I'm not sure that I'd be killing the *real apache* (more on that below...)

I made a bit of progress today working with DH support, and they were able to get a generic rails app going by setting the secret token for production.  I had a hunch that the problem was that I wasn't running the app in production (though I was told that running apps in development is totally fine), and I'm thinking that's pretty much confirmed.  So apparently if you want to run rails apps on DreamHost, you have to run them in production mode.  At one point I tried running Umlaut in production and setting the secret token but I think I messed something up in one of the config files when I attempted to set it up (at that point, the sum total of my knowledge about switching a rails app from development to production mode was about 25 minutes of Googleing, so that's assuredly user error).

re: not having access to all the DH settings:  There's definitely some weirdness going on with my VPS.  I can actually edit/create/save Apache configuration files on the server, but those files are apparently somehow not *real* because editing them and restarting Apache has no effect.  I think it has something to do with the way DreamHost manages all of its subdomains by user - each subdomain, theoretically, seems to have its own Apache settings (at least, you can edit some of those settings in the DreamHost management panel when you provision a subdomain on your VPS).  So the typical location for Apache (/etc/apache2) isn't *real* because there are, apparently somewhere else, Apache settings that manage subdomains spawned by the panel.  At least that's how I'm explaining it to myself.  When I asked where the Apache settings were for my VPS, I was told I don't have access to edit them by DH support.

It's a bummer because I actually think DreamHost has excellent service otherwise if you're running PHP apps!  But perhaps Jonathan is right and another host that does more stuff with Rails is the way to go.  I've also heard good things about Digital Ocean, but I may just go the route for now of local dev until I get the hosting situation worked out.  

Lauren


You received this message because you are subscribed to a topic in the Google Groups "Umlaut" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/umlaut-software/gRwrG8eyC28/unsubscribe.
To unsubscribe from this group and all its topics, send an email to umlaut-softwa...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.



--
Systems & Emerging Technologies Librarian, CSUN
Development Coordinator, PALNI

Ross Singer

unread,
Sep 22, 2015, 8:07:58 AM9/22/15
to umlaut-...@googlegroups.com
Well, that's sad to hear, because I think it would have been a good avenue for people to "try before they buy (into)" Umlaut: cost/maintenance-wise, it's easier to sidestep the vagaries of local IT policies and cheap enough that it doesn't require a purchase order.

Thanks for giving it a go, though! 

-Ross.

Jason Ronallo

unread,
Sep 22, 2015, 8:39:47 AM9/22/15
to umlaut-...@googlegroups.com
Lauren,

So you have Apache running on port 80 and you're trying to have nginx listen on port 80 too? I don't think that can work. Could you have nginx listen on a different port and either 1) serve Umlaut from that port or 2) have Umlaut run on that port and then use Apache to proxy nginx/Umlaut to a sub-URI (sub-directory)?

But maybe the problem is deeper?

Jason

Jonathan Rochkind

unread,
Sep 22, 2015, 9:45:54 AM9/22/15
to umlaut-...@googlegroups.com

Googling, it looks like ​Dreamhost VPS starts at $15/month for 1GB RAM, 30GB storage, unlimited bandwidth. 


(Dreamhost shared hosting is of course cheaper, but googling Rails on Dreamhost shared gets even MORE tales of woe). 


DigitalOcean actually starts at $5/month, but I think that server is too small to run a Rails app. 


For $10/month on DO, you can get a 1GB RAM, 30GB storage server, same as Dreamhost. It is bandwidth limited to 2GB of transfer, and you pay for extra bandwidth, but for simple "try before you buy" that's probably fine -- and I think that the overage pricing is $0.02 per GB, so that extra $5 would get you 250GB of transfer. 


I think you probably CAN get a Rails app running on a Dreamhost VPS. But from Googling people trying and explaining some of the weird workarounds, I think it's probably not worth it, just use DO, which gets very good reviews for Rails use. 


From Lauren's description is sounds like a Dreamhost VPS won't let you run what you choose on port 80, but has some built-in infrastructure meant to run Passenger (or other rack servers?) for you, using your config.ru, as configured in a control panel. It sounds like you don't have full access to configure what you want in the ordinary ways. It sounds like people definitely have made it work, and Dreamhost claims to support Rails -- but it sounds like it's a pain and extra headache. 



From: umlaut-...@googlegroups.com <umlaut-...@googlegroups.com> on behalf of Ross Singer <rossf...@gmail.com>
Sent: Tuesday, September 22, 2015 8:07 AM

To: umlaut-...@googlegroups.com
Subject: Re: [umlaut] Umlaut and Passenger: Internal Server Error

Lauren Magnuson

unread,
Sep 22, 2015, 12:06:16 PM9/22/15
to umlaut-...@googlegroups.com
Hi Jason, 

I definitely could have the app run on nginx on another port (like port 3000, or pretty much any other port) but I was hoping to be able to be able to run it on port 80 so that it loads on the root domain easily (i.e., mywebsite.com).  Some of the libraries that will be using the app have campus wifi networks set up to block any traffic from non-standard ports (80 or 443) so down the line I'd have to figure a way to run it on port 80 or do some port-forwarding stuff that I may or may not have access to via the VPS.  I think I've sort of figured out the problem with DreamHost support - my problem had to do with making sure the app is running in production instead of development - but it's definitely been a bit of a headache!

Lauren

Jonathan Rochkind

unread,
Sep 22, 2015, 12:13:08 PM9/22/15
to umlaut-...@googlegroups.com
Yes, you are definitely right that you want to run on port 80 or 443 -- that SFX installed on a non-standard port itself caused us loads of trouble when we were still using SFX directly. (Does SFX still do that? It's a mistake!). We had to change all of our many dozens of vendor registrations to a new OpenURL base.  It was a challenge.

One typical way to deploy Rails apps with apache is to run them on a non-standard port, but use Apache reverse proxy configuration to point to them -- so to the end-users, it's on port 80, they connect to apache on port 80, but apache sends the traffic to the rails app running on another port (which is not accessible to the outside world itself).

That, for instance, is the standard way to deploy with Unicorn or Puma -- which are alternatives to Passenger.

That apache reverse proxy is not usually required with Passenger, since Passenger has integration with apache on an apache module. But you CAN deploy Passenger that way too, running passenger standalone on another port (as you are doing when you execute the `passenger` command on the command line), but using apache reverse proxy to point to it from a certain apache virtual host, or even a certain path on an apache virtual host.

But that's all assuming you had a server you had full control over and could do what you want with. If you are constrained by Dreamhost restricted environment and/or existing things you already have running on that server you don't want to break, then it's just a question of what those constraints are. Sounds like you're on the way to figuring it out, good luck!

Jonathan
Reply all
Reply to author
Forward
0 new messages