nginx errors - "Too many open files"

[Chris]

I just moved a site over to EC2 (through Scalr of course) late last
night and I'm seeing some odd errors in the nginx error log and have
gotten some sporadic reports of Internal Server Errors. Here's an
example of an error I'm seeing a bunch of....

2009/02/28 21:07:58 [alert] 4385#0: *5603516 socket() failed (24: Too
many open files) while connecting to upstream, client: 99.250.5.208,
server: ec2-75-101-241-112.compute-1.amazonaws.com, URL: "/ad.php?
adslot=forums_ss", upstream: "http://10.250.54.64:80/ad.php?
adslot=forums_ss", host: "www.hockeybuzz.com", referrer: "http://
www.hockeybuzz.com/boards/thread.php?thread_id=61444&page=12"

Any ideas?

[Nickolas Toursky]

Seems like you have hit open files limit. Consider raising 'ulimit -n'
on the www instance.

[Chris]

I'm not familiar with that value. What would you recommend that I set
it to?

On Feb 28, 9:17 pm, Nickolas Toursky <hin...@gmail.com> wrote:
> Seems like you have hit open files limit. Consider raising 'ulimit -n'
> on the www instance.
>

[Nickolas Toursky]

It should be 2-3 times greater than 'worker_connections' nginx setting.

[Chris]

Nick,

I've got that setting corrected, but we're still experiencing some
pretty significant issues. Even though I do have app instances up and
running, often time users are getting this message when visting the
site:

No app instances configured
Possible reasons:
# You did not add app role to this farm.
# No app instances started yet.

Any ideas why this would be happening?

Chris

On Mar 1, 6:26 am, Nickolas Toursky <hin...@gmail.com> wrote:
> It should be 2-3 times greater than 'worker_connections' nginx setting.
>

[Elvis]

We're getting the same "No App Instances Configured", intermittantly
and without any explanation in error logs. It seem sto happen under a
bit of load, but nothing close to our scaling limit.

Any ideas?
//Noam

On Mar 1, 6:33 pm, Chris <chris.degr...@gmail.com> wrote:
> Nick,
>
> I've got that setting corrected, but we're still experiencing some

> pretty significant issues.  Even though I do haveappinstancesup and

> running, often time users are getting this message when visting the
> site:
>
> Noappinstancesconfigured
> Possible reasons:
> # You did not addapprole to this farm.

> #Noappinstancesstarted yet.

>
> Any ideas why this would be happening?
>
> Chris
>
> On Mar 1, 6:26 am, Nickolas Toursky <hin...@gmail.com> wrote:
>
>
>
> > It should be 2-3 times greater than 'worker_connections' nginx setting.
>
> > On Sun, Mar 1, 2009 at 5:22 AM, Chris <chris.degr...@gmail.com> wrote:
>
> > > I'm not familiar with that value.  What would you recommend that I set
> > > it to?
>
> > > On Feb 28, 9:17 pm, Nickolas Toursky <hin...@gmail.com> wrote:
> > >> Seems like you have hit open files limit. Consider raising 'ulimit -n'
> > >> on the www instance.
>
> > >> On Sun, Mar 1, 2009 at 4:15 AM, Chris <chris.degr...@gmail.com> wrote:
>
> > >> > I just moved a site over to EC2 (through Scalr of course) late last
> > >> > night and I'm seeing some odd errors in the nginx error log and have
> > >> > gotten some sporadic reports of Internal Server Errors.    Here's an
> > >> > example of an error I'm seeing a bunch of....
>
> > >> > 2009/02/28 21:07:58 [alert] 4385#0: *5603516 socket() failed (24: Too
> > >> > many open files) while connecting to upstream, client: 99.250.5.208,
> > >> > server: ec2-75-101-241-112.compute-1.amazonaws.com, URL: "/ad.php?
> > >> > adslot=forums_ss", upstream: "http://10.250.54.64:80/ad.php?
> > >> > adslot=forums_ss", host: "www.hockeybuzz.com", referrer: "http://
> > >> >www.hockeybuzz.com/boards/thread.php?thread_id=61444&page=12"
>

> > >> > Any ideas?- Hide quoted text -
>
> - Show quoted text -

[Nickolas Toursky]

The page with 'No app instances configured' is a default handler for
502 503 and 504 errors.
Most likely, your app server takes longer to respond than a default
proxy_read_timeout nginx setting (you can read more at
http://www.nginx-community.org//NginxHttpProxyModule#proxy_read_timeout).
In this case error 504 Gateway timeout is raised.
You can discover more information about nginx configuration at
http://www.nginx-community.org/ site.
Consider raising logging level to see, what exactly causing this problems.