Warden Container ulimit - 1024

[work....@gmail.com]

Logged into warden containers:

ulimit -n

Returns 1024.

I researched this issue and found that for CF v2, ulimit for files should be 16k. Which is clearly not the case although I have CF v194 release deployed presently.

I also understand that this is not a configurable parameter right now.

But I would like it to be at least 16k.

Help would be appreciated.

[work....@gmail.com]

I meant CF release version 194.

[Dieu Cao]

The file descriptor limit per container is actually 16,384 right now [1]

There is a PR [2] prioritized in our backlog that will make the value manifest configurable.

-Dieu

CF Runtime PM

[1] https://github.com/cloudfoundry/cloud_controller_ng/blob/master/lib/cloud_controller/config.rb#L330

[2] https://www.pivotaltracker.com/story/show/85215352

--
You received this message because you are subscribed to the Google Groups "Cloud Foundry Developers" group.
To view this discussion on the web visit https://groups.google.com/a/cloudfoundry.org/d/msgid/vcap-dev/db7284aa-41a5-4d7d-b744-5d626e07055f%40cloudfoundry.org.

To unsubscribe from this group and stop receiving emails from it, send an email to vcap-dev+u...@cloudfoundry.org.

[work....@gmail.com]

@Dieu Cao

I understand that the limit per container is 16,384. But my containers are showing it up as 1024. Needed some help to understand how this could happen. Is it based on the stemcell used?

[work....@gmail.com]

@Dieu

My bad. Should have gone through all the links that you gave.

Checked them out and let me see whats wrong with my installation and get back to you.

[Dieu Cao]

Yes, I'm not sure why ulimit -n is not reporting the correct value, as when I run ulimit -n I also see it's 1024, but if you test it you'll see that it is actually 16384.

-Dieu

--
You received this message because you are subscribed to the Google Groups "Cloud Foundry Developers" group.

To view this discussion on the web visit https://groups.google.com/a/cloudfoundry.org/d/msgid/vcap-dev/c050a0e3-85f1-4092-9c8f-5e6c8285272a%40cloudfoundry.org.

[Matthew Sykes]

How are you checking the limit?  If you're "logging into the container" by doing a bin/wsh, you're inheriting the default soft limit on the system. If you have an ubuntu stemcell, that's 1024.

If you want to see the limit that was applied to the app process, enter the container, find the pid, and then cat /proc/:pid/limits to see the limits that apply to the process:

root@18do8pufg2q:~# ps ax

  PID TTY      STAT   TIME COMMAND

    1 ?        S<s    0:00 wshd: 18do8pufg2q    

   31 ?        S<sl   0:00 ruby /home/vcap/app/vendor/bundle/ruby/2.0.0/bin/rackup config.ru -p 61002

   35 pts/0    S<s    0:00 /bin/bash

   45 pts/0    R<+    0:00 ps ax

root@18do8pufg2q:~# cat /proc/31/limits 

Limit                     Soft Limit           Hard Limit           Units     

Max cpu time              unlimited            unlimited            seconds   

Max file size             unlimited            unlimited            bytes     

Max data size             unlimited            unlimited            bytes     

Max stack size            8388608              8388608              bytes     

Max core file size        0                    0                    bytes     

Max resident set          unlimited            unlimited            bytes     

Max processes             512                  512                  processes 

Max open files            16384                16384                files     

Max locked memory         65536                65536                bytes     

Max address space         unlimited            unlimited            bytes     

Max file locks            unlimited            unlimited            locks     

Max pending signals       1024                 1024                 signals   

Max msgqueue size         819200               819200               bytes     

Max nice priority         0                    0                    

Max realtime priority     0                    0                    

Max realtime timeout      unlimited            unlimited            us        

root@18do8pufg2q:~# ulimit -n

1024

That's the output I get from an app hosted on a v195 DEA. You'll notice the app limit is 16385 even though the limit in my shell is 1024. It has to do with the fact that wshd is actually going out of its way to set the resource limits for the processes it spawns. Those limits should then be inherited by processes created by the app.

To view this discussion on the web visit https://groups.google.com/a/cloudfoundry.org/d/msgid/vcap-dev/53486f43-93d6-4dbd-8ec1-3b3eac0718d8%40cloudfoundry.org.

To unsubscribe from this group and stop receiving emails from it, send an email to vcap-dev+u...@cloudfoundry.org.

--

Matthew Sykes
matthe...@gmail.com

[Daniel Mikusa]

Try pushing an app and looking at ulimit through the app.  Last time I looked at this, it was different than just connecting into the container.

Dan

--
You received this message because you are subscribed to the Google Groups "Cloud Foundry Developers" group.

To view this discussion on the web visit https://groups.google.com/a/cloudfoundry.org/d/msgid/vcap-dev/9ef7f6da-4cc3-4ec5-9b24-75b20d33aa57%40cloudfoundry.org.

[work....@gmail.com]

Yep. Got it. Thanks for the inputs. I installed websocketd (https://github.com/jbayer/websocketd-cloudfoundry) and it shows up the correct limit as expected.

Which is good and bad news because that means that the "Out of memory error" problem I am facing is related to user process limit (ulimit -u) as described by Troy in this blog:

http://blog.troyastle.com/2014/10/debugging-javalangoutofmemoryerror.html

ulimit -u is 512. Is there anyway I can increase that?

[work....@gmail.com]

Thanks Matthew.

The ulimit -n is correct @ 16384 as expected.

Any chance you would know how to increate the user process limit? ulimit -n? Its just 512 right now.

 

[work....@gmail.com]

User process limit is 'ulimit -u' my bad.

[Matthew Sykes]

Warden supports it as a configurable resource limit but the DEA has it hard coded[1] at 512. It should be pretty straightforward to make it configurable via PR.

[1]: https://github.com/cloudfoundry/dea_ng/blob/3b99bce6e9c1625f35493f7d296dae070c458dd4/lib/dea/starting/instance.rb#L24

--

You received this message because you are subscribed to the Google Groups "Cloud Foundry Developers" group.

To view this discussion on the web visit https://groups.google.com/a/cloudfoundry.org/d/msgid/vcap-dev/42a92ac5-8446-4d97-a6b4-d7d9c97a5294%40cloudfoundry.org.

To unsubscribe from this group and stop receiving emails from it, send an email to vcap-dev+u...@cloudfoundry.org.

--

Matthew Sykes
matthe...@gmail.com

[work....@gmail.com]

Thanks. Will get in touch with CF team to see if they will entertain this PR.

In the meantime, thanks to you I know where to make the code change if I need it..

Appreciate the help.