Rack VMSize increase after streaming and never went down.

[leo chan]

Hi all,

  I have a rails app that stream data from another server and serve to client. It is basically act as a pass through. Here is the example of how I do the streaming :

    self.response_body = Enumerator.new do |y|

      Net::HTTP.start(url.host, url.port) do |http|

        http.request(file_request) do |web_host_response2|

          web_host_response2.read_body do |chunk|

            y << chunk

          end

        end

      end

    end

The issue when streaming large file like 1Gb, then when I do passenger-memory-status and it show the Rack process VMsize went to a large size like 4Gb to 5Gb

.  Anyone has this issue before or know what I should do?:

1969   328.0 MB   ?        Rack: /var/www/apps/nobu/current

22460  208.4 MB   ?        PassengerWatchdog

22463  817.2 MB   ?        PassengerHelperAgent

22465  110.5 MB   ?        Passenger spawn server

22470  148.1 MB   ?        PassengerLoggingAgent

22501  305.3 MB   ?        Passenger ApplicationSpawner: /var/www/apps/nobu/current

22508  5772.0 MB  ?        Rack: /var/www/apps/nobu/current

23495  4225.5 MB  ?        Rack: /var/www/apps/nobu/current

[leo chan]

Sorry, I should hav mentioned that I am using rails 3.2 and running on passenger 3.0.19

[Hongli Lai]

The vm size doesn't matter. It is a meaningless metric of actual memory usage, for reasons that are too difficult to explain in a short email. You should only look at the private dirty rss, which is '"?" in your output, probably because you didn't mount /proc properly or because you didn't run passenger-memory-stats as root. Do either of those and you should be able to see a better metric for memory usage.

Sent from my Android phone.

Op 10 mei 2013 02:57 schreef "leo chan" <leoy...@gmail.com> het volgende:

--
You received this message because you are subscribed to the Google Groups "Phusion Passenger Discussions" group.
To unsubscribe from this group and stop receiving emails from it, send an email to phusion-passen...@googlegroups.com.
To post to this group, send email to phusion-...@googlegroups.com.
Visit this group at http://groups.google.com/group/phusion-passenger?hl=en-US.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

[leo chan]

Thank you for replying.  You are right about not running passenger-memory-status as root.  We did that and the physical dirty rss also very large like the VMsize.  We found out what the issue is.  It turned out it is Rack-Cache caching our streaming data in the memory because we set the Cache-Control to public when we do the streaming and Rails 3 will automatically cache that in Rack-Cache.  We ended disable Rack-Cache and the issue is resolved. Thanks for the help.