jemalloc with Nginx + Phusion Passenger

[Piotr D. Kaczorowski]

Hello,

How to change memory allocator when I'd like to user NGINX + Phusion Passenger ?

Has anyone tried something like this?

Kinds regards,

Piotr

[Tinco Andringa]

Hi, we have not tried this. May I ask what your reason would be for
trying out a different memory allocator?

Kind regards,
Tinco

> --
> You received this message because you are subscribed to the Google Groups
> "Phusion Passenger Discussions" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to phusion-passen...@googlegroups.com.
> To post to this group, send email to phusion-...@googlegroups.com.
> Visit this group at http://groups.google.com/group/phusion-passenger.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/phusion-passenger/39d77816-2555-43fa-90b9-35c3ca536ca9%40googlegroups.com.
> For more options, visit https://groups.google.com/d/optout.



--
Phusion | Web application deployment, scaling and monitoring solutions

Web: http://www.phusion.nl/
E-mail: in...@phusion.nl
Chamber of commerce no: 08173483 (The Netherlands)

[Piotr D. Kaczorowski - KACZOROWSKI-CAPITAL]

I've made it.  Better allocator could improve speed and memory usage.

This is the solution:

1. You should install jemalloc via: apt-get install libjemalloc1 libjemalloc-dev

2. Create file /usr/sbin/nginx-loader that includes:

#!/bin/sh
/usr/local/bin/je /usr/sbin/nginx

3. Change attributes:

# chmod a+x /usr/sbin/nginx-loader

4. Correct /etc/init.d/nginx 

DAEMON=/usr/sbin/nginx-loader

5. Run service:

# service nginx start

6. Check if everything is ok

# pmap `pidof PassengerLoggingAgent` | grep -qi jemalloc && echo "Passenger uses jemalloc."

Kind regards,

Piotr

You received this message because you are subscribed to a topic in the Google Groups "Phusion Passenger Discussions" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/phusion-passenger/6SMOIF44Amk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to phusion-passen...@googlegroups.com.

To post to this group, send email to phusion-...@googlegroups.com.
Visit this group at http://groups.google.com/group/phusion-passenger.

To view this discussion on the web visit https://groups.google.com/d/msgid/phusion-passenger/CAGW%3D8Rp7hVpV4SJTrBs-QrjAvcRPQixkigT6GgiTyWfvX2LG9g%40mail.gmail.com.

[Tinco Andringa]

Hi,

Thanks for sharing your solution, did you perform any tests, was it
significantly better?

Kind regards,
Tinco

> https://groups.google.com/d/msgid/phusion-passenger/CAJvzLHht-DD%2B2UDWk05d9Wxx%2BqnjZ8jjPyjca9gKYzLEN_S%2Bpw%40mail.gmail.com.

[Piotr D. Kaczorowski - KACZOROWSKI-CAPITAL]

Yes.. Few months ago I did some tests after reading:

1) http://marianposaceanu.com/articles/improve-rails-performance-by-adding-a-few-gems

2) https://gist.github.com/dakull/5f3d22eff5fffb43e511

3) https://github.com/jemalloc/jemalloc

4) https://www.facebook.com/notes/facebook-engineering/scalable-memory-allocation-using-jemalloc/480222803919

5) http://goog-perftools.sourceforge.net/doc/tcmalloc.html

6) http://locklessinc.com/

7) http://samsaffron.com/archive/2014/04/08/ruby-2-1-garbage-collection-ready-for-production

In my opinion the best one is jemalloc (by Facebook).  After all you could install it via apt-get and take updates from official repositories.

Standard GC/allocator is not ready for rubust production enviroment.  You should consider that Google and Facebook are using custom memory allocator as well.

Best regards,

Piotr

To view this discussion on the web visit https://groups.google.com/d/msgid/phusion-passenger/CAGW%3D8RpR72QvHumsk4tZYk_JBSkzwUryV_8TwkJLwSdmdtWguQ%40mail.gmail.com.

[Hongli Lai]

The approach that we're going for in future versions of Phusion
Passenger, is to allocate memory as little as possible. It's true that
jemalloc is faster for many kinds of workloads and results in less
fragmentation, but even better than a faster malloc is not malloc'ing
at all.

On Mon, Sep 29, 2014 at 4:09 PM, Piotr D. Kaczorowski -

> https://groups.google.com/d/msgid/phusion-passenger/CAJvzLHgDJe6P98yWr-jJvk0Cy6EvduPE4xM%2BsHH6djiABfb6%3Dw%40mail.gmail.com.

>
> For more options, visit https://groups.google.com/d/optout.



--

Phusion | Web Application deployment, scaling, and monitoring solutions

[Piotr D. Kaczorowski - KACZOROWSKI-CAPITAL]

Well.. The better way to not allocating memory is to turn off system ;)

The reason of providing jemalloc with nginx/passenger is its integration with RoR, that works dramatically better with custom memory allocator.

To view this discussion on the web visit https://groups.google.com/d/msgid/phusion-passenger/CAM3ce8G3W%3D6aE%2B07PFtgmq_tu4ha-RYsZ63ALkKFojjhMm%3DLew%40mail.gmail.com.

[Tinco Andringa]

Hey Piotr,

Are you sure RoR gets the jemalloc if you run it like that? Passenger
launches the application as subprocesses. An easy way to just enable
jemalloc for ruby with passenger would be to make a wrapper for ruby,
and point the passenger_ruby variable to that wrapper.

Kind regards,
Tinco

On Mon, Sep 29, 2014 at 6:18 PM, Piotr D. Kaczorowski -

> https://groups.google.com/d/msgid/phusion-passenger/CAJvzLHgkVvsDs2HCS3%2BgTAddorVD9H-2J5dPDh%2BAmcpbFoXhig%40mail.gmail.com.

[Piotr D. Kaczorowski - KACZOROWSKI-CAPITAL]

100%.

You can test it with pmap (and pidof):

 # pmap `pidof PassengerLoggingAgent` | grep -qi jemalloc && echo

Children process have also LD_PRELOAD (that was set vi "/usr/bin/je") variable in enviroment so will inherit allocator settings.

Best regards,

Piotr

To view this discussion on the web visit https://groups.google.com/d/msgid/phusion-passenger/CAGW%3D8Ro0mwF%3DfjstKkLKap1d049JQ8o%2BJJVCCEYWkC%2Bm05XC_w%40mail.gmail.com.

[Piotr D. Kaczorowski - AUDYOO]

Hey Tinco,

Happy New Year at first at the beginning! 

You are doing good job and I'm still big fan of Phusion Passenger.  

I'm thinking about Phusion Passenger 5.0 as prodcution environment in the near future released project.  I wonder what could be a time horizon when PP 5.0 will be stable version?  This is kind of this month or rather three monhs?

Do you have any expected release date ?

Support for jemalloc will be included?  Ruby 2.2 has experimental support (in fact not working during compilation).

Best regards and greetings from Warsaw,

Piotr

To view this discussion on the web visit https://groups.google.com/d/msgid/phusion-passenger/CAGW%3D8Ro0mwF%3DfjstKkLKap1d049JQ8o%2BJJVCCEYWkC%2Bm05XC_w%40mail.gmail.com.

[Hongli Lai]

On Wed, Jan 7, 2015 at 12:53 AM, Piotr D. Kaczorowski - AUDYOO
<pio...@audyoo.net> wrote:
> Hey Tinco,
>
>
> Happy New Year at first at the beginning!
>
>
> You are doing good job and I'm still big fan of Phusion Passenger.
>
> I'm thinking about Phusion Passenger 5.0 as prodcution environment in the
> near future released project. I wonder what could be a time horizon when PP
> 5.0 will be stable version? This is kind of this month or rather three
> monhs?
> Do you have any expected release date ?

Hi Piotr. We're aiming for end January / begin February. The targeted
release date for beta 3 is January 13, and hopefully this is the last
beta. The targeted release date for Release Candidate 1 is January 24.
If everything goes well, 5.0.1 will be released soon after.

> Support for jemalloc will be included? Ruby 2.2 has experimental support
> (in fact not working during compilation).

No, but since you can use LD_PRELOAD you don't really need us to support it.

[Piotr D. Kaczorowski - AUDYOO]

Hi Piotr. We're aiming for end January / begin February. The targeted
release date for beta 3 is January 13, and hopefully this is the last
beta. The targeted release date for Release Candidate 1 is January 24.
If everything goes well, 5.0.1 will be released soon after.

 

That's a great news. So we will provide on our prepared production env right now PP 5.0 beta that will be changed after few weeks for final version. That's the plan.

No, but since you can use LD_PRELOAD you don't really need us to support it.

That's right.  This is how I still do it in Ruby 2.2 and previous PP. 

Best regards,

Piotr

[Piotr D. Kaczorowski - AUDYOO]

Hey Tinco,

Could be helpfull.  This is "Benchmarking Ruby with GCC".  There is no comparition with GCC 5.0, but is considered to slower in comparision with 4.9.2, in case of shorter time of compiling.

More about that here:

https://p8952.info/ruby/2014/12/12/benchmarking-ruby-with-gcc-and-clang.html?utm_source=rubyweekly&utm_medium=email

So proparbly best performance for PP (and RoR) could achived when both compiled with GCC 4.9 O2.

Best,

Piotr

--
You received this message because you are subscribed to a topic in the Google Groups "Phusion Passenger Discussions" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/phusion-passenger/6SMOIF44Amk/unsubscribe.
To unsubscribe from this group and all its topics, send an email to phusion-passen...@googlegroups.com.
To post to this group, send email to phusion-...@googlegroups.com.
Visit this group at http://groups.google.com/group/phusion-passenger.

To view this discussion on the web visit https://groups.google.com/d/msgid/phusion-passenger/CAM3ce8HR7HufecXiMO8GRY-fmaUyupX%3DxLNHAKKS4PHZ6J-EFQ%40mail.gmail.com.

[davi...@gmail.com]

With the release of Ruby 2.2, you can now compile it with jemalloc, by passing the --with-jemalloc flag, to ./configure. If you are using RVM: rvm install 2.2.0 --with-jemalloc.