Too high count of threads on "open source" passenger core process

[Pascal Chambon]

Hello,

I've deployed an open soucre version of passenger, meaning that passenger_thread_count is not available, the single process "passenger core" is supposed to be single-threaded.

However htop shows 70+ threads in the passenger core process, only 2-3 of which are dedicated to the RubyApp (redmine).

Passenger-status show 40 request handling threads, as described here :
https://www.phusionpassenger.com/library/admin/apache/overall_status_report.html#dealing-with-common-process-related-problems
It seems logical because I'm on a shared 40-cores server.

I don't need so many passenger threads, since anyway only 1 will handle the app request (theoretically), and because my host limit the number of threads on my accounts, these 70+ threads are blocking my other applications (python etc.).

Is teher any way to limit the number of connection-handling threads in passenger core, in the open source version ?

My performance conf in NGINX:
    passenger_max_pool_size 1;
    passenger_min_instances 1;
    passenger_spawn_method conservative;

Thanks a lot, regards,
P. C.

[Pascal Chambon]

Follow-up:

by browsing the passenger open-source code, I've found these two settings :

    DEFAULT_APP_THREAD_COUNT = 1
    ...
    options.setDefaultInt("app_thread_count", DEFAULT_APP_THREAD_COUNT);  // THIS seems to be the real "single-threaded server" setting

    // THIS seems to be the problem with excessive request-handling threads :
    printf("      --threads NUMBER      Number of threads to use for request handling.\n");
    printf("                            Default: number of CPU cores (%d)\n",
        boost::thread::hardware_concurrency());
   ...
    if (options.getInt("core_threads") < 1) {
        fprintf(stderr, "ERROR: you may only specify for --threads a number greater than or equal to 1.\n");
        ok = false;

The question is so : is there any way to set this "core threads" setting, when using passenger with nginx and not as standalone (in which case this " --threads" setting should do the trick) ?

[Hongli Lai]

Use you can 'passenger_ctl core_threads 1'. It defaults to the number
of CPU cores on your system.

But I don't think that's the full story. The Passenger core controller
processes requests in a hybrid multithreaded/evented manner, but there
are additional background threads for handling stuff like I/O,
spawning, stdout capturing, etc. Those cannot be configured.

> --
> You received this message because you are subscribed to the Google Groups
> "Phusion Passenger Discussions" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to phusion-passen...@googlegroups.com.
> To post to this group, send email to phusion-...@googlegroups.com.
> Visit this group at https://groups.google.com/group/phusion-passenger.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/phusion-passenger/57187b89-8519-45a5-98f1-e64ec3f6c58c%40googlegroups.com.
>
> For more options, visit https://groups.google.com/d/optout.



--
Phusion B.V. | Web Application deployment, scaling, and monitoring solutions

Web: http://www.phusion.nl/
E-mail: in...@phusion.nl
Chamber of commerce no: 63501007 (The Netherlands)

[Pascal Chambon]

Thanks for your answer,

by hacking the sources of standalone startup scipt (in ruby), I've forced to launche only 2 core threads, and in total the passanger-core now uses a dozen threads, which seems fine to me.

I'dl like to use the nginx integration and not this hacked standalone, but I can't find the passenger_ctl command, is it specific to Passenger Enterprise ?
I only have passenger-status and passenger-config which resemble this, but they don't allow these command line args...
And as far as I've seen, nginx-integrated passenger doesn't expose the core-threads option.

thanks,
regards,
P. Chambon

[Daniel Knoppel]

passenger_ctl is not a command but a configuration option that you can use in your nginx config.

- Daniel

[Pascal Chambon]

Thanks a lot for the information,

since I've already moved to a standalone server, here is an example Passengerfile.json which uses an nginx backend and sets core_threads to 2, in case it might help someone.
It passes "passenger_ctl core_threads 2' to the internal nginx conf by using "ctls" syntax, without having to resort to a custom nginx template.

{
  "address": "0.0.0.0",
  "port": "22159",
  "instance_registry_dir": "/home/user/redmine323/tmp",
  "environment": "production",
  "ruby": "/home/user/redmine323/bin/ruby"
  "app_type": "rack",
  "startup_file": "config.ru",
  "spawn_method": "direct",
 
  "envvars": {},
 
  "engine": "nginx",
  "ctls": ["core_threads=2"],              // HERE is the important part
  "max_pool_size": 1,
  "min_instances": 1
  "daemonize": true,
  "log_file": "/home/user/logs/user/error_redmine323.log",
 
  "debug_nginx_config": false,
  "friendly_error_pages": false,
  "log_level": 3,
}

[William Hatt]

Is there any way to do this for people that are using Apache? 

[William Hatt]

In the apache conf file you can do this:

PassengerCtl core_threads 20

Helpful on systems with hundreds of cores :)