Website under heavy load (Queue Full)

[Dancho Working]

Hi guys and girls,

I'm running concerto 2.3.6 on Debian Linux (The OVA image you can download from GitHub).
Currently I have 189 screens configured, all publicly accessible, displaying only graphics content.
I'm facing a problem is sometimes I'm getting "Website under heavy load (Queue full)".
I changed my /etc/apache2/sites-enabled/concerto.conf to this:

<VirtualHost *:80>
  ServerName concerto
  DocumentRoot /usr/share/concerto/public
  <Directory /usr/share/concerto/public>
    RackEnv production
    AllowOverride all
    Options -MultiViews
  </Directory>
  PassengerMaxRequests 5000
  PassengerMinInstances 3
</VirtualHost>

It improved the situation a lot, but from time to time I'm still getting this Queue full.
My plans are to increase the screens by 30 this year alone, and by 100 in next 3 to 5 years.
If this issue continues it will be very inconvenient for me to use Concerto.

Anyone can help me resolve this issue? Anyone faced it before?

[Dancho Working]

Hi guys,
Me and the bot figured it out.
Before following the instructions bellow, please make sure you are having a backup, VM snapshot or any other type of plan B if your Concerto Server goes south.
It turns out Passenger configuration is not suitable for my setup and needed a tweak.
I increased the resources of the virtual machine to 4vCPUs, 8GB of memory.
***I edited /etc/apache2/sites-enabled/concerto.conf to:

<VirtualHost *:80>
  ServerName concerto
  DocumentRoot /usr/share/concerto/public
  <Directory /usr/share/concerto/public>
    RackEnv production
    AllowOverride all
    Options -MultiViews
  </Directory>

  PassengerMinInstances 12
  PassengerMaxInstancesPerApp 24
  PassengerMaxRequestQueueSize 500
  PassengerMaxRequests 5000
  PassengerPreStart http://localhost/
</VirtualHost>

*** edited also /etc/apache2/mods-available/passenger.conf to:

<IfModule mod_passenger.c>
  PassengerRoot /usr/lib/ruby/vendor_ruby/phusion_passenger/locations.ini
  PassengerDefaultRuby /usr/bin/passenger_free_ruby
  PassengerMaxPoolSize 24
  PassengerPoolIdleTime 300
  PassengerInstanceRegistryDir /var/run/passenger-instreg
</IfModule>

p.s.: What each setting does:

✅ PassengerMaxPoolSize 24 → allows up to 24 Passenger processes total across all apps (you only have one, so effectively up to 24)
✅ PassengerMinInstances 12 → preloads 12 processes so your app is “hot” on startup
✅ PassengerMaxInstancesPerApp 24 → allows your single app to scale up to 24 processes
✅ PassengerMaxRequests 5000 → gracefully restarts a process after 5000 requests to avoid memory bloat
✅ PassengerPreStart http://localhost/ → triggers pre-spawning when Apache starts, avoiding cold startup delays

*** edit /etc/security/limits.conf and added 4 lines:

www-data soft nofile 65536
www-data hard nofile 65536
www-data soft nproc  16384
www-data hard nproc  16384

*** edited both /etc/pam.d/common-session and /etc/pam.d/common-session-noninteractive
added a line session required pam_limits.so to both files.

*** edited both /etc/sysctl.conf and added a line
fs.file-max=2097152

*** edited /etc/apache2/envvars and had to uncomment the following line:
APACHE_ULIMIT_MAX_FILES='ulimit -n 65536'

do a reboot after changing all there so the new changes will take effect.
some useful commands you can use to check if the changes are applied:

cat /proc/sys/fs/file-max

cat /proc/$(pidof apache2 | awk '{print $1}')/limits
ulimit -a