Running a busy Production Janus server. Recommended Settings for process , nice value?

[Kaplan]

Hi Folks,

I have been running Janus for a while using this systemd service config file. The nice thing about it is that if janus ever crashes, it gets restarted automatically by systemd. I do the same for the fronting nginx server that handles the https connections to janus.

The server where this runs, only runs Janus, so I don't mind Janus taking most of the resources.

 I am trying to fight an Issue I been running into lately where the whole machine becomes unresponsive for many seconds, over and over.  wondering if running Janus at CPUSchedulingPolicy = RR (realtime) and nice value of -10 could be the issue.  Interested in what others do ti run a busy production Janus server?  If I look at 
the process on the machine, I see Janus running with PRI -90, and NICE of -10

[Unit]

Description=Janus WebRTC gateway

After=network.target

[Service]

Type=simple

ExecStart=/opt/janus/bin/janus -o

Restart=always

RestartSec=3

Nice=-10

TasksMax=infinity

LimitCORE=infinity

LimitNOFILE=1000000

LimitNPROC=60000

LimitSTACK=250000

LimitRTPRIO=infinity

LimitRTTIME=7000000

IOSchedulingClass=realtime

IOSchedulingPriority=2

CPUSchedulingPolicy=rr

CPUSchedulingPriority=89

[Install]

WantedBy=multi-user.target

[m...@xabe.de]

... three years later :-)

Have you found a solution? We are running with quite identical issues and are facing the same issues.Sometimes even if the server has a low load.

[Mirko Brankovic]

I've had this happen, but on older versions of janus, specifically websocket transport plugin that was deadlocked.

Maybe details of your system would help.

--
You received this message because you are subscribed to the Google Groups "meetecho-janus" group.
To unsubscribe from this group and stop receiving emails from it, send an email to meetecho-janu...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/meetecho-janus/1df5f060-20b7-4592-bf3f-2a02e14d1a93n%40googlegroups.com.

[m...@xabe.de]

Hm. We have three issues with a poor behaviour

1) Ping which goes sporadically high like 1 second

2) dropouts in video and audio (Which correspond with high ping)

3) Very sporadic timeouts to the janus-websocket via localhost (we have a java app running on the same server)

We are using two cloud vendors

- 4.19.0-16-cloud-amd64 #1 SMP Debian 4.19.181-1 (2021-03-19) x86_64 GNU/Linux

- 4.19.0-14-amd64 #1 SMP Debian 4.19.171-2 (2021-01-30) x86_64 GNU/Linux

Janus:

- Janus commit: 8ab7a004009831fc4d0e8b2ef84155030bdfbb2e
- Apr 24 21:29:11 act-61fa4f6b-149f-4a05-8950-5bc9c71c043a janus[909]: Compiled on:  Fri 23 Apr 2021 10:08:23 PM CEST

Resolution:

Reducing the janus.service to this: https://pastebin.com/cXkt7UFr

Reproducing was also quite easy. Take a small 4core cloud instance, put 20 users with webcam and microphone (audiobridge) on it, and after some time you will see the described behaviour.

For us, it is solved. Lessons learned: Less is sometimes more (especially using params you don't really understand). Hope this helps others, who might stumble upon this.

Still, if anyone has some ideas how to improve our settings, I am happy to hear :)