Scheduling processing of recordings

[Chris Halliwell]

Hello,

I built a stand alone, dedicated, bigbluebutton server a couple of weeks ago, which is minimal spec:

CPU: i5-4590 CPU; 3.30GHz. 4 cores but does not support hyperthreading

RAM: 8Gb

OS: Ubuntu 16.04.6 Server Ed. (Kernel 4.4.0-176-generic)

BBB vers: 2.2.5

Interfaces to bigbluebutton are both Greenlight, and the Moodle plugin, and over all things are working very well.

Problems kick in when recorded sessions end, and begin to encode. ffmpeg launches and drives the CPU usage across cores from a very passable 30% - 50% straight up to 170% - 200%. Large seminars in the afternoon can be in for a rough ride as the morning lectures process.

I was hoping someone with more experience and insight into both (a) the bigbluebutton recording process, and (b) the ruby language, might be able to advise if it's feasible to delay processing to outside of teaching hours? My idea is that the recordings process overnight and be available the next day.

I notice that ffmpeg is called by presentation.rb, itself called by rap-process-worker.rb. But might it make more sense to stall things once an archive is created? So archives could go on being created throughout the day but further processing on each of them deferred? Might rap-archive-worker.rb then be the script to modify?

At the moment, my crude and ineffectual workaround is a bash script which checks for new process IDs for ffmpeg every minute, and renices them as much as possible, but this is not making much difference to load. ffmpeg appears to have no child processes, and I'm not aware of any way of renicing each thread outside of - precisely - renicing the process that spun the threads itself.

I realise that probably the right thing to do here is build a dev environment, learn more about ruby, and hack till it works. But like many IT people in education at the moment I'm coping with unprecedented demand for online learning tools and time and resources are stretched, so I'm asking here incase someone has already dealt with this or a similar problem and is willing to share their insights.

Many thanks for the extraordinary software.

Chris

[Betim Gashi]

Hello

Please have a look here https://groups.google.com/forum/?utm_medium=email&utm_source=footer#!msg/bigbluebutton-users/hQ-PXKzXdLQ/TJlBKgdABQAJ

This is my solution for making records to get processed at specific time for example at night and by doing this I reduced 50% processor usage of my virtualization Hypervisor.

[Devrim Seral]

Hello Chris,

Problem is related rap-process-worker.rb. Because in BBB every record randomly generated and rap-process-worker.rb did not respect their creation timestamp. 

It uses ordinary file sort.

We did some patch for it to fix at least time order problem to order process time. 

https://github.com/bigbluebutton/bigbluebutton/pull/8923

This patch may help you to process recordings with time ordered..

Regards

Devrim

[Chris Halliwell]

Hi Betim,

That's exactly what I was looking for, thanks so much for sharing your insights. I tested it last night and it worked perfectly on the server here so have implemented it going forward.

All the best,

Chris

[Chris Halliwell]

Hi Devrim,

Thanks for this. That's an improvement on the existing sort and it makes a lot of sense, I don't see a way of really moving from there to what I'm trying to achieve, which is not so much ordering the processing of recordings in a particular way as deferring all processing to run at a particular time, so I have gone with Betim's solution for now.

All the best,

Chris