New post script to export recordings as a video file

[Daniel Petri]

Hi there,

I just wanted to quickly present a script I’ve been working on for the past few weeks now.

Enabling users to download a recording as a video is the second-most commented issue on BBB’s GitHub repository, with current workarounds relying on headless Chrome browsers to perform a screen capture in the background or missing critical parts of the recording such as the annotations on the whiteboard.

To address this, I’ve written a Ruby postscript that runs on completely unmodified servers to recreate the recordings from the web player. It shows the webcams, deskshares, cursor, polls, slides, text, chat, zooms and annotations.

 Here’s a quick demonstration of the script’s output on BBB 2.3-dev:

https://drive.google.com/file/d/1H5004sX6OLdZBrs6gS-nWsm2HTyuRhUy/view?usp=sharing

The scripts can be found in the following repo:

https://github.com/danielpetri1/bbb-recording-exporter

It is based mostly on FFmpeg compiled with librsvg to render the visible frames of the recording.

I’d love any feedback on the project and would really appreciate some beta testing, as it is still very much a work in progress. There’s a lot to optimize: the frames that compose the annotations, chat and cursor contain a lot of redundant information, which for very long / annotation-intense meetings slow the export down a lot and can use quite a bit of storage space as well.

Still, for a standard use case, I’ve been seeing 4-5x faster exports than a screen recording would be able to achieve. With this proof of concept, my goal is to make a pull request after the optimizations are performed to close issue #1969, which was opened in 2015.

Kind regards and thanks in advance!

Daniel

 

[Mario Gasparoni Junior]

Very nice work, Daniel.

Cheers.

--
You received this message because you are subscribed to the Google Groups "BigBlueButton-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bigbluebutton-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bigbluebutton-dev/521c515d-062c-4c00-a9c8-5d898584e75cn%40googlegroups.com.

--

Att. Mário Gasparoni.

www.inf.ufrgs.br/~mcgjunior

[Juan Pablo Miranda]

Hi, Daniel.

Very easy to install, find and download the mp4 file.

Codecs and its configurations, more than sufficient to see and hear the record clearly.

Very usefull contribution.

Thanks a lot your development.

Greetings

JP

[Fred Dixon]

Nice work Daniel!

Regards,... Fred

--

You received this message because you are subscribed to the Google Groups "BigBlueButton-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bigbluebutton-...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/bigbluebutton-dev/49e2a9bf-8ed4-4c73-a66e-81bdb5c96919n%40googlegroups.com.

--

BigBlueButton Developer

http://bigbluebutton.org/

Like BigBlueButton?  Tweet us at @bigbluebutton

[Hiroshi Suga]

Congratulations for your work! Working pretty well.

A quick question. Do you think it would be possible to use the "mask" and "clip-path" elements for rendering the whiteboard annotations in your script?

I am working on BBB PRs to enable eraser (https://github.com/bigbluebutton/bigbluebutton/pull/11018) and marker (https://github.com/bigbluebutton/bigbluebutton/pull/11021) functions, which do not work in the current implementation (nothing appears for eraser, a simple pencil drawing appears for marker). I guess we need to replace xlink:href attribute in the mask and clip-path elements. Any suggestion is greatly appreciated.

[Daniel Petri]

Thanks everyone for the positive feedback!

@Hiroshi Suga:  librsvg did manage to render the mask in this example and the clip paths in this tutorial, so I do believe it should be possible to support the marker and eraser in the script with a few changes. I'll fetch your PR to test it out over the course of the next few days to find out why the eraser and marker did not show up properly.

Kind regards,

Daniel

[Uzaktan Eğitim]

Hello, great work. I think it would be perfect if the following features were also added.

 

1- Is it possible to write the names of the users on the camera images in the recordings taken with BBB?

 

2- Can we add the user list to the records? In this way, we can see who attended the lesson.

3- Records take up a lot of space over time and need to be deleted manually. Can we have both the original bbb recordings and the mp4 converted recordings automatically deleted from the server after a certain period of time?

4- Finally, can we get a button to download the recordings in Moodle?

Waiting for your urgent reply. Respects.

30 Mayıs 2021 Pazar tarihinde saat 15:54:51 UTC+3 itibarıyla Daniel Petri şunları yazdı:

[Hiroshi Suga]

Hi. 

All of your questions / requests are not really related to this thread. You will have better response if you submit it as a separate post.

For the first question, nevertheless, https://github.com/bigbluebutton/bigbluebutton/pull/10277 might help.

[Daniel Petri]

Hi, thanks for trying out the script.

Concerning your first point, I was about to recommend you the pull request Hiroshi has already sent in here. As for the list of who participated, I believe that that already is a feature of BBB2.3.

Within the coming week I will push a new version of the script that does not create auxiliary files for the cursor and chat, saving a lot of time and storage space. Code for that is already on one of the development branches of the script.

My current priority is to get the functionality done for features already present on BBB before moving onto new ones, the UI, and integrations with platforms like Moodle. Still, there's already an integration with Greenlight if desired. Writing a small cronjob to automatically delete meeting.mp4 file every x days shouldn't be too hard either.

Kind regards

Daniel

[Uzaktan Eğitim]

Hello, thank you very much for your feedback.

Where exactly do we get the list of exhibitors in BBB version 2.3. I would appreciate it if you could give some more details.

The reason I want this is to determine which students attended the class or not, when the recorded videos are later reviewed by the inspectors. Students may not always be able to turn on their cameras. At least, including the list of participants in the videos or as an additional file will make this job easier and students will not lose their rights.

Thank you for your interest and work.

18 Haziran 2021 Cuma tarihinde saat 11:17:13 UTC+3 itibarıyla Daniel Petri şunları yazdı:

[Uzaktan Eğitim]

Hello there,
How can I do this so that the usernames are written on the videos while playing the recordings. What exactly do I need to do in Ubuntu 18.04 BBB version 2.3? I would be very happy if you could write the installation step by step. Thanks.

Hiroshi Suga <suga...@gmail.com>, 18 Haz 2021 Cum, 03:33 tarihinde şunu yazdı:

You received this message because you are subscribed to a topic in the Google Groups "BigBlueButton-dev" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/bigbluebutton-dev/0v8rLgM3OSY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to bigbluebutton-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bigbluebutton-dev/bf7c4ce3-abc9-4f8b-942f-2c2f742d005cn%40googlegroups.com.

[Hiroshi Suga]

Hi,

The easiest way to implement the PR 10277 is to find the files to be modified under /usr/local/bigbluebutton/core of your server and manually apply the changes included in the PR. 

For obtaining the user list, you can do it from the user panel during the meeting. After a meeting ends, https://github.com/bigbluebutton/bbb-events would be helpful for extracting the information from events.xml file under /var/bigbluebutton/published/presentation/* directory.

Hope it helps.

[Uzaktan Eğitim]

Hello, thanks for your interest and help. I followed all the steps one by one. However, the recordings are no longer listed on moodle.

Hiroshi Suga <suga...@gmail.com>, 19 Haz 2021 Cmt, 15:14 tarihinde şunu yazdı:

To view this discussion on the web visit https://groups.google.com/d/msgid/bigbluebutton-dev/85390a32-f854-44e2-8e54-c52e8fff3ab4n%40googlegroups.com.

[Wes]

First, let me say thanks because this looks amazing.

1) I am using BBB 2.2.14. Your guide keeps mentioning render_whiteboard.rb, but I can't find this file. Was that the original name for it, but it actually got changed to export_presentation.rb somewhere along the way and you forgot to change the file name in the guide? All of the settings (REMOVE_REDUNDANT_SHAPES, SVGZ_COMPRESSION, FFMPEG_REFERENCE_SUPPORT, CONSTANT_RATE_FACTOR) can be found in export_presentation.rb. 

2) I installed the latest version of FFMPEG first, and then saw the later step to download the source code first and replace librsvgdec.c with the one you have posted. Since I already have FFMPEG installed can I just find that file on my server and replace it with the one you have provided, or does it have to be done before installing FFMPEG? If so, where exactly does that file get put after installation?

Sorry for my noobness. I only touch this stuff once or twice a year.

[Daniel Petri]

Hi Wes,

Yes, that is exactly what happened, I forgot to change the file name in the guide. Thanks for pointing that out.

You don't have to replace the librsvgdec.c file,  you can run the scripts directly if you have FFMPEG installed by leaving FFMPEG_REFERENCE_SUPPORT as false. The only difference is that the script will write more data on disk temporarily; for most recordings this shouldn't be a problem unless you're dealing with very long recordings often. Since you're on BBB 2.x, just make sure to set REMOVE_REDUNDANT_SHAPES to true, this will have the most impact on the render speed.

The C file provides a patch for FFMPEG so that it can load embedded images from SVGs (specifically the whiteboard slides in this case), instead of performing a conversion step. Just replacing the file is not enough since you would need to recompile FFMPEG to apply the changes, so that would entail downloading the source code, changing that one file, and compiling the code again.  I'll submit this to FFMPEG, hopefully in the future it will be available by default.

Here's what coming to the script soon:

+ Caption support

+ Chapter marks in the resulting MP4 video

+ Chat linebreaks

Cheers!

[BigBlueButton ТСПК]

Good afternoon! Please tell me at what stage is the development of this script now? Will it work on 2.3.14?

[Juan Pablo Miranda]

Hi,

Last version works perfectly on BBB 2.3.13, 2.3.14 and 2.3.15. This version works on 2.4 RC 3 as well.

The 'export_presentation.rb' file has an issue that is easily to correct as it is explained here: https://github.com/danielpetri1/bbb-recording-exporter/issues/58

Greetings

JP

[Ray DeBruyn]

Hi Daniel,

This seems to be a perfect fit for my needs to be able to record to mp4. I plan to check it out on one of our bbb servers.

It made me think of another requirement on a possible project. As I understand, you're stitching together the events and using ffmpg to encode the result to mp4. ffmpeg can also output to rtmp and I was wondering if it's a stretch to make that work. I would think it's possible, but not sure if it's feasible to have one or more running meetings and be encoding/streaming at the same time on the same server.

It's the age old issue of wanting 1000's of people viewing real time and 10's of people actively joining a meeting.

Sorry if it's not the place for the 2nd question. I wanted to message you, but did not have permissions to message the author.

Ray

[Hiroshi Suga]

Hi, have you tried this one?

https://github.com/aau-zid/BigBlueButton-liveStreaming

It was working well in my place a year ago, and the project still seems to be updated.

[detlef Köppel]

or try this    https://github.com/manishkatyan/bbb-streaming    it is almost the same but maybe a little easier to install

Detlef

[Ray DeBruyn]

Cool ty - I'll check it out

[Ray DeBruyn]

Cool ty - I'll check it out

[Ray DeBruyn]

This seems to be a version of katyan's code I've seen. He's running a session in a headless chrome browser, using ffmpeg to screen capture, encode and stream to rtmp. My understanding is that this would be quite resource intensive.

Daniel's approach seems to be more about creating the video by stitching together the events much the way bigbluebutton does. It doesn't require joining the session in a headless browser and doing a screen capture. This makes it far less resource intensive from what I understand.

This is what I gather from reading and walking through the code so far. I may be wrong. Please correct me if I've got it wrong.

Daniel's code uses ffmpeg to encode individual events like webcam. and, I think I recall, a final ffmpeg call that would create the resulting mp4 file. The output of that could be adjusted to rtmp, but I'm not so sure now that this could be executed live. It may be that this can only be done after the meeting is finished.

bigbluebutton itself seems to handle all this in the html5 client. It seems to add items to the page in layers like chat, presentation, screen share, webcams etc. You interact with the page, by opening and closing chat, resizing the presentation etc. So even bigbluebutton is not creating it's video live. It gets created after the meeting is done.

Oh there's so much I don't know lol

I've heard in discussions that it's possible to convert those events live and stream, but that it would be a lot of work to develop. I got excited when I read briefly what Daniel's code does.

When the requirement was discussed, I thought the solution would be to join the session in a browser, screen record, encode and stream. So katyan's solution is a good one. I've heard the encoding is quite resource intensive and better done on another server.

My code so far is run in PLSQL on an Oracle DB. I make api calls to create the meeting and present links to the logged in users to join. I was considering writing info to a shared directory from the DB and having a dedicated server join the session and start streaming.

I'm not even sure we'll get the project. It was a friend we were talking to and he started going down the route of joining the meeting on a browser, handle the screen capture with OBS Studio and stream the output with some other tool. It would require someone technically savvy enough with those tools every time a meeting was started.

On Saturday, April 23, 2022 at 3:21:50 a.m. UTC-4 detlef....@gmx.de wrote:

[Daniel Petri]

Hi Ray,

Nice to hear you're trying out the script.

You're right - unfortunately, the conversion into MP4 relies on data stored after the meeting has already ended. So it would not be possible to use it to stream to RTMP.

Since presenters already have the ability to share their screen during the meeting, it may be possible to do some kind of screen capture of the tab in the background without relying on additional headless browsers. (Just thinking out loud, I'm not too familiar with the HTML5 architecture)

In the meantime, joining as a dummy guest user on a separate device and then starting an OBS feed is how I've previously seen it being done as a workaround. OBS could then also directly capture an MP4 file.

[Ray DeBruyn]

Thanks for replying Daniel :)

It's too bad the conversion has to happen after the meeting has finished. It was fairly apparent once I browsed your code.

As a developer, I still like the headless browser solution since it could be automated. I could make the api call to create the meeting and then write the details including join url to a directory the new server could read (when the instructor joins). It could have a cron job that checks for a new meeting in the directory and runs the script to start the headless browser, record and stream.

Either way, the project only happens if the customer wants to proceed.

I hope to try your mp4 solution today. I need to discuss it with the boss first and get him to agree to use one of our bbb servers as a test.

Nice work on the code. I wish I could follow everything that's being done. As a developer I know I can't be an expert in every language, but I sure want to try lol.

Ray

[Uzaktan Eğitim]

Hi, I can download the recordings as mp4. But only the teacher's camera is visible in the recording. How can I show students' cameras in mp4 recording too?

26 Nisan 2022 Salı tarihinde saat 20:09:01 UTC+3 itibarıyla Ray DeBruyn şunları yazdı:

[Daniel Petri]

Hi,

This MP4 script uses the same processed webcam video as the Playback format. So either no student shared their camera during the meeting or some setting was enabled to prevent their feeds showing up in recordings. If that is not the case, feel free to share the playback link with me so that I can generate the MP4 locally to take a look into it.

Best,

Daniel

[Uzaktan Eğitim]

Hello. During the session, teachers and students turn on their cameras. However, in order to protect personal data, the "See other viewer webcams" option is disabled so that only the teacher can see the student cameras and the students do not see each other. Even in this state, I want all cameras that are open in the session to appear in the recording.

https://prnt.sc/xyh2I8sSGNk5

Daniel Petri <daniel...@gmail.com>, 15 Eyl 2022 Per, 19:49 tarihinde şunu yazdı:

--

You received this message because you are subscribed to a topic in the Google Groups "BigBlueButton-dev" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/bigbluebutton-dev/0v8rLgM3OSY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to bigbluebutton-...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/bigbluebutton-dev/dd8dcb53-7446-43fc-b722-7ddbef926eb8n%40googlegroups.com.

[manish katyan]

Thanks Detlef for sharing my work. 

@Ray: if you face any issues, raise a request on the Github page. 

- Manish

--

You received this message because you are subscribed to the Google Groups "BigBlueButton-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bigbluebutton-...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/bigbluebutton-dev/e274a9d9-264c-4e93-8307-3206a54ad3cbn%40googlegroups.com.