RAW recording deletion - 2.3 Build

116 views
Skip to first unread message

G H

unread,
Sep 23, 2021, 7:12:17 AM9/23/21
to BigBlueButton-Setup
Hi,

I have one live BBB and one test BBB server setups.

On both servers the it is 2.3 build.

The 2 new meeting recordings got deleted on LIVE automatically so I am concerned about the mechanism that deletes the RAW recordings automatically.

I just understood that vi /etc/cron.daily/bigbluebutton file has following entries for recording deletion. 

----------------------------------------------------------------------
#
# Delete old raw dirs from recordings properly published using 'presentation' scripts.
#

remove_raw_of_published_recordings(){
    #TYPES=$(cd /usr/local/bigbluebutton/core/scripts/process; ls *.rb | sed s/.rb//g)
    logger "Removing raw directory of old recordings:"
    TYPES="presentation"
    cd /var/bigbluebutton/recording/raw/
    old_meetings=$(find . -name events.xml -mtime +$published_days | cut -d"/" -f2 )
    for meeting in $old_meetings
    do
        PROPERLY_PUBLISHED="true"
        FAILED_TYPES=""
        for type in $TYPES
        do
            file="/var/bigbluebutton/recording/status/published/$meeting-$type.done"
            if ! [ -f "$file" ]; then
                PROPERLY_PUBLISHED="false"
                FAILED_TYPES="$FAILED_TYPES $type"
            fi
        done
        if [ "$PROPERLY_PUBLISHED" == "true" ]; then
            logger "$meeting properly published, removing raw dir."
            rm -r /var/bigbluebutton/recording/raw/$meeting/
        else
            logger "$meeting was not properly published in [ $FAILED_TYPES ]"
        fi
    done
}

remove_raw_of_published_recordings

#
# Remove old *.afm and *.pfb files from /tmp directory (if any exist)
#
find /tmp -name "*.afm" -mtime +$history -delete
find /tmp -name "*.pfb" -mtime +$history -delete
----------------------------------------------------------------------------------

I can see that none of these lines are commented. Especially the function call (i.e. remove_raw_of_published_recordings )

With uncommented lines above are the RAW recordings set to be deleted?

If so, why a large amount of recordings on test server's RAW directory not yet deleted even though RAW recordings are set to be deleted after 14 days?

And MOST IMPORTANTLY, if these uncommented lines are supposed to delete RAW recordings then why are they by default uncommented? I read in other post that by default no recordings will be deleted for self hosted BBB servers.

What will completely disable ALL automatic deletions of RAW recordings?

distancelearning.cloud

unread,
Sep 23, 2021, 10:08:53 AM9/23/21
to BigBlueButton-Setup
# remove_raw_of_published_recordings  <-- this was always commented out before, and it looks like the logic changed in that function since i last looked.  dealing with events.xml     before just deleted raw based on age..

so you might have found issue.  need to look at the commits to see when changed, i''ll do a test on 2.4

distancelearning.cloud

unread,
Sep 23, 2021, 12:24:28 PM9/23/21
to BigBlueButton-Setup
2.3.3  is by default keeping just the last 14 days of archives,  the function in not commented, and using published-days set at beginning of file.  to 14

G H

unread,
Sep 24, 2021, 6:02:28 AM9/24/21
to BigBlueButton-Setup
Thanks for quick comments. Good that we found what was wrong so we can get that addressed.

So you mean as a workaround all I have to do is just comment out the remove_raw_of_published_recordings line and that should not remove the RAW recordings ? or there is some other logic applied somewhere else too? About 8 years ago I think it was not about changing such script but just adding or removing crontab entry that was removing or avoiding removal of RAW recordings.

Thanks again.

sd...@distancelearning.cloud

unread,
Sep 24, 2021, 6:31:56 AM9/24/21
to bigbluebu...@googlegroups.com

Should be able to just # out that, and the raws will remain forever. 

 

Regards,

Stephen

--
You received this message because you are subscribed to the Google Groups "BigBlueButton-Setup" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bigbluebutton-s...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bigbluebutton-setup/7b3e4b40-1572-44ba-9f56-8567b4bc31ean%40googlegroups.com.

G H

unread,
Sep 24, 2021, 6:41:08 AM9/24/21
to BigBlueButton-Setup
Perfect. Thanks again.

Appreciate it.

G H

unread,
Aug 18, 2022, 5:29:39 AM8/18/22
to BigBlueButton-Setup
Hello Team,

Just to inform you so that our community is not affected by this.

I did fresh installation of 2.5.4 on new server, copied all the recordings. 
Next day I remembered in past I had lost a lot of raw files (too much painful as I can't convert those recordings to MP4 anymore due to no raw files for a lot of recordings which were deleted).

When I go to  /etc/cron.daily/bigbluebutton then again the,

remove_raw_of_published_recordings

is no longer commented. This means it will be executed and because of,

published_days = 14

raw recordings of all published recordings older than last 14 days will be deleted.

Or is this how it is supposed to be and all raw recordings by default must be removed which are older than 14 days?

Jean Pluzo

unread,
Aug 18, 2022, 8:24:25 AM8/18/22
to BigBlueButton-Setup
Hi,

This means, if you want to keep the raw data or any other files, you should either comment the remove_raw_of_published_recordings function in the /etc/cron.daily/bigbluebutton or change the published_days variables to a longer period of time (30 days, which is what I did).

Regards,

J.

Pablo - BBBPlugin.com

unread,
Aug 18, 2022, 10:57:10 AM8/18/22
to BigBlueButton-Setup
You know that you really do not need the raw files unless you need to rebuild the recording, right?
In other words, recordings that have already been processed (published or unpblished) no longer use those raw files.
For most people it is alright to have the raw files deleted after 14 days or something like that.
Reply all
Reply to author
Forward
0 new messages