Upload Published Recording to S3 using Post Script [post_publish is not behaving as of documentation]

[Kernel Subbiah]

I am trying to upload the published playback files to s3.

I have added a post script in /usr/local/bigbluebutton/core/scripts/post_publish as follows

=========

#

#POST PUBLISH SCRIPT

#

require "trollop"

require File.expand_path('../../../lib/recordandplayback', __FILE__)

opts = Trollop::options do

  opt :meeting_id, "Meeting id to archive", :type => String

end

meeting_id = opts[:meeting_id]

logger = Logger.new("/var/log/bigbluebutton/post_publish.log", 'weekly' )

logger.level = Logger::INFO

BigBlueButton.logger = logger

published_files = "/var/bigbluebutton/published/presentation/#{meeting_id}"

meeting_metadata = BigBlueButton::Events.get_meeting_metadata("/var/bigbluebutton/recording/raw/#{meeting_id}/events.xml")

#

# BELOW IS THE SCRIPT USED FOR UPLOADING 

# MEETING FILES TO S3

#

BigBlueButton.logger.info("S3 Uploader Running.")

meeting_id = "#{meeting_id}"

aws_s3_cmd = "aws s3 sync " + published_files + " s3://some-s3-bucket/" + meeting_id

BigBlueButton.logger.info(aws_s3_cmd)

status = system(aws_s3_cmd)

BigBlueButton.logger.info(status)

BigBlueButton.logger.info("End Script")

exit 0

=========

aws_s3_cmd = "aws s3 sync " + published_files + " s3://some-s3-bucket/" + meeting_id

I tried the above code with sudo included

aws_s3_cmd = "sudo aws s3 sync " + published_files + " s3://some-s3-bucket/" + meeting_id

I found the below info in the /var/log/bigbluebutton/post_publish.log file

I, INFO -- : false

I, INFO -- : #<Process::Status: pid 23915 exit 1>

I, INFO -- : 1

This means the ruby system() call is returning the error. 

However the aws sync fails. I see that when the script is being processed, the /var/bigbluebutton/published/presentation/#{meeting_id} is not present. Once the script has executed the files exists and if I run the sync then it works.

aws s3 sync /var/bigbluebutton/published/presentation/{meeting_id} s3://my-bucket/{meeting_id}

When I tried afterwards with the above command S3 is getting synced

Is this the expected behaviour, from the documentation I feel that the files would be available when post_publish script is called.

Is there any other way to do the same?

Also,

I thought that there may be a delay in copying the files to published/presentation by the corresponding process before the post_publish.rb is called, So I added

sleep 300 #seconds

before executing the aws_s3_cmd(refer code) so that it may delay the execution before the actual files are being created/copied in the said location.

But it is of no use. Any idea. Help Please

Note:

I think the thread/process associated to copying/creating published meeting files is not finishing the job until the whole script is executed, so that this issue prevailing. If this is the case how can we overcome this.

[Kernel Subbiah]

FOUND A SOLUTION

# /usr/local/bigbluebutton/core/scripts/post_publish/post_publish.rb

#!/usr/bin/ruby

# encoding: UTF-8

require "trollop"

require 'aws-sdk-s3'

require File.expand_path('../../../lib/recordandplayback', __FILE__)

opts = Trollop::options do

opt :meeting_id, "Meeting id to archive", :type => String

opt :format, "Playback format name", :type => String

end

meeting_id = opts[:meeting_id]

logger = Logger.new("/var/log/bigbluebutton/post_publish.log", 'weekly' )

logger.level = Logger::INFO

BigBlueButton.logger = logger

published_files = "/var/bigbluebutton/published/presentation/#{meeting_id}"

meeting_metadata = BigBlueButton::Events.get_meeting_metadata("/var/bigbluebutton/recording/raw/#{meeting_id}/events.xml")

#

# S3 Upload Code

#

def object_uploaded?(s3_client, bucket_name, file, object_key)

if File.directory?(file)

response = s3_client.put_object({

bucket: bucket_name,

key: object_key,

})

else

data = File.open(file)

response = s3_client.put_object({

bucket: bucket_name,

key: object_key,

body: data

})

data.close

end

if response.etag

return true

else

return false

end

rescue StandardError => e

puts "Error uploading object: #{e.message}"

return false

end

bucket_name = 'BUCKET-NAME'

region = 'REGION'

s3_client = Aws::S3::Client.new(

region: region,

credentials: Aws::Credentials.new('AWS-CLIENT-ID','AWS-CLIENT-SECRET')

)

BigBlueButton.logger.info("S3 Uploader Running for meeting #{meeting_id}")

Dir.glob("#{published_files}/**/*") {|file|

tmp = file

bucket_file = tmp.gsub("/var/bigbluebutton/published/presentation/","")

puts "File => #{file}"

if object_uploaded?(s3_client, bucket_name, file, bucket_file)

BigBlueButton.logger.info("#{bucket_file} uploaded successfully!")

else

BigBlueButton.logger.info("#{bucket_file} upload failed!")

end

}

BigBlueButton.logger.info("S3 Upload completed for meeting #{meeting_id}")

exit 0

[İsmail Erden]

Hello, thank you. Is it working properly?

8 Haziran 2021 Salı tarihinde saat 23:11:07 UTC+3 itibarıyla Kernel Subbiah şunları yazdı:

[sd...@distancelearning.cloud]

This is a post from 2021, if you try to run it,  remember you need to install the required ruby gems before running.

 

require "trollop"

require 'aws-sdk-s3'

 

regards,

Stephen

--
You received this message because you are subscribed to the Google Groups "BigBlueButton-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bigbluebutton-...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bigbluebutton-dev/8eb72173-fd81-4889-8993-88cd28699470n%40googlegroups.com.