Recordings not Displaying in Moodle and Greenlight
347 views
Skip to first unread message
David Heuring
Mar 20, 2020, 7:49:54 AM3/20/20
to BigBlueButton-dev
Recordings were showing up normally in both Moodle and Greenlight until yesterday. Now short or long ones are not displaying. Where would I check on the server to see what is preventing them from showing. Thanks for any help. It's a bit of an urgent matters because of the current COVID-19 situation. Thanks!
Pedro Beschorner Marin
Mar 20, 2020, 11:51:43 AM3/20/20
to BigBlueButton-dev
You can search for .fail tags at /var/bigbluebutton/recording/status. If thats the case, go after logs for the failed recordings at /var/log/bigbluebutton. But due to heavy service usage the recording process may take some extra time to finish
David Heuring
Mar 20, 2020, 9:19:01 PM3/20/20
to BigBlueButton-dev
Thanks, I checked the status file but there is a blank screen.
When I check bbb-record --watch the recordings are not being processed. I also see "Condition failed" in the following:
bbb-rap-archive-worker.service - BigBlueButton recording and playback archive worker
Loaded: loaded (/usr/lib/systemd/system/bbb-rap-archive-worker.service; static; vendor preset: enabled)
Active: inactive (dead) since Fri 2020-03-20 19:06:11 EDT; 1h 34min ago
Condition: start condition failed at Fri 2020-03-20 20:40:01 EDT; 10s ago
ConditionPathExistsGlob=/var/bigbluebutton/recording/status/recorded/*.done was not met
Process: 30415 ExecStart=/usr/local/bigbluebutton/core/scripts/rap-archive-worker.rb (code=exited, status=0/SUCCESS)
Main PID: 30415 (code=exited, status=0/SUCCESS)
● bbb-rap-sanity-worker.service - BigBlueButton recording and playback sanity check worker
Loaded: loaded (/usr/lib/systemd/system/bbb-rap-sanity-worker.service; static; vendor preset: enabled)
Active: inactive (dead) since Fri 2020-03-20 14:23:48 EDT; 6h ago
Condition: start condition failed at Fri 2020-03-20 20:40:01 EDT; 10s ago
ConditionPathExistsGlob=/var/bigbluebutton/recording/status/archived/*.done was not met
Process: 15223 ExecStart=/usr/local/bigbluebutton/core/scripts/rap-sanity-worker.rb (code=exited, status=0/SUCCESS)
Main PID: 15223 (code=exited, status=0/SUCCESS)
● bbb-rap-publish-worker.service - BigBlueButton recording and playback publish worker
Loaded: loaded (/usr/lib/systemd/system/bbb-rap-publish-worker.service; static; vendor preset: enabled)
Active: inactive (dead) since Fri 2020-03-20 20:23:16 EDT; 16min ago
Condition: start condition failed at Fri 2020-03-20 20:40:01 EDT; 10s ago
ConditionPathExistsGlob=/var/bigbluebutton/recording/status/processed/*.done was not met
Process: 570 ExecStart=/usr/local/bigbluebutton/core/scripts/rap-publish-worker.rb (code=exited, status=0/SUCCESS)
Main PID: 570 (code=exited, status=0/SUCCESS)
Any idea what is wrong? I have BBB 2.2 installed on Unbuntu 16.04. We've had a huge increase in usage lately. Could this be an insufficient server capability or is it something internal with the BBB set up. Please advise.
David Heuring
Mar 21, 2020, 4:05:43 AM3/21/20
to BigBlueButton-dev
In an effort to solve why BBB stopped processing recordings. I've provided the output from bbb-record --list, bbb-conf --check, and dpkg -l | grep bbb. The problem seems to be related to the line below - ConditionPathExistsGlob=/var/bigbluebutton/recording/status/recorded/*.done was not met.
Any help would be highly appreciated.
David
bbb-conf --list
● bbb-rap-archive-worker.service - BigBlueButton recording and playback archive worker
Loaded: loaded (/usr/lib/systemd/system/bbb-rap-archive-worker.service; static; vendor preset: enabled)
Active: inactive (dead) since Sat 2020-03-21 03:47:13 EDT; 8min ago
Condition: start condition failed at Sat 2020-03-21 03:55:09 EDT; 4s ago
ConditionPathExistsGlob=/var/bigbluebutton/recording/status/recorded/*.done was not met
Process: 6789 ExecStart=/usr/local/bigbluebutton/core/scripts/rap-archive-worker.rb (code=exited, status=0/SUCCESS)
Main PID: 6789 (code=exited, status=0/SUCCESS)
Mar 21 03:47:12 ns1 systemd[1]: Started BigBlueButton recording and playback archive worker.
● bbb-rap-sanity-worker.service - BigBlueButton recording and playback sanity check worker
Loaded: loaded (/usr/lib/systemd/system/bbb-rap-sanity-worker.service; static; vendor preset: enabled)
Active: inactive (dead) since Fri 2020-03-20 14:23:48 EDT; 13h ago
Condition: start condition failed at Sat 2020-03-21 03:55:09 EDT; 4s ago
ConditionPathExistsGlob=/var/bigbluebutton/recording/status/archived/*.done was not met
Process: 15223 ExecStart=/usr/local/bigbluebutton/core/scripts/rap-sanity-worker.rb (code=exited, status=0/SUCCESS)
Main PID: 15223 (code=exited, status=0/SUCCESS)
Mar 20 14:23:47 ns1 systemd[1]: Started BigBlueButton recording and playback sanity check worker.
● bbb-rap-process-worker.service - BigBlueButton recording and playback process worker
Loaded: loaded (/usr/lib/systemd/system/bbb-rap-process-worker.service; static; vendor preset: enabled)
Active: inactive (dead) since Sat 2020-03-21 00:36:26 EDT; 3h 18min ago
Condition: start condition failed at Sat 2020-03-21 03:55:09 EDT; 4s ago
ConditionPathExistsGlob=/var/bigbluebutton/recording/status/sanity/*.done was not met
Process: 28283 ExecStart=/usr/local/bigbluebutton/core/scripts/rap-process-worker.rb (code=exited, status=0/SUCCESS)
Main PID: 28283 (code=exited, status=0/SUCCESS)
Mar 20 21:20:01 ns1 systemd[1]: Started BigBlueButton recording and playback process worker.
● bbb-rap-publish-worker.service - BigBlueButton recording and playback publish worker
Loaded: loaded (/usr/lib/systemd/system/bbb-rap-publish-worker.service; static; vendor preset: enabled)
Active: inactive (dead) since Sat 2020-03-21 00:36:31 EDT; 3h 18min ago
Condition: start condition failed at Sat 2020-03-21 03:55:09 EDT; 4s ago
ConditionPathExistsGlob=/var/bigbluebutton/recording/status/processed/*.done was not met
Process: 4513 ExecStart=/usr/local/bigbluebutton/core/scripts/rap-publish-worker.rb (code=exited, status=0/SUCCESS)
Main PID: 4513 (code=exited, status=0/SUCCESS)
Mar 21 00:36:26 ns1 systemd[1]: Started BigBlueButton recording and playback publish worker.
---------------------------------------------------------------------------------------
bbb-conf --check
BigBlueButton Server 2.2.2 (1786)
Kernel version: 5.3.11-x86_64-linode131
Distribution: Ubuntu 16.04.6 LTS (64-bit)
Memory: 16380 MB
CPU cores: 6
/usr/share/bbb-web/WEB-INF/classes/bigbluebutton.properties (bbb-web)
bigbluebutton.web.serverURL: https://bbb.xxxxxxxxxxxxxxx.org
defaultGuestPolicy: ALWAYS_ACCEPT
svgImagesRequired: true
/etc/nginx/sites-available/bigbluebutton (nginx)
server name: bbb.xxxxxxxxxxxxxxx.org
port: 80, [::]:80
port: 443 ssl
bbb-client dir: /var/www/bigbluebutton
/var/www/bigbluebutton/client/conf/config.xml (bbb-client)
Port test (tunnel): rtmp://bbb.xxxxxxxxxxxxxxx.org
red5: bbb.xxxxxxxxxxxxxxx.org
useWebrtcIfAvailable: true
/opt/freeswitch/etc/freeswitch/vars.xml (FreeSWITCH)
local_ip_v4: xxxxxxxxxxxxxxx
external_rtp_ip: stun:stun.freeswitch.org
external_sip_ip: stun:stun.freeswitch.org
/opt/freeswitch/etc/freeswitch/sip_profiles/external.xml (FreeSWITCH)
ext-rtp-ip: $${local_ip_v4}
ext-sip-ip: $${local_ip_v4}
ws-binding: :5066
wss-binding: :7443
/usr/local/bigbluebutton/core/scripts/bigbluebutton.yml (record and playback)
playback_host: bbb.xxxxxxxxxxxxxxx.org
playback_protocol: https
ffmpeg: 4.2.2-0york0~16.04
/etc/bigbluebutton/nginx/sip.nginx (sip.nginx)
proxy_pass: xxxxxxxxxxxxxxx
/usr/local/bigbluebutton/bbb-webrtc-sfu/config/default.yml (Kurento SFU)
kurento.ip: xxxxxxxxxxxxxxx
kurento.url: ws://127.0.0.1:8888/kurento
localIpAddress: xxxxxxxxxxxxxxx
recordScreenSharing: true
recordWebcams: true
codec_video_main: VP8
codec_video_content: VP8
/usr/share/meteor/bundle/programs/server/assets/app/config/settings.yml (HTML5 client)
build: 862
kurentoUrl: wss://bbb.xxxxxxxxxxxxxxx.org/bbb-webrtc-sfu
enableListenOnly: true
# Potential problems described below
-------------------------------------------------------------------------------------
dpkg -l | grep bbb
ii bbb-apps 1:2.2.0-39 amd64 BigBlueButton applications for Red5
ii bbb-apps-akka 2.2.0-73 all BigBlueButton Apps (Akka)
ii bbb-apps-screenshare 1:2.2.0-39 amd64 BigBlueButton screenshare module for Red5
ii bbb-apps-sip 1:2.2.0-11 amd64 BigBlueButton SIP module for Red5
ii bbb-apps-video 1:2.2.0-36 amd64 BigBlueButton video module for Red5
ii bbb-apps-video-broadcast 1:2.2.0-8 amd64 BigBlueButton video broadcast module for Red5
ii bbb-client 1:2.2.0-33 amd64 BigBlueButton Flash client
ii bbb-config 1:2.2.0-160 amd64 BigBlueButton configuration utilities
ii bbb-etherpad 1:2.2.0-48 amd64 The EtherPad Lite components for BigBlueButton
ii bbb-freeswitch-core 2:2.2.0-90 amd64 BigBlueButton build of FreeSWITCH
ii bbb-freeswitch-sounds 1:1.6.7-6 amd64 FreeSWITCH Sounds
ii bbb-fsesl-akka 2.2.0-58 all BigBlueButton FS-ESL (Akka)
ii bbb-html5 1:2.2.0-862 amd64 The HTML5 components for BigBlueButton
ii bbb-mkclean 1:0.8.7-3 amd64 Clean and optimize Matroska and WebM files
ii bbb-playback-presentation 1:2.2.0-28 amd64 BigBluebutton playback of presentation
ii bbb-record-core 1:2.2.0-60 amd64 BigBlueButton record and playback
ii bbb-red5 1:1.0.10-16 amd64 The Red5 server for BigBlueButton
ii bbb-transcode-akka 2.2.0-8 all BigBlueButton Transcoder
ii bbb-web 1:2.2.0-114 amd64 BigBlueButton API
ii bbb-webhooks 1:2.2.0-16 amd64 BigBlueButton Webhooks
ii bbb-webrtc-sfu 1:2.2.0-89 amd64 BigBlueButton WebRTC SFU
ii bigbluebutton 1:2.2.0-5 amd64 Open source web conferencing platform (bbb)
David Heuring
Mar 21, 2020, 5:21:48 AM3/21/20
to BigBlueButton-dev
And here is the bbb-rap-worker file. I don't see any errors. We just had a huge increase in the use of BBB with users making a lot of recordings. Is the problem my server's processing capacity = 6 CPU, 320GB, 16GB RAM?
David Heuring
Mar 21, 2020, 8:17:28 PM3/21/20
to BigBlueButton-dev
Update. The recordings are eventually showing but sooooo slow. I'll increase the server's RAM and see if that helps. If anyone with expertise can take a look at what included in these posts to see if there is anything wrong, I'd really really appreciate it.
Chad Pilkey
Mar 23, 2020, 2:08:13 PM3/23/20
to BigBlueButton-dev
You can use top or atop to see resource usage on your server. That should show you if you're running out of RAM or if CPU is an issue.
The rap-X-workers function by looking for .done files to know that the previous step finished and that it needs to run its part. As long the start condition line says that it was tried recently (<1 min) they should be in working order.
To troubleshoot a recording you need to have a particular recording that you want to look into and then follow the steps here, http://docs.bigbluebutton.org/dev/recording.html#troubleshooting. It walks through all the steps to determine how far along the recording is in the processing steps and where to look if you think it's stuck or not processing.
Reply all
Reply to author
Forward
0 new messages