Older recordings not playing 404 not found error

[G H]

Hi,

I created new instance and installed fresh BBB server on it. Then transferred all the recordings in published and unpublished (also raw ones) and then tested few recordings which all worked. But now I realized that older recordings 2019 or earlier are not playing and gives the error of 404. What could be wrong?

Need quick help if there is anyway I can resolve this quickly.

There must be some setting somewhere that doesn't support older recordings with new BBB installs. Appreciate your comment. Let me know if you need any setting details to check to suggest solution.

Regards,
JG

[G H]

I think it is quite possible that the playback version 2.3 might not be able to play previous recordings.

Just to try, how can I change the playback version back to 2.2 ?

[G H]

No luck. I tried to change the playback versions from 2.3 -> 2.0 -> 0.9.0 -> 0,81 all the playback versions without luck. Recording playback still returns 404 Not Found.

Post copy of all the recordings I had double checked the sizes and the counts of the files in published, unpublished and raw folders and no issues were found.

Not sure why recordings are not found. I can see the folders are very much there in /var/bigbluebutton/published/presentation folder. Still all versions in playback URL not playing the recordings and rather complains about 404 Not Found.

[G H]

Ok found something.

So if I play with 0.9.0/playback.html?meetingid=<meeting id> then recording plays (at least I successfully tried one, so should also work for others.)

Question is why 2.3 is forced onto these recording playbacks then?
And how to fix this. Surprisingly, these recordings worked fine with 2.2 BBB means I think 2.0 playback versions. Not able to successfully test on 2.0 with or without playback.html in URL.

[Ali Alhaidary]

what metadata.xml file say about these recordings?

Ali

--
You received this message because you are subscribed to the Google Groups "BigBlueButton-Setup" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bigbluebutton-s...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/bigbluebutton-setup/3db0b1f5-e8f9-45c5-ad82-d3d107d77147n%40googlegroups.com.

[G H]

Found metadata.xml that has playback URL in it. But with these old recordings the playback URL is showing 2.0 playback version. But I have tried that URL and also other versions. Only 0.9.0 playback version works.
Now. Because this 2.0 playback must have worked in my BBB 2.2 server, I think this bug has been introduced in BBB 2.3 server. Something is not right. If metadata -> presentation playback URL worked with BBB 2.2 with playback 2.0 then with BBB 2.3 and playback 2.0 it should have worked. But it doesn't seem to work.

any idea?

[G H]

Ah thanks Ali. In Metadata the playback URL version is 2.0.

But with BBB 2.3 latest build when I try to use exactly same playback URL with 2.0 playback version, then somehow that URL changes to 2.3 (looks like this is the issue). And that means I can't play with playback 2.0 anymore with new 2.3 BBB server build. Which is what the issue is I think.

[G H]

For a workaround I am thinking to try changing 2.0 to 0.9.0 in metadata of the affected recordings so that at least users can start watching the recordings.
Will let you know how it goes.

But permanent solution with a bit deeper insights for what was the intended outcome and where what went wrong for me so that I can apply permanent forward looking solution rather than backwards looking solution.

[G H]

That's it.

When I change the metadata's playback URL to 0.9.0 from 2.0 playback version then it works fine. This means I need to manually go and update those all affected. Wish there was a better way to resolve this.

What if I rebuild such recording. Would that bring the playback content compatible to 2.3 ?

[Ali Alhaidary]

We had a similar issue months back an wrote a script to change the content of metadat.xml in one go, look for it, it might help...

Ali

--

You received this message because you are subscribed to the Google Groups "BigBlueButton-Setup" group.
To unsubscribe from this group and stop receiving emails from it, send an email to bigbluebutton-s...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/bigbluebutton-setup/3ba59db5-ad8c-45d9-83f0-b143ff51ed2an%40googlegroups.com.

[Pedro Beschorner Marin]

The new playback compatibility with really old recordings like 0.81 and 0.9.0 should be considered experimental. 2.0 and higher should work out of the box.

Could you enable the player debug mode (https://github.com/bigbluebutton/bbb-playback#url-query-strings) and send the logs from your browser console?

[G H]

Thanks Ali. Will look for it.

[G H]

Thanks Pedro.

Yeah makes sense. I rebuilt the recording to experiment and after rebuilt, it didn't play in any version at all. I tried to play it in 0.9.0, 2.0, 2.3 and didn't with any of them. I am copying original files back now to fix that particular recording.

So the only solution that will work will be to change the playback version in url to 0.9.0.

Overall I think it is ok if old recordings don't work with playback 2.3. But they should have worked with 2.0 and they worked well before the upgrade with playback 2.0.
Now I am setting them to play on 0.9.0 losing all the improvements from 0.9.0 playback to 2.0 playback. This shouldn't have been necessary. I still think there is more to it than what was intended.

Where can I find the original version of the recordings to find out whether they are compatible with 2.3 or not based on your guidance above. ( I mean for the not so old recordings that I have ).

Thanks again for quick comment.

[G H]

ok so upgrade to 2.3 has been a total disaster for me to be honest.

I could have applied workaround, but there are some external meeting ids that I can see in my website Database, I can see them on front end page behind the button (I clearly see same external id when I inspect the page)  that is going to pull the recording streaming. But when I go check bbb-conf --tointernal 340 then I get no internal meeting ids for all such problematic recordings. And this behaviour is same on both BBB setups. i.e. older BBB 2.2.35 and newer BBB 2.3.9. Mericulously recording plays accurately and fine on BBB 2.2.35, but because of the issue now I have with these recordings on BBB 2.3.9 I can't find the internal meeting id and hence I can't fix those recordings. Unless if there is anyway from logs etc... I can play it on BBB 2.2.35 and find out the internal meeting id and then fix that on BBB 2.3.9. If someone can help with this, then I can still stay on current new BBB 2.3.9. Else I don't have any choice but to go back if it is possible, due to impact on users but to go back.

I have changed the hosting provider which means if I go back, then I will have to setup new instance on this new hosting provider.
Regarding this going back to BBB 2.2.35, my next question is, if I setup Ubuntu 16.04.x then through bbb_install.sh script, can I install just BBB 2.2.35 and not BBB 2.3.9 ?

Thanks in advance.

[G H]

Hi again,

Although bbb-record --tointernal <externam meeting id> command doesn't find the internal meeting id, with some linux commands I could find those on BBB 2.3.9. Which means now hopefully I don't need to go back at least.

It would have been better that playback 2.0 was still used for all those recordings playback where 2.0 was able to handle them.

If this helps someone, the file /var/log/bigbluebutton/bbb-web.log basically captures log of which external meeting id on which the request comes along with it's checksum and then gets the controller number, which so far I have found unique to a meeting may be. I also noticed that this controller is first part of the meeting id in the format of <first part>-<second part>. This means if you are not able to find internal meeting id for any recording for which bbb-record --tointernal command doesn't work then you can try to access that recording with external id that you know and then check this log file to get controller, then find the internal meeting id through grep -Ril or some other quicker commands like find.

Thanks again Ali and Pedro. Appreciate your comments.

[G H]

Hi Pedro,

You said 2.0 and above should work out of the box. But in my case I never rebuilt the recordings nor did I modify any metadata.xml files. I mean all metadata.xml files I have been finding in trouble and not playing on 2.3.9 BBB are "2.0". Do you know why I am getting 404 Not Found for the published recording ready to be played back and marked as 2.0 (which as I understood must have been recorded when playback 2.0 would have already been available) ?
The reason I am asking this is because I would like them to be played by 2.3 player rather then 0.9.0 for all the improvements of multiple speeds , captions etc....

Appreciate if you can help with your comments.

[Ali Alhaidary]

You are more than welcome, is your problem solved ?

Ali

To view this discussion on the web visit https://groups.google.com/d/msgid/bigbluebutton-setup/5fa4fedb-b391-4bf1-966a-7b6848bad50en%40googlegroups.com.

[G H]

Yes thanks Ali. Problem is solved by workaround of changing the playback version to 0.9.0 which was 2.0 earlier. And as I have understood in 2.3.9 BBB the 2.0 playback is forced to play on 2.3, which ends up being 404 not found even though the presentation is there to play.

Permanent solution would have been if 2.0 playback recordings were able to be played on 2.3 playback mechanism.

Thanks again Ali.

By the way, I searched for that script but couldn't find it. Do you have a link that I can jump and check? Actually linux also has mass replacement commands that can also do things in quick and dirty way. Off course a script can do more sanity checks and be more robust. I may need to do something if get some positive way out of this.

[Ali Alhaidary]

I think it was something like this:

find . -name 'metadata.xml' -exec sed -i 's/bbb.the/almajlis.the/g' {} +

where I wanted to replace 'bbb.the' with 'almajlis.the' of all metadata.xml in one of the version upgrades that bbb-conf -setip did not do the job when I changed the host name and did fresh installation, and recordings did not play.

Ali

To view this discussion on the web visit https://groups.google.com/d/msgid/bigbluebutton-setup/9d456770-c2cc-41af-9250-d66a4723471cn%40googlegroups.com.

[G H]

Great, thanks Ali.

Will use that based on what happens with all recordings marked as 2.0 but not working with 2.3.

Appreciate your quick comments.

[Ali Alhaidary]

More than welcome, however, I assume you know that you will need to verify it on sample file, and have your backup nearby :-)

Ali

To view this discussion on the web visit https://groups.google.com/d/msgid/bigbluebutton-setup/811eba02-5682-4d22-ad34-4b0dff3dff84n%40googlegroups.com.

[G H]

Got that. This is the reason why I didn't apply quick mass replace command so far.
Thanks Ali.

[JarrodJay]

I found this issue a little while ago, and believe it was because the old recordings were missing something in their metadata. I believe this fixed it- https://groups.google.com/g/bigbluebutton-dev/c/UPNN2emwcR8/m/iGF2DWKSCgAJ 

[G H]

Thanks Jarrod. Appreciate your comments and extending help.

I had fixed the recordings by manually editing the metadata file of those recordings. Although old recordings stopped working again. Not sure what is going on.
I will run one script for 1.1 version recordings and see if it works. If it does, will try 0.9, 0.8 and 1.0 also. See how it goes.

Thanks again.

[G H]

Hi Jarrod,
I run that script, but that didn't make any difference.

In summary what happened is that after fresh installation of BBB 2.3 and migrating recordings from BBB 2.2 server, a lot of recordings stopped working. Through some checks I found that metadata.xml of these recordings was set to "playback/2.0/..." and as soon as I changed that to "playback/0.9.0/" then recordings started working. To me it looks like those recordings processed on BBB 2.0 didn't work on BBB 2.3 and only way they all worked for me was by changing metadata.xml for the downgrade to 0.9.0 playback.

Two basic areas of improvement here, that I can objectively think of during any upgrade BBB or not,
- Maintain database of all settings and separate that completely from software coding upgrades
- Maintain backwards compatibility with a clear and reasonable roadmap plan of any affected functionalities or cessations or phase-outs

Anyway, if you can think of anyway I can still upgrade those recordings back to 2.0 or even better to 2.3 playback then let me know. I had tried rebuilding one of the recordings right after migration a month  or two ago, because recordings stopped working. But after rebuild the recording stopped working completely. That is why I haven't given thought of rebuilding all recordings as that would be bigger blunder.

I will create another post in users group, but I noticed that earlier "Automatic Raw Files Deletion" was not by default enabled. I noticed that two of my RAW presentation files after upgrade just disappeared. I am yet to find facts about it, but critical such as this, I hope it is not by default set for "Automatic Raw Files Deletion" post whatever number of days.

[brainwhistler Whistler]

I also had the fresh installation. Migrated all the recording from v2.2 to v2.3 by following the document. https://docs.bigbluebutton.org/admin/customize.html#transfer-published-recordings-from-another-server

Then run the command "sudo bbb-conf --setip bigbluebutton.example.com". Domain name was replaced with the actual FQDN.

After that I found some recordings were present and some where not. Few classes were have all the recordings, and few does not have at all. And few classes have mix, i.e. some recording present and some are missing. The issue was found that the command 'sudo bbb-conf --setip bigbluebutton.example.com' did not update the FQDN in the metadata.xml file. Which I had to update by running the script.

Once FQDN was updated in the metadata file all recordings started running from the new server.

I think the command "sudo bbb-conf --setip bigbluebutton.example.com" needs more optimization as it missed a huge amount of recordings.

Thought sharing this experience useful for others.

Thanks