Opencast 13 Paella Player, Captions and timing

[steven cox]

Hey All, 

I'm testing Opencast 13 with transcriptions provided by Whisper before we move to production.

Using the information from here https://github.com/opencast/helper-scripts/tree/master/whisper-AI-exec-script, I have the captions added to the event, after the successful addition I republish the metadata, but I'm unable to see the captions.

Once the media has gone through Whisper and attached to the event, I check to see that the caption is present, it has the correct type (captions/VTT+en) the MIMEtype is (text/vtt) and it's tags are archive, engage-download  these are the same settings as in the production instance of Opencast 8.

Looking in the config.json for Paella player (not 7) all the necessary caption related commands appear to be set to true.

I was wondering if it was the way the captions are being added to the event, as when checking the production (OC 8.11) instance with captions attached an icon representing the activation of the caption is visible, however, when there are no captions, the button is not visible.  But, if I retract and republish, I can then see the subtitle button and activate the subtitles, but the subtitles are out of sync with the media ( the media is cut), so It's obviously a timing issue.

So I guess 

Q1. Is it possible to grab the edited media, extract the audio and send that to Whisper if so what source/flavor will need to be sent?

Q2. if Q1 isn't possible, is it possible to add the subtitle as a track that could be cut

or am I completely wrong.

Thanks in advance 

regards

Steve Cox 

UofT

[Stefanos Georgopoulos]

Hi Steve,

I had this problem and I asked about it in the Summit. We don't use any
engage server or paella for video/captions publishing. We have our own
video portal.

We produced closed captions with Whisper for all our videos and we have
students that need to correct these transcripts. You can enable the
option to upload a .vtt file in opencast configs, but in the admin UI it
is uploaded as attachment and therefore is not be used from new editor.
The editor needs the .vtt file to be tagged as a track.

I end up using the ingest script

https://github.com/opencast/helper-scripts/blob/master/ingest/ingest-addtrack.sh

I add the delivery video and the .vtt as tracks and started a new
workflow. That way our students can access the editor and edit the
closed captions file.

Hope that helps a little bit ;)

Best regards,
Stefanos

[steven cox]

Hey Stefanos, 

This gives me a start of where to look, thanks for the information.  

We are already using an adjusted version of the script you mentioned to import the schedules for the next days events. We use a presentation node, but only as a pass through so that Students can see their media in Canvas via LTI.

Currently we have 150+ Extrons (and rising) with the instructors opting in their course (assuming it's in a Extron enabled room) so any material produced is only done on the Extrons with the instructor deciding if they want an automatic/manual publishing option.  

My current plan is when the prof decides they want to publish their media, the Whisper generation process becomes part of the publishing process (whether automatic or manual) so I'm going to look at changing the helper script here  https://github.com/opencast/helper-scripts/tree/master/whisper-AI-exec-script so it adds as a track instead of an attachment (don't know how yet)

OC admins, Hope you don't mind but I'm testing the cutting of a track that contains a subtitle track to see the effect on stable.opencast.org

Again, thanks for the pointer

Regards

Steve

[steven cox]

Hey Stefanos,

So I edited the media https://stable.opencast.org/paella/ui/watch.html?id=3a07bd58-fcea-4d04-8a39-55c03a52e897, duplicating the media first, then editing out sections using the admin UI editor at 2-3 minutes and a subsequent edit at 3-4 minutes, unfortunately, the subtitles didn't cut.  So with regards the editor, which are you referring to, on our dev system we have access to the admin UI editor (as do authenticated accounts for publishing /editing )  admins can access the alternate editor by changing the url as necessary or globally if we make the appropriate adjustments to our config, unfortunately we are experiencing CORS errors so the video is not available on the admin node (another thing, we're trying to resolve)

Regards

Steve 

[Stefanos Georgopoulos]

Hi Steve,

glad to know that I helped you a little bit. As discussed in the Summit,
in the next versions of Opencast, there will be some changes making CCs
first class citizen in the project.

In our case, we don't have a lot of Videos to edit the subtitles, so the
basic functionality of the new editor seems to do the job for us.

Best regards,
Stefanos

[steven cox]

Hey Stefanos and all, 

Thanks for your earlier help, so we ended up using a slightly changed version of the Whisper workflow https://github.com/opencast/helper-scripts/tree/master/whisper-AI-exec-script, listening to other comments we changed the import of the VTT via the XML workflow to import the expected type of a track instead of an attachment.  

We did find that after the process was completed the icon for subtitles wasn't present so the subtitles were inaccessible, testing revealed that we had to retract and republish, which definitely wasn't what we wanted.  to get around this we added a publish engage operation - it still doesn't show the subtitles option initially, but if we republish metadata it works fine, so we'll be testing adding this in as well. 

We pulled the editor and are testing now https://github.com/opencast/opencast-editor, it works well when calling against develop.opencast.org, but we're now looking at transferring this to our dev environment for testing, so I'm wondering how I add it (specifically what and where) directly to the dev system?  We'd be looking at replacing the stand alone editor 

As always, thanks for your help

Regards

Steve

[steven cox]

Hey all, 

Whilst investigating how to integrate the https://github.com/opencast/opencast-editor, we thought we'd look into why the stand alone editor included with V13 would not load the video, when we attempt to open the editor via the scissors icon on the admin node, The Chrome console developer tools shows the message below

Mixed Content: The page at 'https://adminserver.opencast.utoronto.ca/editor-ui/index.html?mediaPackageId=8f59d9a4-357d-4b0a-b9e9-8e677bd851f9' was loaded over HTTPS, but requested an insecure manifest 'http://adminserver.opencast.utoronto.ca/admin-ng/login.html;jsessionid=node01txqeopkiso0q13cr41kobsbe913096.node0'. This request has been blocked; the content must be served over HTTPS.

If we change the URL to the presentation node 'http://presserver.opencast.utoronto.ca/admin-ng/login.html;jsessionid=node01txqeopkiso0q13cr41kobsbe913096.node0' the media loads

Just wondering if there's something we've missed in the Config to force the manifest request to HTTPS

Just in case this has a bearing on why this is happening we are using the following configuration

- Admin node -

Shibboleth Authentication for instructors/staff to admin interface

Apache Proxy

Allows staff (once granted access to their series) to edit/download/publish their media

  

- Presentation Node - 

Ngnix proxy

Serves media via LTI links to our LMS to LTI authenticated course members

Access to Engage UI only available to Admins

Regards

Steve

[Greg Logan]

Hey Steve,

I'm wondering if https://github.com/opencast/opencast/issues/3538#issuecomment-1502961057 is what you're looking for.  Seems like it was patched *after* the latest release, should hopefully (I think?) be fixed in the next 13 release.

G

--
To unsubscribe from this group and stop receiving emails from it, send an email to users+un...@opencast.org.

[steven cox]

Hey Greg, 

Thanks again for the pointer, we’ll look at this and attempt to change the directives for Apache.

We did subsequently notice that using the original or new editor interface as a standard user resulted in the inability to load the media because it’s asking for it from the presentation node.  The users who edit their media, all authenticate to the admin node and never actually touch the presentation node at all resulting in 403 errors. Grabbing the url for the media and changing the server name allows the media to be accessed.

So we’re left with a few options.

1. Stay on v.8 - not really an option for a number of reasons

2. Wait until a later version (poss 13.6?)

3. Make the necessary changes in Apache

4. Build and integrate a Shibboleth module into Nginx replace Apache on the Admin node - probably a good idea to do this anyway as all nodes will then be running on Nginx.

5. Build the new editor interface and replace it - we have a version running from GitHub, but don’t have a clue yet how to move it so it sits on the admin node

So many choices, so little time (the plan was moving through the versions next week)

Any further pointers, specifically on point 5 would be gratefully received

Regards

Steve

Transported by the Email trolls

On Apr 27, 2023, at 5:46 PM, Greg Logan <gregor...@gmail.com> wrote:



[steven cox]

Hey All,

Greg, in answer to your reply, didn't actually work for us as we're using Apache for the Shibboleth authentication to our back end authorization systems, in discussion with others there is hesitance to use Nginx and Shibboleth, so I guess we're waiting until 13.6 (does anyone know of a release date or some way we can beta test it?)

I did some more testing on Opencast 13.5, regardless of the use of internal or stand alone editors, we get the same issues.  Just so it's not necessary to search through the thread, this is our current config in dev

 

- Admin node -

Shibboleth Authentication for instructors/staff to admin interface

Apache Proxy

Allows staff (once granted access to their series) to edit/download/publish their media

  

- Presentation Node - 

Ngnix proxy

Serves media via LTI links to our LMS to LTI authenticated course members

Access to Engage UI only available to Admins

So when the instructor needs to publish their media, they access the admin interface and edit the media they want to edit/publish

The stand alone player is more verbose and after a bit of digging we found mention of a file called by the admin node during the edit process (edit.json - I know this is mentioned in multiple posts) - this file specifically mentions the presentation server is there somewhere we can force this to the admin node when the file is created?

So I looked at the TOML file and made changes to the one on the admin node, there is now mention of two settings keys - subtitles.icon and opencast.local which are ignored as they aren't present in another file called config.ts - so we need to move across the entire build.

We have a version of the Editor deployed via npm as a test, just wondering what we need to do to replace the current on dev and more specifically where?

Or do we wait, no problems trashing the dev system if someone can point us in the right direction

Regards

Steve

[Greg Logan]

Hey Steve,

You're in luck if you're feeling brave: We've got nightlies!  If you're looking for tarballs try https://s3.opencast.org/opencast-daily/opencast-dist-allinone-13-SNAPSHOT.tar.gz.  For Debs use [1].  Lars can probably chime in with RPMs, or I can build you one (but don't tell him ;)).

Matthias is also supposed to be back from holidays tomorrow, and mentioned cutting a release as soon as he's back.  Expect that in the various testing repos within a day or two.

G

1: https://ci.opencast.org/debian 13.x unstable, signed by https://ci.opencast.org/keys/public-signing-key.gpg

[steven cox]

Thanks Greg,

We can definitely wait a while, thanks for the offer, I won't tell Lars you offered

I'll keep a look out for the notification

Regards

Steve

[steven cox]

Hey Greg and All, 

We waited, installed OC13.6 and it's working, relieved that we don't have to mess around with Nginx and Shibboleth.

Thanks for all your hard work on this

Regards

Steve