UV and VTT video thumbnails and captions

[Michael Levy]

Is there support in Universal Viewer for displaying an mp4 video stream along with WebVTT to support time-based thumbnails and captions (and maybe chapters)?

[Michael Levy]

Since I wrote this I looked closer. It looks like UV is using MediaElement.js http://mediaelementjs.com and they do say they support WebVTT. The thumbnails example they have got show large slides displayed to the side of the video, rather than the common small thumbnails displayed over the slider rail as one moves the cursor along the rail.

Jason Ronallo has a project where he uses rail thumbnails and MediaElement.js

https://github.com/jronallo/mep-feature-time-rail-thumbnails

And this seems to be a good example of what we'd like to have, e.g.

http://d.lib.ncsu.edu/collections/catalog/ua024-002-bx0113-248-001

We are testing Universal Viewer as a way of displaying collections of resources including groups of archival page images and photographs. If you are looking for feature requests, it would be great to use UV for streaming media as well, but we'd need to have a way to support WebVTT files for both thumbnails and captions. 

[Tom Crane]

Hi Michael,

At present the UV just plays the video. However, the IIIF community are very interested in extending the model to support Audio, Video and 3D, and as the specification evolves the UV will begin to implement features like this in an interoperable way. The current UV implementation is an interim simple model until there is an agreed standard. 

Last week there was a workshop in London to lay the grounds for IIIF to extend beyond images, and a report from this will be published very soon. That details a large number of use cases including this one. I'll post the details here as soon as the first report has been assembled. 

With an agreed data model, UV can implement feature requests like this and ensure interoperability with others publishing manifests using the same model.

As an interim solution, you could modify the UV's use of mediaElement to render the WebVTT contents. In your manifest, you need a way of stating that the webVTT file is available using the current UV interim model. You could annotate the video with the webVTT file in the same way that this manifest has an annotation on the video that is a PDF transcript:

http://wellcomelibrary.org/iiif/b16748967/manifest

The "format" would be text/vtt instead of application/pdf

Tom