Sign-language support in edX video player [New Feature]

[Omar Al-Ithawi]

Hi All,

Just wanted to notify you that I'm going to work adding sign-language video support to edX video player,

I'm going to add a new xfield for example:

 

   youtube_id_1_5 = String(
        help=_("..."),
        display_name=_("Sign-language alternative video"),
        scope=Scope.settings,
        default=""
    )

And the UI will have an extra button to toggle this video.

I haven't started working on it yet, I'm planning to work on it starting from Feb 2nd.

Your feedback is appreciated,

Thanks,

[Mark Sadecki]

Hi Omar,

Thank you for considering this work!  Having a sign language option would be a wonderful addition to the edX media player.  I have some thoughts on this, particularly on your approach, that I would like to share:

Ideally, You wouldn't want to have a separate video for a sign language video.  There is often important visual context in the primary media that would be useful to the viewer.  HTML5 video players can be styled using CSS, which permits a whole range of possibilities for including sign language video.  I think a better approach would be to display the sign language video as a smaller video player (perhaps 20% of the viewable player area) with a higher Z-index so that it appears as an overlay on the primary media.  This is a standard paradigm often referred to as Picture-in-Picture or PiP.  This allows the viewer to see the primary media as well as the sign language media simultaneously.  The HTML5 media-controller can be used to synchronize the playback of both videos:
<http://www.w3.org/TR/html5/embedded-content-0.html#synchronising-multiple-media-elements>

Theoretically, the approach you outline below will be possible after our next release which introduces the "Cohorted Courseware" feature.  A course author can create a cohort of hearing-impaired users, or users who prefer sign language video.  When they author their course, they would add both videos, the primary media, viewable by everyone, and the sign language video, viewable by those in the cohort.  I think the one outlined above would be more elegant.

Thoughts?

Mark

[Omar Al-Ithawi]

> Ideally, You wouldn't want to have a separate video for a sign language video.  There is often important visual context in the primary media that would be useful to the viewer.  HTML5 video players can be styled using CSS, which permits a whole range of possibilities for including sign language video.  I think a better approach would be to display the sign language video as a smaller video player (perhaps 20% of the viewable player area) with a higher Z-index so that it appears as an overlay on the primary media.

We thought about this, but I'm not a big fan of synchronizing things :). Initially we will provide two videos, one without sign language and the other is the exact same video but with the 20% sign-language video burned on. I believe this approach is:

• Less error-prone
• Easy for the course manager to manage
  
• Can be easily uploaded to YouTube and/or S3 without special hacks

> Theoretically, the approach you outline below will be possible after our next release which introduces the "Cohorted Courseware" feature.  A course author can create a cohort of hearing-impaired users, or users who prefer sign language video.  When they author their course, they would add both videos, the primary media, viewable by everyone, and the sign language video, viewable by those in the cohort.

It is up to the course managers to decide on using the cohorts to confine the sign video or not,

Hopefully mid Feb, we'll see an actual working prototype of the video and collect some feedback,

Thanks,

[Mark Sadecki]

On Thu, Jan 22, 2015 at 3:13 AM, Omar Al-Ithawi <oit...@qrf.org> wrote:

> Ideally, You wouldn't want to have a separate video for a sign language video.  There is often important visual context in the primary media that would be useful to the viewer.  HTML5 video players can be styled using CSS, which permits a whole range of possibilities for including sign language video.  I think a better approach would be to display the sign language video as a smaller video player (perhaps 20% of the viewable player area) with a higher Z-index so that it appears as an overlay on the primary media.

We thought about this, but I'm not a big fan of synchronizing things :). Initially we will provide two videos, one without sign language and the other is the exact same video but with the 20% sign-language video burned on. I believe this approach is:

• Less error-prone
• Easy for the course manager to manage
  
• Can be easily uploaded to YouTube and/or S3 without special hacks

Agreed.

> Theoretically, the approach you outline below will be possible after our next release which introduces the "Cohorted Courseware" feature.  A course author can create a cohort of hearing-impaired users, or users who prefer sign language video.  When they author their course, they would add both videos, the primary media, viewable by everyone, and the sign language video, viewable by those in the cohort.

It is up to the course managers to decide on using the cohorts to confine the sign video or not,

Hopefully mid Feb, we'll see an actual working prototype of the video and collect some feedback,

Would you consider expanding your solution to incorporate multiple video options (not just sign language)?  This approach could be used by a user to switch to a video with an Audio Description track or a video that has been dubbed into another language.

Best,

Mark

[Ned Batchelder]

Omar, thanks for the work, and especially for bringing the idea here to flesh out first.  More below,

On Thu, Jan 22, 2015 at 9:32 AM, Mark Sadecki <msad...@edx.org> wrote:

On Thu, Jan 22, 2015 at 3:13 AM, Omar Al-Ithawi <oit...@qrf.org> wrote:

> Ideally, You wouldn't want to have a separate video for a sign language video.  There is often important visual context in the primary media that would be useful to the viewer.  HTML5 video players can be styled using CSS, which permits a whole range of possibilities for including sign language video.  I think a better approach would be to display the sign language video as a smaller video player (perhaps 20% of the viewable player area) with a higher Z-index so that it appears as an overlay on the primary media.

We thought about this, but I'm not a big fan of synchronizing things :). Initially we will provide two videos, one without sign language and the other is the exact same video but with the 20% sign-language video burned on. I believe this approach is:

• Less error-prone
• Easy for the course manager to manage
  
• Can be easily uploaded to YouTube and/or S3 without special hacks

Agreed.

> Theoretically, the approach you outline below will be possible after our next release which introduces the "Cohorted Courseware" feature.  A course author can create a cohort of hearing-impaired users, or users who prefer sign language video.  When they author their course, they would add both videos, the primary media, viewable by everyone, and the sign language video, viewable by those in the cohort.

It is up to the course managers to decide on using the cohorts to confine the sign video or not,

Hopefully mid Feb, we'll see an actual working prototype of the video and collect some feedback,

Would you consider expanding your solution to incorporate multiple video options (not just sign language)?  This approach could be used by a user to switch to a video with an Audio Description track or a video that has been dubbed into another language.

​Multiple video options seems like a more flexible and powerful idea than a field specifically for sign language.  I'm sure if we scratch the surface, we'll come up with a dozen reasons people might need alternate videos. 

--Ned.

[Omar Al-Ithawi]

Thanks for the suggestions,

You're both right, about the multiple videos feature, our focus (at Edraak) on sign-language.

If you can elaborate on more examples of people who could need multiple videos it would be great.

This way I can avoid adding too-much flexibility (i.e. impractical one) to the XModule.

--

Omar Al-Ithawi

Senior Engineer, Edraak.org

Queen Rania Foundation

for Education & Development

T +962 6 4016464  Ext. 700

M +962 7 90574405

E oit...@qrf.org

[Sarina Canelake]

Hi Omar,

Just wanted to weigh in that this sounds like a very cool feature and I look forward to a prototype.

I think Mark had already given two good examples of people who may need multiple videos: switch to a video with an Audio Description track or a video that has been dubbed into another language.  I can't think of more uses offhand but I can try!

Best

Sarina

[Omar Al-Ithawi]

Hi Sarina,

It would be great if you provide a third one :)

Three is the magic number of the re-usability.

Thanks,

[Mark Sadecki]

Hey Omar,

By my count, we are already at 3 good examples ;)

* Sign language
* Audio description
* Foreign language dub

You could even deliver a choice of videos that are localized (same language, but content differs by location/region).
Authors can create videos that aim to teach the same thing, but aimed at different learner profiles (visual learners vs tactile learners for instance)

Mark

​

 

Best,

Mark

Thanks,

<http://www.w3.org/TR/html5/embedded-content-0.html#synchronising-multiple-media-elements - private>

Theoretically, the approach you outline below will be possible after our next release which introduces the "Cohorted Courseware" feature.  A course author can create a cohort of hearing-impaired users, or users who prefer sign language video.  When they author their course, they would add both videos, the primary media, viewable by everyone, and the sign language video, viewable by those in the cohort.  I think the one outlined above would be more elegant.

Thoughts?

Mark

On Wednesday, January 21, 2015 at 4:14:44 AM UTC-5, Omar Al-Ithawi wrote:

Hi All,

Just wanted to notify you that I'm going to work adding sign-language video support to edX video player,

I'm going to add a new xfield for example:

 

   youtube_id_1_5 = String(
        help=_("..."),
        display_name=_("Sign-language alternative video"),
        scope=Scope.settings,
        default=""
    )

And the UI will have an extra button to toggle this video.

I haven't started working on it yet, I'm planning to work on it starting from Feb 2nd.

Your feedback is appreciated,

Thanks,

--

Omar Al-Ithawi

Senior Engineer, Edraak.org - private

Queen Rania Foundation

for Education & Development

T +962 6 4016464  Ext. 700

M +962 7 90574405

E oit...@qrf.org

 - private - private - private - private

--

Omar Al-Ithawi

Senior Engineer, Edraak.org - private

Queen Rania Foundation

for Education & Development

T +962 6 4016464  Ext. 700

M +962 7 90574405

E oit...@qrf.org

 - private - private - private - private

[Omar Al-Ithawi]

Thanks Mark, very helpful indeed :)

Omar Al-Ithawi

[Nimisha Asthagiri]

Hi Omar,

(Sorry for joining late in this discussion.)

I'm the technical lead of the mobile team here at edX and was brought to attention your post on this topic.

I'd like to learn more of what you're thinking.

At edX, we have recently introduced additional encodings of videos to support mobile devices.  We are also planning to support audio-only encodings in the near future.

To support this, we have introduced a Video Abstraction Layer (VAL) that maps a logical edX Video ID to multiple encodings.  So a course author can uniquely identify logical videos by their edx Video IDs while VAL allows a client (UI renderer) to choose whichever mapped encoding(s) it supports.  You can see this field indicated as "EdX Video ID" in the Advanced Setting of the video module in Studio.  This is relatively new technology and we are still flushing out the details.

So rather than introducing additional fields in the video module, we are using VAL as an intermediary service to query encodings for a video.  As of now, the video encoding types we create for VAL are desktop_mp4, desktop_webm, mobile_high, mobile_low, and youtube.

Some of the advantages to VAL are:

(1) Additional encoding types can be created and added without affecting the video modules.

(2) Encodings can be created and updated in bulk (and in real-time) without affecting the video modules.

(3) The video assets and their map of encodings can be shared across course-runs, courses, (and in future) organizations.

(4) It can be configured and managed independently of other course assets.

You can find the source code for VAL at https://github.com/edx/edx-val.

It would be great if your design and approach is aligned with our future plans in this area.

Thanks,

Nimisha

-- 

Mobile, Technical Lead

edX | nim...@edx.org

[Omar Al-Ithawi]

Hi Nimisha,

I quickly reviewed the repo, and here's my initial thoughts: 

VAL is extremely useful to automate the management of different video formats, speeds, hosting, etc with identical content. However, what about different content, like supporting color-blind, adding sign-language. This is no longer automatic, and requires manual video editing and/or special material design. 

I'm actively working on this feature which we need internally late Feb. The initial implementation for now is relatively simple, and so far I plan to go forward with this unless something changes regarding for example VAL.

I have those questions on my mind and it would be great if you help me to answer them:

• Does/will VAL include support for different content i.e. content that requires manual changes like adding sign-language subtitling, or having an audio captions video (more info on this thread above)?
• If it does/will. Where the info will be stored, in the video module (i.e. modulestore) or VAL has special storage for this?

Thanks,

On Thu, Feb 5, 2015 at 12:40 PM, Omar Al-Ithawi <oit...@qrf.org> wrote:

Thanks a lot nimisha for the input,

It's not too late, I actually didn't like having the course author to manage too many videos either,

Hopefully the VAL helps to solve this,

Omar Al-Ithawi

--

Omar Al-Ithawi

Senior Engineer, Edraak.org

Queen Rania Foundation

for Education & Development

T +962 6 4016464  Ext. 700

M +962 7 90574405

E oit...@qrf.org

[Omar Al-Ithawi]

Thanks a lot nimisha for the input,

It's not too late, I actually didn't like having the course author to manage too many videos either,

Hopefully the VAL helps to solve this,

On Thu, Feb 5, 2015 at 3:19 AM, Nimisha Asthagiri <nasth...@edx.org> wrote:

--

Omar Al-Ithawi

[Nimisha Asthagiri]

Hi Omar,

Yes, we do have plan to support different content, such as audio-only version of a video.

Although this feature will probably come in the next quarter and hasn't been fully designed yet, I would imagine we would store the URL for the new content in VAL, along with the other video encodings.

Nimisha