Questions about accessibility properties

[Cheetham, Anastasia]

The IDRC is starting work on a set of metadata authoring supports: tools that can be incorporated into authoring environments to assist with metadata creation. We're going through specific use cases and we've encountered some questions about how to mark them up. I'm hoping the community might be able to clarify whether or not we're interpreting the Schema.org spec properly.

We are working on tools that will embed a11y metadata into HTML markup. Our first use case is a web page that displays a video, using the HTML5 <video> and <track> elements. The base markup (without the schema.org metadata) would look like this:

<video src="sample.ogv">
<track kind="captions" src="sampleCaptions.srt" srclang="en" />
</video>

In this case, the video and its caption are in separate files (the caption is not embedded in the video), but the HTML references both of them, and an HTML5 browser will automatically make the caption available to the viewer.

Our first pass at adding a11y (and other) metadata to this use case is shown below:

<video src="sample.ogv" itemscope itemtype="http://schema.org/Movie">
<meta itemprop="accessMode" content="visual" />
<meta itemprop="accessMode" content="auditory" />
<meta itemprop="accessibilityFeature" content="captions" />
<meta itemprop="hasAdaptation" content="sampleCaptions.srt" />
<track kind="captions" src="sampleCaptions.srt" srclang="en" />
</video>

First, may I ask general double-check type questions about schema.org non-a11y metadata?

i) Is it appropriate to add the itemtype:Movie to a <video> element, or is that redundant?
ii) Is it correct to add itemscope to the <video> element, or do I need to wrap the element in a <div>?

Particular a11y-related questions:

a) Is setting accessibilityFeature:captions correct here, given that the video itself doesn't have an embedded caption? i.e. does the scope of the metadata encompass the collected video+caption files?

b) Is setting hasAdaptation:sampleCaptions.srt appropriate here? Is it appropriate to have both the accessibilityFeature and the hasAdaptation to convey the same information?


Thanks for any thought you can give to these questions – answers to the questions and any other advice. Things always seem to make perfect sense to me in the abstract, but then when the rubber hits the road, it seems not quite so clear.

--
Anastasia Cheetham Inclusive Design Research Centre
ache...@ocadu.ca Inclusive Design Institute
OCAD University

[Matt Garrish]

i) Is it appropriate to add the itemtype:Movie to a <video> element, or is
that redundant?

The movie class would encapsulate all information about the movie, so adding
to the video doesn't look right, for reasons in the next answer. (You might
just use VideoObject if it's just a snippet of video, too.)

ii) Is it correct to add itemscope to the <video> element, or do I need to
wrap the element in a <div>?

I'd personally use a div wrapper, since the internals of the video element
(excepting source and track) are designed as a fallback for user agents that
don't support the element. While I don't believe your markup will impact on
processing/extraction of the information as done (the google tool doesn't
care), it's a slightly odd place to encapsulate it, especially if there is
information that is intended to be viewed. I'm not sure for seo if the
internals are ignored or not, though.

It's one of those places where you need to be careful to explain the
implications and not to copy and adapt, if you plan to advocate its use
generally, that is.

Chaals would probably have better insight into this question, though.

a) Is setting accessibilityFeature:captions correct here, given that the
video itself doesn't have an embedded caption? i.e. does the scope of the
metadata encompass the collected video+caption files?

Yes, I wouldn't see the two as separate in the case of HTML5 video. While
true that the captions are not a track in the video container, that
simplification of being able to combine different sources into a single
video to me doesn't change the fact that captions are available. (Lack of
support for the track element being no different than lack of support for an
embedded caption track in the container, for example.)

This is probably a case where the metadata could be inferred from context,
to be honest, but we're not expecting metadata to be inferred. It would
require two different processings of the data to generate the picture.

b) Is setting hasAdaptation:sampleCaptions.srt appropriate here? Is it
appropriate to have both the accessibilityFeature and the hasAdaptation to
convey the same information?

I'm not sure what will happen to adaptations for 1.1, but I would lean
towards no. While you can argue the caption file is like a transcript, I'm
not sure it's the kind of adaptation someone would be looking for, is it?

But, I'm always open to be wrong on any of these counts.

Matt

--
You received this message because you are subscribed to the Google Groups
"Accessibility Metadata Project" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to a11y-metadata-pr...@googlegroups.com.
To post to this group, send email to a11y-metad...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Martin Quiazon]

On 12/9/13 4:12 PM, "Matt Garrish" <matt.g...@bell.net> wrote:

>i) Is it appropriate to add the itemtype:Movie to a <video> element, or
>is
>that redundant?
>
>The movie class would encapsulate all information about the movie, so
>adding
>to the video doesn't look right, for reasons in the next answer. (You
>might
>just use VideoObject if it's just a snippet of video, too.)

Seconded; the VideoObject item type is appropriate for actual embedded
video.
The Movie class covers creative work details -- title, summary, and the
like --
which are typically located elsewhere on a page.

>a) Is setting accessibilityFeature:captions correct here, given that the
>video itself doesn't have an embedded caption? i.e. does the scope of the
>metadata encompass the collected video+caption files?
>
>Yes, I wouldn't see the two as separate in the case of HTML5 video. While
>true that the captions are not a track in the video container, that
>simplification of being able to combine different sources into a single
>video to me doesn't change the fact that captions are available. (Lack of
>support for the track element being no different than lack of support for
>an
>embedded caption track in the container, for example.)
>
>This is probably a case where the metadata could be inferred from
>context,
>to be honest, but we're not expecting metadata to be inferred. It would
>require two different processings of the data to generate the picture.

As Matt says, you should assert that the video has captions. Don't dwell
on the separation of video and subtitle files -- you're describing the
accessibility of the video presentation as a whole, and being composed of
multiple sources is an implementation detail.

>
>b) Is setting hasAdaptation:sampleCaptions.srt appropriate here? Is it
>appropriate to have both the accessibilityFeature and the hasAdaptation
>to
>convey the same information?

I suspect the transcript is insufficient. If it's just a text rendering of
the video's spoken content, there's nothing in there providing equivalent
presentation of the visual components, so it's inadequate to communicate
the
video's full content.

-martin

[Anastasia Cheetham]

The IDRC is starting work on a set of metadata authoring supports: tools that can be incorporated into authoring environments to assist with metadata creation. We're going through specific use cases and we've encountered some questions about how to mark them up. I'm hoping the community might be able to clarify whether or not we're interpreting the Schema.org spec properly.

We are working on tools that will embed a11y metadata into HTML markup. Our first use case is a web page that displays a video, using the HTML5 <video> and <track> elements. The base markup (without the schema.org metadata) would look like this:

<video src="sample.ogv">
<track kind="captions" src="sampleCaptions.srt" srclang="en" />
</video>

In this case, the video and its caption are in separate files (the caption is not embedded in the video), but the HTML references both of them, and an HTML5 browser will automatically make the caption available to the viewer.

Our first pass at adding a11y (and other) metadata to this use case is shown below:

<video src="sample.ogv" itemscope itemtype="http://schema.org/Movie">
<meta itemprop="accessMode" content="visual" />
<meta itemprop="accessMode" content="auditory" />
<meta itemprop="accessibilityFeature" content="captions" />
<meta itemprop="hasAdaptation" content="sampleCaptions.srt" />
<track kind="captions" src="sampleCaptions.srt" srclang="en" />
</video>

First, may I ask general double-check type questions about schema.org non-a11y metadata?

i) Is it appropriate to add the itemtype:Movie to a <video> element, or is that redundant?

ii) Is it correct to add itemscope to the <video> element, or do I need to wrap the element in a <div>?

Particular a11y-related questions:

a) Is setting accessibilityFeature:captions correct here, given that the video itself doesn't have an embedded caption? i.e. does the scope of the metadata encompass the collected video+caption files?

b) Is setting hasAdaptation:sampleCaptions.srt appropriate here? Is it appropriate to have both the accessibilityFeature and the hasAdaptation to convey the same information?

Thanks for any thought you can give to these questions – answers to the questions and any other advice. Things always seem to make perfect sense to me in the abstract, but then when the rubber hits the road, it seems not quite so clear.


--

Anastasia Cheetham a...@anastasiacheetham.ca
Inclusive Design Research Centre OCAD University

[Cheetham, Anastasia]

Matt and Martin, thanks for your responses. This conversation is very helpful.

On 2013-12-09, at 7:12 PM, Matt Garrish wrote:

>> i) Is it appropriate to add the itemtype:Movie to a <video> element, or is that redundant?
>>
> The movie class would encapsulate all information about the movie, so adding to the video doesn't look right, for reasons in the next answer. (You might just use VideoObject if it's just a snippet of video, too.)
>
>> ii) Is it correct to add itemscope to the <video> element, or do I need to wrap the element in a <div>?
>>
> I'd personally use a div wrapper, since the internals of the video element (excepting source and track) are designed as a fallback for user agents that don't support the element.

Hm. Ok, I see your point about the internals being a fallback. I assume you suggest a div wrapper to avoid encapsulating that fallback content. How exactly could I encapsulate the video and track elements without encapsulating the fallback content?

>> b) Is setting hasAdaptation:sampleCaptions.srt appropriate here? Is it appropriate to have both the accessibilityFeature and the hasAdaptation to convey the same information?
>>
> I'm not sure what will happen to adaptations for 1.1, but I would lean towards no. While you can argue the caption file is like a transcript, I'm not sure it's the kind of adaptation someone would be looking for, is it?

and Martin answered

> I suspect the transcript is insufficient. If it's just a text rendering of
> the video's spoken content, there's nothing in there providing equivalent
> presentation of the visual components, so it's inadequate to communicate
> the video's full content.

I think we might need a bit of clarification on what "hasAdaption" means.

In the various accessibility specifications and standards that the Schema.org a11y-md properties are based on, the word "adaptation" is used to refer to any form of alternate presentation of the content. Using that definition, captions _are_ a text-based adaptation of the audio portion of the video. So using "hasAdaption" to refer to the caption would be fine, if you use this understanding. Does that make sense?

Given that understanding (i.e. that a caption _is_ an adaptation), my question was about the duplication of information, i.e. using both accessibilityFeature and hasAdaptation (which I realize is not yet in the Schema.org properties) to refer to the same adaptation: the caption.


On the topic of hasAdaptation, this is probably a reasonable time to introduce a slightly less straightforward use case, which is the motivation for wanting to use "hasAdaptation:" the case where an adaptation is available but not as part of the current resource. Let's say there's another website that has the full screenplay for our video. That screenplay would be a fine text-based alternative to the video, so it would be nice for the metadata on the video to convey that. I'd like to be able to say:

<video src="sample.ogv" itemscope itemtype="http://schema.org/VideoObject">

<meta itemprop="accessMode" content="visual" />
<meta itemprop="accessMode" content="auditory" />

<meta itemprop="hasAdaptation" content="http://othersite.com/screenplay.txt" />
</video>

or perhaps

<div itemscope itemtype="http://schema.org/VideoObject">
<video src="sample.ogv">

<meta itemprop="accessMode" content="visual" />
<meta itemprop="accessMode" content="auditory" />

<meta itemprop="hasAdaptation" content="http://othersite.com/screenplay.txt" />
</video>
</div>

if the encapsulating div is more appropriate.

[Matt Garrish]

> How exactly could I encapsulate the video and track elements without
> encapsulating the fallback content?

You do encapsulate the fallback content inside the video element with the
track, but you shouldn't put content inside the video element that isn't
fallbacks. I wouldn't consider statements about the nature of the video
fallbacks, as they will only render if the video element is not supported.

The source and track element children are intrinsically linked to the video
and are not counted as fallback content. I gather the html folks now don't
like this model, which is why adaptive images have taken a turn for the odd
(src-n attributes), but that doesn't change the fact, more just an aside.

In other words, simply shift the properties up to be siblings of the video
and encapsulate the whole in a div, along these lines:

<div itemprop="video" itemscope itemtype="http://schema.org/VideoObject">

<meta itemprop="accessMode" content="visual" />
<meta itemprop="accessMode" content="auditory" />
<meta itemprop="hasAdaptation"
content="http://othersite.com/screenplay.txt" />

<video itemprop="contentUrl" src="sample.ogv">
<track type="captions" src="captions.srt" />
<p>Sorry, but your browser does not support the HTML5 video
element.</p>
</video>
</div>

When you generate this metadata, you will get the same picture of the video
resources.

Does this answer your question?

Matt

-----Original Message-----
From: Cheetham, Anastasia
Sent: Tuesday, December 10, 2013 10:18 AM
To: a11y-metad...@googlegroups.com
Cc: Matt Garrish ; Martin Quiazon
Subject: Re: [a11y-metadata-project] Questions about accessibility
properties

[Matt Garrish]

> the word "adaptation" is used to refer to any form of alternate
> presentation of the content

Right, but a caption track is designed to be fed into video playback for
overlay. I don't really view it as a standalone content adaptation, but if
there are tts programs that can be fed the file format and render the text
in the absence of a video my position would change. That's why I like fence
sitting in these situations. :)

Matt

-----Original Message-----
From: Cheetham, Anastasia

Sent: Tuesday, December 10, 2013 10:18 AM
To: a11y-metad...@googlegroups.com
Cc: Matt Garrish ; Martin Quiazon

Subject: Re: [a11y-metadata-project] Questions about accessibility
properties

[Cheetham, Anastasia]

On 2013-12-10, at 7:57 PM, Matt Garrish wrote:

> <div itemprop="video" itemscope itemtype="http://schema.org/VideoObject">
> <meta itemprop="accessMode" content="visual" />
> <meta itemprop="accessMode" content="auditory" />
> <meta itemprop="hasAdaptation" content="http://othersite.com/screenplay.txt" />
> <video itemprop="contentUrl" src="sample.ogv">
> <track type="captions" src="captions.srt" />
> <p>Sorry, but your browser does not support the HTML5 video element.</p>
> </video>
> </div>

Gotcha. That makes total sense. I'm sorry, I don't know why I wasn't wrapping my head around this properly. Thanks.

[Cheetham, Anastasia]

On 2013-12-10, at 8:04 PM, Matt Garrish wrote:

>> the word "adaptation" is used to refer to any form of alternate presentation of the content
>
> Right, but a caption track is designed to be fed into video playback for overlay. I don't really view it as a standalone content adaptation, but if there are tts programs that can be fed the file format and render the text in the absence of a video my position would change. That's why I like fence sitting in these situations. :)

You're right, it's not a standalone adaptation, but it's still an adaptation. In earlier versions of the Access for All specifications, we actually had a flag to indicate whether an adaption was "partial" or "full" but we've lost that ability. In my mind, though, a caption is still an adaptation, just as a sign language translation is, or a transcript, etc.

I wonder if Madeleine or Andy might have any thoughts on this, since they're also on the IMS Accessibility working group with me?

[Madeleine Rothberg]

Is part of the confusion here that we haven't recently discussed
"hasAdaptation" in detail? We changed IMSAfAv3:adapatationType into
Schema.org:accessibilityFeature, so originally the whole list of features
were known as adaptations. It does not relate to standalone or not. But
that name change didn't percolate to hasAdaptation because we set it aside
because it was not going to be included in Schema.org v1.

The role of the hasAdaptation pointer is to point to some other place
where you can find resources that make this resource more accessible. I
think in Anastasia's example the caption file is available on the same
page, so I don't think that pointer is needed. Marking up the caption as
an accessibilityFeature may be sufficient. (Apologies if I am
misunderstanding the markup needs.)

Also, the Schema.org recommendation was that we hold off on making our own
pointer properties and use the exampleOfWork pointers that are coming from
the bibliographic group. So we aren't in control of the names of our
pointers in the long run.

-Madeleine

[Cheetham, Anastasia]

Madeleine, thanks for your comments, they're very helpful.

On 2013-12-11, at 2:26 PM, Madeleine Rothberg wrote:

> The role of the hasAdaptation pointer is to point to some other place
> where you can find resources that make this resource more accessible. I
> think in Anastasia's example the caption file is available on the same
> page, so I don't think that pointer is needed. Marking up the caption as
> an accessibilityFeature may be sufficient. (Apologies if I am
> misunderstanding the markup needs.)

Ok, this is part of what I was wondering; it did seem at best redundant, and at worst inappropriate because, as you say, the pointer is to reference something "elsewhere," whereas in my use case, the caption is "here." I will recommend to our team that we not use pointers if the caption is referenced by the <track> element; we'll only use pointers to reference adaptations that are "elsewhere."

> Also, the Schema.org recommendation was that we hold off on making our own
> pointer properties and use the exampleOfWork pointers that are coming from
> the bibliographic group. So we aren't in control of the names of our
> pointers in the long run.

Yes, we did hold off, but I guess I wasn't clear on whether or not we would be adding the pointers once we had discussed them.

I'm just looking now at the CreativeWork relationships proposal found at
http://www.w3.org/community/schemabibex/wiki/CreativeWork_Relationships

Is this the "exampleOfWork" property we're considering for referencing "elsewhere" adaptions? I'm finding it a challenge wrapping this proposal around accessibility adaptations. It's not fitting very well. This CreativeWork relationship proposal seems to be trying to describe the relationship between a conceptual thing and its concrete instantiations. They don't seem to be using the relationship directly between the book and the derived movie, do they?

[Madeleine Rothberg]

I'm not sure -- and if this is the intended set, it also seems better
suited to what we used to call "full" adaptations and not to "partial"
ones, in that a fragment of an adaptation doesn't seem like a full
instance of the work. Also, in some cases we want to point to things that
aren't the same "work" at all, but other good resources on the same topic
in different access modes. But maybe we can pick useful terms within that
group of relationships, depending which are adopted.

-Madeleine

[Charles Myers]

I'd actually go one step further on the "hasAdapatation." I don't think that the subtitle file itself counts as as a real "adapatation." The adaptation should be a full-fledged adaptation or both the media and the subtitle together.

So, just assert the accessibility feature of "captions" and embed the link to the SRT file in a way that the media player would understand (which differs by the technology)

I agree with the rest of the comments.

[Madeleine Rothberg]

There are a huge range of cases out there, from libraries of subtitles
(not stored with the film itself) to separate captioned videos, to "hey,
this news story would work better for someone who can't see the pictures
in that photo gallery." Any of those can be a "real adaptation." When the
alternative is available in the same location, there's no need for a
pointer, but there will be cases where metadata authors want to point to
alternatives that are not "full-fledged adaptations" of the resource being
described, by your description. I don't think we should be overly
prescriptive in defining those relationships.

-Madeleine

[Matt Garrish]

I've certainly seen my bias showing in this thread, as I'd been equating
adaptation with the full adaptation concept, or at least with the directly
renderable as-is and not for additional/alternate enhancement.

But I don't think the schema bibex work is exclusive to works, as it would
filter down to audio, video and image objects by contribution to
CreativeWork. One could, presumably, point to caption files, descriptions
and other items that could be used/substituted. It's only at the higher
level concepts, like books, that you would presumably only list full
adaptations.

It might be worth posing the question to that group, though.

Matt

-----Original Message-----
From: Madeleine Rothberg
Sent: Wednesday, December 11, 2013 4:28 PM
To: Charles Myers ; Martin Quiazon ; Cheetham, Anastasia
Cc: Matt Garrish ; a11y-metad...@googlegroups.com
Subject: Re: [a11y-metadata-project] Questions about accessibility
properties

[Emmanuelle Gutiérrez y Restrepo]

Matt, Anastasia and all,

Just a note: Warn user that your browser does not support HTML5 without
offering any alternative is not really an alternative equivalent or
"fallback content". Instead it should provide the user a way to access the
video by other means, such as via download in different formats.

All the best,
Emmanuelle Gutiérrez y Restrepo
Patrono y Directora General
Fundación Sidar - Acceso Universal
Email: coor...@sidar.org
Personal: Emman...@sidar.org
Web: http://sidar.org


-----Mensaje original-----
De: a11y-metad...@googlegroups.com
[mailto:a11y-metad...@googlegroups.com] En nombre de Matt Garrish
Enviado el: miércoles, 11 de diciembre de 2013 1:58
Para: Cheetham, Anastasia; a11y-metad...@googlegroups.com
CC: Martin Quiazon
Asunto: Re: [a11y-metadata-project] Questions about accessibility properties

[matt.g...@bell.net]

Yes, absolutely. That was a hasty example meant only to highlight that it's not accessible alternatives that go inside, only fallback for a lack of support for the element. I'm finding a perception that transcripts and alternate layouts can go inside the element, despite the message in the HTML5 spec warning against this.
 
Matt