Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
DASH: help with nsBuiltinDecoderStateMachine
21 views
Skip to first unread message
Steve Workman
Apr 13, 2012, 5:54:36 PM4/13/12
to dev-...@lists.mozilla.org
So, here's where I'm at with DASH:
As per Roc's suggestion, I have implemented a proxy decoder and MediaResource: nsDASHDecoder and DASHMediaResource, and I have a hardcoded stream switching happening, but I need some help hacking nsBuiltinDecoderStateMachine. I have provided a proposed flow below for feedback. The assumption is that this flow is the correct approach - please comment/correct/critique for feedback.
Info on the new proxy objects:
-- nsDASHDecoder is instantiated in nsHTMLMediaElement like any other decoder; i.e. based on the mime type (="application/dash+xml").
-- DASHMediaResource is also created in nsHTMLMediaElement, like any other MediaResource. It is a subclass of ChannelMediaResource, although it does not use nsMediaCache/Stream.
Before looking at the flow, it's important to note that:
1. from the WebMDASH spec, there is one Segment per media period, i.e. one URL, i.e. the media files are non-chunked, although they are separated into audio and video.
2. only one audio stream is allowed, so we only need to worry about switching video streams.
The flow is as follows (a lot of it is similar to current flow with regular decoders; bolded items are new/changed and I would appreciate your comments):
1. element creates nsDASHDecoder object:
member var initialization to default values, including a/v sub-decoders
nsBuiltinDecoder and nsMediaDecoder constructors are called.
nsBuiltinDecoder::Init() is called
element creates new MediaResource (DASHMediaResource):
ChannelMediaResource()
mCacheStream created (unused in DASHMediaResource)
mOffset set to 0 etc.
MediaResource()
mDecoder = nsDASHDecoder (created earlier)
2. element calls nsDASHDecoder->Load() = nsBuiltinDecoder->Load()
DASHMediaResource::Open()
does sec. checks for CORS based on element/principal
setup HTTP headers to: request byte-ranges
accept audio and video only
disable 'gzip,deflate' (to get Content-Length)
set the referrer
channel->AsyncOpen with stream listener set to private MPDStreamListener
(OnStart..., OnData... and OnStop... async callbacks discussed later)
nsBuiltinDecoderStateMachine is NOT created at this point, unlike other decoders
3. element finishes decoder setup after channel is opened and state machine
thread is started:
element added to media URI table
sets flag (mPausedForInactiveDocument) to note that element did not
suspend the resource from loading
sets volume, played/seeked status (for content layer +)
if element state is not paused, calls mDecoder->Play()
sets mBegun = TRUE
-- At this point all the main objects are created to download and parse
the MPD.
4. MPD Download (Async DASHMediaResource channel callbacks)
OnMPDStartRequest()
Usual networking checks for status, origin etc.
decoder->SetDuration() if duration in HTTP response headers
Check for byte range request capability (not so important for MPD)
Create byte buffer for raw MPD data (rather than nsMediaCacheStream)
Sends decoder an initial progress event; starts "stalled" timer
OnMPDDataAvailable()
Copies data to the byte buffer
OnMPDStopRequest()
Dispatches an MPD Download complete event
NotifyMPDDownloadComplete()
ResourceLoaded() - parses MPD XML to DOM to MPD classes
Start()
gets Segment URLs from MPD for audio stream, and video streams
creates http channels, ChannelMediaResources and nsWebMDecoders
stores decoders as member vars (to forward function calls later)
nsWebMDecoder->Load() for audio and video streams
nsBuiltinDecoder->Load(), ChannelMediaResource->Open(), creating
nsMediaCacheStream etc.
-- At this point the Segment media streams are managed by individual
ChannelMediaResource objects, nsWebMDecoders, nsWebMReaders and
nsBuiltinDecoderStateMachines:
nsHTMLMediaElement
1 <--------> 1 nsDASHDecoder 1 <--------> 1 DASHMediaResource
| 1 | 1
nsBuiltinDecoderStateMachine 1 <-| |
|
| *
nsBuiltinDecoderStateMachine 1 - 1 nsWebMDecoder 1 - 1 ChannelMediaResource
These on their own should work as normal, but events from element and the
main DASH state machine need to be handled and forwarded correctly.
Note, I also think that a proxy element object would be useful at the DASH level of things, so that the sub-decoders can have their notifications aggregated before sending to the real nsHTMLMediaElement.
You'll notice that ALL the video decoders are Loaded after the MPD is parsed. Currently, this is to get around errors in my working code at switching time (the 2nd video's state machine complains because I called a Seek immediately after a Load - it looks like it need s a little time to get metadata first). This is obviously not what we want, because it starts downloading both files. My assumption is that we need to get the metadata for both files very early on to get the nsWebMDecoders and state machines initialized properly, BEFORE we can Seek to the right place in the second stream, so that it can take over playing from the first stream.
Re switching, at this is stage it's just a simple timer event, and the APIs are not clean at all. The point has been to figure out how to control the decoders etc. before finalizing the API which will initiate the switch (and adaptation will be dealt with after that).
So, I need some help with hacking the state machine, so that it loads metadata only and then stops. Only for the alternative streams, of course. If this early metadata loading doesn't make sense, or you see another way, please let me know.
Many thanks!
Steve.
As per Roc's suggestion, I have implemented a proxy decoder and MediaResource: nsDASHDecoder and DASHMediaResource, and I have a hardcoded stream switching happening, but I need some help hacking nsBuiltinDecoderStateMachine. I have provided a proposed flow below for feedback. The assumption is that this flow is the correct approach - please comment/correct/critique for feedback.
Info on the new proxy objects:
-- nsDASHDecoder is instantiated in nsHTMLMediaElement like any other decoder; i.e. based on the mime type (="application/dash+xml").
-- DASHMediaResource is also created in nsHTMLMediaElement, like any other MediaResource. It is a subclass of ChannelMediaResource, although it does not use nsMediaCache/Stream.
Before looking at the flow, it's important to note that:
1. from the WebMDASH spec, there is one Segment per media period, i.e. one URL, i.e. the media files are non-chunked, although they are separated into audio and video.
2. only one audio stream is allowed, so we only need to worry about switching video streams.
The flow is as follows (a lot of it is similar to current flow with regular decoders; bolded items are new/changed and I would appreciate your comments):
1. element creates nsDASHDecoder object:
member var initialization to default values, including a/v sub-decoders
nsBuiltinDecoder and nsMediaDecoder constructors are called.
nsBuiltinDecoder::Init() is called
element creates new MediaResource (DASHMediaResource):
ChannelMediaResource()
mCacheStream created (unused in DASHMediaResource)
mOffset set to 0 etc.
MediaResource()
mDecoder = nsDASHDecoder (created earlier)
2. element calls nsDASHDecoder->Load() = nsBuiltinDecoder->Load()
DASHMediaResource::Open()
does sec. checks for CORS based on element/principal
setup HTTP headers to: request byte-ranges
accept audio and video only
disable 'gzip,deflate' (to get Content-Length)
set the referrer
channel->AsyncOpen with stream listener set to private MPDStreamListener
(OnStart..., OnData... and OnStop... async callbacks discussed later)
nsBuiltinDecoderStateMachine is NOT created at this point, unlike other decoders
3. element finishes decoder setup after channel is opened and state machine
thread is started:
element added to media URI table
sets flag (mPausedForInactiveDocument) to note that element did not
suspend the resource from loading
sets volume, played/seeked status (for content layer +)
if element state is not paused, calls mDecoder->Play()
sets mBegun = TRUE
-- At this point all the main objects are created to download and parse
the MPD.
4. MPD Download (Async DASHMediaResource channel callbacks)
OnMPDStartRequest()
Usual networking checks for status, origin etc.
decoder->SetDuration() if duration in HTTP response headers
Check for byte range request capability (not so important for MPD)
Create byte buffer for raw MPD data (rather than nsMediaCacheStream)
Sends decoder an initial progress event; starts "stalled" timer
OnMPDDataAvailable()
Copies data to the byte buffer
OnMPDStopRequest()
Dispatches an MPD Download complete event
NotifyMPDDownloadComplete()
ResourceLoaded() - parses MPD XML to DOM to MPD classes
Start()
gets Segment URLs from MPD for audio stream, and video streams
creates http channels, ChannelMediaResources and nsWebMDecoders
stores decoders as member vars (to forward function calls later)
nsWebMDecoder->Load() for audio and video streams
nsBuiltinDecoder->Load(), ChannelMediaResource->Open(), creating
nsMediaCacheStream etc.
-- At this point the Segment media streams are managed by individual
ChannelMediaResource objects, nsWebMDecoders, nsWebMReaders and
nsBuiltinDecoderStateMachines:
nsHTMLMediaElement
1 <--------> 1 nsDASHDecoder 1 <--------> 1 DASHMediaResource
| 1 | 1
nsBuiltinDecoderStateMachine 1 <-| |
|
| *
nsBuiltinDecoderStateMachine 1 - 1 nsWebMDecoder 1 - 1 ChannelMediaResource
These on their own should work as normal, but events from element and the
main DASH state machine need to be handled and forwarded correctly.
Note, I also think that a proxy element object would be useful at the DASH level of things, so that the sub-decoders can have their notifications aggregated before sending to the real nsHTMLMediaElement.
You'll notice that ALL the video decoders are Loaded after the MPD is parsed. Currently, this is to get around errors in my working code at switching time (the 2nd video's state machine complains because I called a Seek immediately after a Load - it looks like it need s a little time to get metadata first). This is obviously not what we want, because it starts downloading both files. My assumption is that we need to get the metadata for both files very early on to get the nsWebMDecoders and state machines initialized properly, BEFORE we can Seek to the right place in the second stream, so that it can take over playing from the first stream.
Re switching, at this is stage it's just a simple timer event, and the APIs are not clean at all. The point has been to figure out how to control the decoders etc. before finalizing the API which will initiate the switch (and adaptation will be dealt with after that).
So, I need some help with hacking the state machine, so that it loads metadata only and then stops. Only for the alternative streams, of course. If this early metadata loading doesn't make sense, or you see another way, please let me know.
Many thanks!
Steve.
Chris Pearce
Apr 17, 2012, 1:01:34 AM4/17/12
to Steve Workman
On 14/04/2012 9:54 a.m., Steve Workman wrote:
> 1. from the WebMDASH spec, there is one Segment per media period,
i.e. one URL, i.e. the media files are non-chunked, although they are
separated into audio and video.
Just so we're clear, there's a manifest file, and one video file per
> 1. from the WebMDASH spec, there is one Segment per media period,
i.e. one URL, i.e. the media files are non-chunked, although they are
separated into audio and video.
video stream, and one file for the audio stream?
> Note, I also think that a proxy element object would be useful at the
DASH level of things, so that the sub-decoders can have their
notifications aggregated before sending to the real nsHTMLMediaElement.
> You'll notice that ALL the video decoders are Loaded after the MPD is
parsed.
resource contains 1 high quality stream, 1 low quality stream, and 1
audio stream, you'd create 3 separate WebMDecoders for each one?
> Currently, this is to get around errors in my working code at
switching time (the 2nd video's state machine complains because I called
to get metadata first).
Yes, as you allude to, the decoder needs enough time to read the Cues
(seek index) before it can seek. If the Cues are stored at the start of
the stream like they're supposed to be, this shouldn't be too bad...
> So, I need some help with hacking the state machine, so that it loads
metadata only and then stops. Only for the alternative streams, of
course. If this early metadata loading doesn't make sense, or you see
another way, please let me know.
attribute, we stop the download after metadata is loaded by calling
nsMediaDecoder::Suspend() when the decoder notifies us that it's loaded
metadata (by calling nsHTMLMediaElement::FirstFrameLoaded()). So in your
proxy which intercepts the nsBuiltinDecoders' calls to
nsHTMLMediaElement, can you call Suspend() on the decoder when you
intercept the alternative decoder's call to FirstFrameLoaded()?
Cheers,
Chris P.
Chris Pearce
Apr 17, 2012, 1:02:22 AM4/17/12
to Steve Workman
On 14/04/2012 9:54 a.m., Steve Workman wrote:
> 1. from the WebMDASH spec, there is one Segment per media period,
i.e. one URL, i.e. the media files are non-chunked, although they are
separated into audio and video.
i.e. one URL, i.e. the media files are non-chunked, although they are
separated into audio and video.
Just so we're clear, there's a manifest file, and one video file per
video stream, and one file for the audio stream?
video stream, and one file for the audio stream?
> Note, I also think that a proxy element object would be useful at the
DASH level of things, so that the sub-decoders can have their
notifications aggregated before sending to the real nsHTMLMediaElement.
This sounds reasonable.
DASH level of things, so that the sub-decoders can have their
notifications aggregated before sending to the real nsHTMLMediaElement.
> You'll notice that ALL the video decoders are Loaded after the MPD is
parsed.
So you load up one decoder for each stream in the resource? So if a
resource contains 1 high quality stream, 1 low quality stream, and 1
audio stream, you'd create 3 separate WebMDecoders for each one?
resource contains 1 high quality stream, 1 low quality stream, and 1
audio stream, you'd create 3 separate WebMDecoders for each one?
> Currently, this is to get around errors in my working code at
switching time (the 2nd video's state machine complains because I called
a Seek immediately after a Load - it looks like it needs a little time
switching time (the 2nd video's state machine complains because I called
to get metadata first).
Yes, as you allude to, the decoder needs enough time to read the Cues
(seek index) before it can seek. If the Cues are stored at the start of
the stream like they're supposed to be, this shouldn't be too bad...
> So, I need some help with hacking the state machine, so that it loads
metadata only and then stops. Only for the alternative streams, of
course. If this early metadata loading doesn't make sense, or you see
another way, please let me know.
metadata only and then stops. Only for the alternative streams, of
course. If this early metadata loading doesn't make sense, or you see
another way, please let me know.
Steve Workman
Apr 17, 2012, 1:41:44 PM4/17/12
to Chris Pearce, mozilla-...@lists.mozilla.org
On Apr 16, 2012, at 10:01 PM, Chris Pearce wrote:
> On 14/04/2012 9:54 a.m., Steve Workman wrote:
>
> > 1. from the WebMDASH spec, there is one Segment per media period, i.e. one URL, i.e. the media files are non-chunked, although they are separated into audio and video.
>
>
> Just so we're clear, there's a manifest file, and one video file per video stream, and one file for the audio stream?
Yup. Manifest file is .mpd = Media Presentation Description. It's in XML.
And yup, one audio file, non-chunked, so just one big file. And one video file per video stream, i.e. one file for each bitrate. These are also non-chunked.
>
> > Note, I also think that a proxy element object would be useful at the DASH level of things, so that the sub-decoders can have their notifications aggregated before sending to the real nsHTMLMediaElement.
>
> This sounds reasonable.
> > Note, I also think that a proxy element object would be useful at the DASH level of things, so that the sub-decoders can have their notifications aggregated before sending to the real nsHTMLMediaElement.
>
Good :)
>
>
> > You'll notice that ALL the video decoders are Loaded after the MPD is parsed.
>
>
> > You'll notice that ALL the video decoders are Loaded after the MPD is parsed.
>
> So you load up one decoder for each stream in the resource? So if a resource contains 1 high quality stream, 1 low quality stream, and 1 audio stream, you'd create 3 separate WebMDecoders for each one?
>
Right, that is the current way of thinking. Do you foresee any problems with this approach? The reason I chose this way to start is because this connection of one file and one decoder already exists in the code; downloading, seeking, playing etc. are already controlled on a per file basis, so it seemed logical to pursue this avenue and reuse as much of the current code/design as possible.
>
>
> > Currently, this is to get around errors in my working code at switching time (the 2nd video's state machine complains because I called a Seek immediately after a Load - it looks like it needs a little time to get metadata first).
>
> Yes, as you allude to, the decoder needs enough time to read the Cues (seek index) before it can seek. If the Cues are stored at the start of the stream like they're supposed to be, this shouldn't be too bad…
> > Currently, this is to get around errors in my working code at switching time (the 2nd video's state machine complains because I called a Seek immediately after a Load - it looks like it needs a little time to get metadata first).
>
The order within the file should be as follows, according to the spec:
EBML header, Segment header, Segment Information and Tracks.
Cues,
Clusters.
However, there is this line at the end of the description:
"If the Cues is placed after the Clusters then a SeekHead shall be contained in the Initialization Segment with a reference to the Cues." [1]
Note: the Initialization Segment is at the start of the file.
>
> > So, I need some help with hacking the state machine, so that it loads metadata only and then stops. Only for the alternative streams, of course. If this early metadata loading doesn't make sense, or you see another way, please let me know.
>
> So normally when a media element is loaded with the preload="metadata" attribute, we stop the download after metadata is loaded by calling nsMediaDecoder::Suspend() when the decoder notifies us that it's loaded metadata (by calling nsHTMLMediaElement::FirstFrameLoaded()). So in your proxy which intercepts the nsBuiltinDecoders' calls to nsHTMLMediaElement, can you call Suspend() on the decoder when you intercept the alternative decoder's call to FirstFrameLoaded()?
Cool. I'll play around with the proxy element to see how I can control all of this. I want to explore how much control of network downloading we have through this, since it's separate threads etc. But, that's a second or later stage. First is getting Suspend working correctly to enable switching.
>
> Cheers,
> Chris P.
Thanks!
Steve.
[1] http://wiki.webmproject.org/adaptive-streaming/webm-dash-specification
Chris Pearce
Apr 19, 2012, 11:39:33 PM4/19/12
to Steve Workman
Wait, I thought the plan was to have DashMediaResource switch between
streams at cluster boundaries so what it exposes looks like a single
resource to the nsWebMDecoder? From your description here it now looks
like your intending to make the switching decisions in nsDashDecoder,
and have multiple nsWebMDecoders, deferring
nsBuiltinDecoderReader::Decode{Video,Audio}Data() calls to the one which
will work at whatever bitrate can be achieved?
>> So you load up one decoder for each stream in the resource? So if a
resource contains 1 high quality stream, 1 low quality stream, and 1
audio stream, you'd create 3 separate WebMDecoders for each one?
>>
> Right, that is the current way of thinking. Do you foresee any
problems with this approach?
Assuming you're going down the multiple decoders path... Maybe I
miswrote... I think what you'd want is a nsWebMReader for each stream,
rather than a nsWebMDecoder for each stream.
Readers manage the demuxing and decoding, they don't manage any playback
state themselves. Whereas each nsBuiltinDecoder comes with a state
machine attached, which isn't probably what you want, and an interface
to interact with the main thread... I think you'd want an nsDashReader
which implements the nsBuitinDecoderReader interface (and responds to
calls by nsBuiltinDecoderStateMachine), and which owns the nsWebMReaders
and decides which reader to read from based on network conditions,
suspending the MediaResource of readers which aren't being used.
nsWebMReader does interact with its nsBuiltinDecoder at times, so you
may have to have a proxy to manage that interaction, or refactor the
nsBuiltinDecoderReader interface/implementations to remove the reader ->
decoder communication.
Hope that makes sense.
Regards,
Chris P.
streams at cluster boundaries so what it exposes looks like a single
resource to the nsWebMDecoder? From your description here it now looks
like your intending to make the switching decisions in nsDashDecoder,
and have multiple nsWebMDecoders, deferring
nsBuiltinDecoderReader::Decode{Video,Audio}Data() calls to the one which
will work at whatever bitrate can be achieved?
>> So you load up one decoder for each stream in the resource? So if a
resource contains 1 high quality stream, 1 low quality stream, and 1
audio stream, you'd create 3 separate WebMDecoders for each one?
>>
> Right, that is the current way of thinking. Do you foresee any
problems with this approach?
miswrote... I think what you'd want is a nsWebMReader for each stream,
rather than a nsWebMDecoder for each stream.
Readers manage the demuxing and decoding, they don't manage any playback
state themselves. Whereas each nsBuiltinDecoder comes with a state
machine attached, which isn't probably what you want, and an interface
to interact with the main thread... I think you'd want an nsDashReader
which implements the nsBuitinDecoderReader interface (and responds to
calls by nsBuiltinDecoderStateMachine), and which owns the nsWebMReaders
and decides which reader to read from based on network conditions,
suspending the MediaResource of readers which aren't being used.
nsWebMReader does interact with its nsBuiltinDecoder at times, so you
may have to have a proxy to manage that interaction, or refactor the
nsBuiltinDecoderReader interface/implementations to remove the reader ->
decoder communication.
Hope that makes sense.
Regards,
Chris P.
Steve Workman
Apr 20, 2012, 7:19:21 PM4/20/12
to Chris Pearce, mozilla-...@lists.mozilla.org
On Apr 19, 2012, at 8:39 PM, Chris Pearce wrote:
> Wait, I thought the plan was to have DashMediaResource switch between streams at cluster boundaries so what it exposes looks like a single resource to the nsWebMDecoder? From your description here it now looks like your intending to make the switching decisions in nsDashDecoder, and have multiple nsWebMDecoders, deferring nsBuiltinDecoderReader::Decode{Video,Audio}Data() calls to the one which will work at whatever bitrate can be achieved?
>
> Assuming you're going down the multiple decoders path... Maybe I miswrote... I think what you'd want is a nsWebMReader for each stream, rather than a nsWebMDecoder for each stream.
No miswriting - multiple decoders is where I was going :)
> Assuming you're going down the multiple decoders path... Maybe I miswrote... I think what you'd want is a nsWebMReader for each stream, rather than a nsWebMDecoder for each stream.
>
> Readers manage the demuxing and decoding, they don't manage any playback state themselves. Whereas each nsBuiltinDecoder comes with a state machine attached, which isn't probably what you want, and an interface to interact with the main thread... I think you'd want an nsDashReader which implements the nsBuitinDecoderReader interface (and responds to calls by nsBuiltinDecoderStateMachine), and which owns the nsWebMReaders and decides which reader to read from based on network conditions, suspending the MediaResource of readers which aren't being used.
>
I've spent some time looking through the code and figuring this out and it seems like a good path to take. Some points for you to verify/correct on my understanding:
> Readers manage the demuxing and decoding, they don't manage any playback state themselves. Whereas each nsBuiltinDecoder comes with a state machine attached, which isn't probably what you want, and an interface to interact with the main thread... I think you'd want an nsDashReader which implements the nsBuitinDecoderReader interface (and responds to calls by nsBuiltinDecoderStateMachine), and which owns the nsWebMReaders and decides which reader to read from based on network conditions, suspending the MediaResource of readers which aren't being used.
>
-- to start, there would be an nsDASHDecoder object and a DASHMediaResource - this is to get the MPD, parse it, and get the URLs and metadata for the streams
-- then, nsDASHDecoder would hand off/forward calls to a single nsWebMDecoder. nsHTMLMediaElement and nsBuiltinDecoderStateMachine would need no changes, and only one of each would be required. They would talk to nsWebMDecoder (via nsDASHDecoder maybe) as normal.
-- at least two nsWebMReaders and ChannelMediaResources would be created; two because we need one for the audio stream and one or more for the video streams.
-- calls from the state machine would go to nsDASHReader, which would forward them to the appropriate nsWebMReader. Appropriate means 1) audio and/or video 2) which video reader, depending on network conditions. This means nsDASHReader needs to have access to metadata associated with each stream - probably I'll keep the metadata and decision in DASHMediaResource (which has the manifest data), and just return or pass an index to nsDASHReader.
-- according to DASH/WebM spec, cuepoint timecodes and clusters are aligned, so we can use the existing model for HTTP byte range requests to seek the right data. I'm noticing in the code that we just seek the start though, and just pull data from there. I might want to revisit this later to pull a specific range for each request, but it seems fine for DASH/WebM and for what we want now.
This looks like a good approach, solving many of the 'proxy' issues I was facing. I'll work on prototyping this next week.
> nsWebMReader does interact with its nsBuiltinDecoder at times, so you may have to have a proxy to manage that interaction, or refactor the nsBuiltinDecoderReader interface/implementations to remove the reader -> decoder communication.
>
> Hope that makes sense.
Thanks,
Steve.
Steve Workman
Apr 24, 2012, 2:11:04 PM4/24/12
to Steve Workman, Chris Pearce, mozilla-...@lists.mozilla.org
On Apr 20, 2012, at 4:19 PM, Steve Workman wrote:
> I've spent some time looking through the code and figuring this out and it seems like a good path to take. Some points for you to verify/correct on my understanding:
>
> -- to start, there would be an nsDASHDecoder object and a DASHMediaResource - this is to get the MPD, parse it, and get the URLs and metadata for the streams
>
> -- then, nsDASHDecoder would hand off/forward calls to a single nsWebMDecoder. nsHTMLMediaElement and nsBuiltinDecoderStateMachine would need no changes, and only one of each would be required. They would talk to nsWebMDecoder (via nsDASHDecoder maybe) as normal.
>
> -- at least two nsWebMReaders and ChannelMediaResources would be created; two because we need one for the audio stream and one or more for the video streams.
>
> -- calls from the state machine would go to nsDASHReader, which would forward them to the appropriate nsWebMReader. Appropriate means 1) audio and/or video 2) which video reader, depending on network conditions. This means nsDASHReader needs to have access to metadata associated with each stream - probably I'll keep the metadata and decision in DASHMediaResource (which has the manifest data), and just return or pass an index to nsDASHReader.
>
> -- according to DASH/WebM spec, cuepoint timecodes and clusters are aligned, so we can use the existing model for HTTP byte range requests to seek the right data. I'm noticing in the code that we just seek the start though, and just pull data from there. I might want to revisit this later to pull a specific range for each request, but it seems fine for DASH/WebM and for what we want now.
>
> This looks like a good approach, solving many of the 'proxy' issues I was facing. I'll work on prototyping this next week.
>
>> nsWebMReader does interact with its nsBuiltinDecoder at times, so you may have to have a proxy to manage that interaction, or refactor the nsBuiltinDecoderReader interface/implementations to remove the reader -> decoder communication.
>>
>> Hope that makes sense.
>
> I think it does. Let me know if my response seems congruent with what you meant.
>
> Thanks,
> Steve.
> _______________________________________________
>> nsWebMReader does interact with its nsBuiltinDecoder at times, so you may have to have a proxy to manage that interaction, or refactor the nsBuiltinDecoderReader interface/implementations to remove the reader -> decoder communication.
>>
>> Hope that makes sense.
>
> I think it does. Let me know if my response seems congruent with what you meant.
>
> Thanks,
> Steve.
> dev-media mailing list
> dev-...@lists.mozilla.org
> https://lists.mozilla.org/listinfo/dev-media
0 new messages