why Dash 2.3.0 do not prune BUFFER_AHEAD?

[Eric Li]

Hi Guys,

 This is Eric Li from Vudu. I have made Dash.js 1.6.0 worked on embedded platform, such as Samsung Tizen TV.

 Now I am updating 1.6.0 to 2.3.0. And met a memory issue.

In Dash 1.6.0, dash player will prune the data which is more than BUFFER_AHEAD_TO_KEEP time later than current playback time.

However in Dash 2.3.0 (maybe since Dash 2.0), dash player do not do this prune any more.

Any reason to remove this prune?

This will lead to too much memory allocated.

For example, our app have a chapter jumping function and instant replay which will playback 15 seconds before current playback time. Without this prune, it will let the buffer keep on rising once I keep on doing instant replay.

Thanks,

Eric

[bernhard...@maxdome.de]

Hi Eric,

Rik already posted this issue some time ago -> see https://groups.google.com/forum/#!topic/dashjs/8jbgnZsb3xU

Unfortunately I still did not have the chance to dig into this any deeper. But some thoughts from my side since my last comment:

In my opinion, automatic pruning of buffer ahead is more dangerous than pruning buffer in the past. Why? Because in a normal user flow the playback position always comes back to the already pruned buffer ahead -> this means multiple download of media chunks -> increased CDN traffic costs. There is also the chance that you get into oscillation: your buffer is filling very aggressively on fast internet connection -> buffer controller tries to prune the buffer ahead and on the other hand it tries to fill up the buffer again. I do not say it is impossible but it's more complex than pruning buffer in the past.

We are currently also evaluating dash.js on low performance devices such as set top boxes and SmartTVs and also experienced some major performance issues which are blocking us deploying it on productive system.

Let me know what you are thinking about it. We appreciate every input from your side and I hope that we can contribute back some performance improvements the next weeks/months.

Thx,

Bernhard

[Lloyd Wallis]

When we originally implemented this feature it was necessary because frequent seeking could result in a broken experience.

Essentially, if we leave pruning to a point where a browser’s automatic cleanup activates, it tends to remove content from the earliest point in its history – so if you seek back it tries to prune content around the time you’re trying to play. I think we had a discussion around this at some point a while back and considered focusing on removing discontinuous ranges ahead of the playhead.

Hopefully the CDN costs should not increase as the browser’s native cache can still keep the previous network requests.

--
You received this message because you are subscribed to the Google Groups "dash.js" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dashjs+un...@googlegroups.com.
To post to this group, send email to das...@googlegroups.com.
Visit this group at https://groups.google.com/group/dashjs.
For more options, visit https://groups.google.com/d/optout.

 

----------------------------

http://www.bbc.co.uk
This e-mail (and any attachments) is confidential and may contain personal views which are not the views of the BBC unless specifically stated.
If you have received it in error, please delete it from your system.
Do not use, copy or disclose the information in any way nor act in reliance on it and notify the sender immediately.
Please note that the BBC monitors e-mails sent or received.
Further communication will signify your consent to this.

---------------------

[Eric Li]

Hi Bernhard,

  Thanks for your explain.

  In reality world, I do not think CDN traffic cost is the main concern. And for embedded system, memory control is more important. Many embedded device, they will limit one process total memory allocation. And before the browser trigger its insufficient memeoy API, it is already stuttered or freezed.

 Dash.js itself limit the memory allocation is essential to us.
 

  (1)First, we should have buffer target which should not be too much, and not large than our prune ahead time. So in normal playback, we will not get into oscilation which you described. Only the seek backward operation will trigger this kind of prune ahead operation.

  (2) I totally agree with you, automatic pruning of buffer ahead is more dangerous than pruning buffer in the past.

       My understanding, the prune job in 1.6.0 is reasonable. We prune just after MSE buffer appended instead of in wall clock ticks

  (3) I do not know how much benefits for Dash.js have different buffer time for , normal quality video, top quality video, and long form video. Just like I said, in normal playback case, which is 99% percent in real world, prune ahead time buffer, will not be triggered.

 (4)Even you only focus on removing discontinuous ranges ahead of the playhead, you still need a BUFFER_AHEAD_TIME, and only remove those discontinuous buffer later than this time. However only remove discontinuous range will not fit our instant replay function, it will seekback 15 seconds, then you play a while, then do it again, it is continuous buffered data and keep on rising.

 Overall, my conclusion is, from our user point of view, limit memory usage and let the player working smoothly is more  critical than to let bandwidth more efficient. and in 99% real user case(no seek, or seek forward is more popular operation than seek backward), your unlimited buffer did not save CDN transaction, just only buffer it early, rather than later.

Thanks,

Eric

--
You received this message because you are subscribed to a topic in the Google Groups "dash.js" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/dashjs/Y-4X6LnCCZs/unsubscribe.
To unsubscribe from this group and all its topics, send an email to dashjs+unsubscribe@googlegroups.com.

[Sparacio, Daniel]

I think we should implemented something that only prunes on a seek back in time, from seekpoint + forward buffer target (manageable with API ) .  This should be safe zone and all else should be pruned.  Seems like a great memory management feature and should not be hard to implement at all. 

 

Eric any cycles you can try to implement.  I have no idea when I would get to it.

 

Dan

To unsubscribe from this group and all its topics, send an email to dashjs+un...@googlegroups.com.

To post to this group, send email to das...@googlegroups.com.
Visit this group at https://groups.google.com/group/dashjs.
For more options, visit https://groups.google.com/d/optout.

 

--

You received this message because you are subscribed to the Google Groups "dash.js" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dashjs+un...@googlegroups.com.

[Eric Li]

I am done the code changes.

I need tons of test on it to avoid any bad side effects

Eric

To unsubscribe from this group and all its topics, send an email to dashjs+unsubscribe@googlegroups.com.

To post to this group, send email to das...@googlegroups.com.
Visit this group at https://groups.google.com/group/dashjs.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "dash.js" group.

To unsubscribe from this group and stop receiving emails from it, send an email to dashjs+unsubscribe@googlegroups.com.

[bernhard...@maxdome.de]

I think there is no big disadvantage in deleting continuous buffer as well as long as you are not hitting the browser internal limit in order to avoid letting the browser deleting buffer near the actual playhead like Lloyd already mentioned.

But you have to take care of the different buffer targets: normal quality and top quality. You should take the maximum of them for calculating a safe zone to prevent oscillation.

Pruning buffer only after MSE buffer appended instead of in wall clock ticks sounds also very reasonable for me.

Browser cache for saving CDN costs is only valid if you did not change the quality level, but as long as you are very careful in preventing oscillation, I think it is not that big deal.

BR

Bernhard

To unsubscribe from this group and all its topics, send an email to dashjs+u...@googlegroups.com.
To post to this group, send email to da...@googlegroups.com.

Visit this group at https://groups.google.com/group/dashjs.
For more options, visit https://groups.google.com/d/optout.

[Eric Li]

My changes are done and at least my own test did not show any problems until now.

I put dash.js 2.3.0 + my customization and changes into one of our products.

the  I will push out to our QA for a full certification test.

I agree with you, my buffer ahead is maximum of all the buffer target + a safe zone.

What I prefer, from design point of view, dash.js itself have a predicable buffer limits, and keep it by dash.js, instead of trigger browser its internal limits is much better design for memory limited embedded environment and is much more portable to different platform.

And if user do not do too many backward search operations, most users do not do that in real world, CDN cost will not increase.

Eric