Multiple audio and text implementation

[Grant Glendinning]

Hi

I have created some MPD's with non-DRM content using GPAC/MP4Box. These contain a single video track, 2 or more audio tracks and 2 or more text (i.e. subtitle) tracks. It is entirely possible a video may have different audio and subtitle tracks in my instance, so is something I shall need to cater for. I am running the Shaka v2 beta which seems to make a lot of older posts irrelevant to me.

I am currently having some difficulty implementing some form of UI controls for these, as I have attempted to replicate the demo option list for selecting the audio and text tracks, but these are not being populated. 

Furthermore, with regards to handling the language aspect, I followed a link to the docs Language guide from this thread to the docs page on language which returns a 404.

If i call player.getTracks(); and log the length, the player is picking up the appropriate number of tracks, so that is now some progress (after suffering several days of broken MPD's!), but am now lost on how to populate the list. I have hunted through the demo files and cannot find how this is happening.

Anybody got any pointers with this one?

Thanks

[Joey Parrish]

Hi Grant,

The "Language" page is still in the v1 docs: http://v1-6-5.shaka-player-demo.appspot.com/docs/tutorial-language.html

If you find it useful, we can update it for v2 and make sure it's included in our next release.

Here's the bit about how Shaka chooses the initial language for you based on user preferences:

When starting a video, the Player will choose streams which best fit the user's language preference. This uses a fuzzy matching algorithm for languages.

 

An exact language match is preferred ("en-US" ≡ "en-US"), followed by the base language of the user's sublanguage ("en" ⊃ "en-US"), followed by other sublanguages with the same base language ("en-GB" ≈ "en-US"). If none of these fuzzy matches work, the first available language from the MPD is used.

In v2, there are separate prefs for audio and text language, and both are configured through player.configure().  Links:

Configure overview tutorial: http://shaka-player-demo.appspot.com/docs/api/tutorial-config.html

Configure API doc: http://shaka-player-demo.appspot.com/docs/api/shaka.Player.html#configure

Config structure doc: http://shaka-player-demo.appspot.com/docs/api/shakaExtern.html#PlayerConfiguration

The v2 demo app sources are divided up by the different sections of the UI.  Track selection is under "Info", so you want https://github.com/google/shaka-player/blob/master/demo/info_section.js

In summary, to populate a track-selection UI, listen for the 'trackschanged' event from the player and call player.getTracks().  The list of tracks returned is all tracks (video, audio, text).  Use track.type to see what type a track is.  Sort the tracks, split them up into their various UI elements by type, and populate the UI.  In our demo, we sort by language, then bandwidth.  When a new track is selected, we call player.selectTrack().

One more note on v2 language support.  player.selectTrack() is essentially manual adaptation, so it disables automatic adaptation.  If you want to change languages without manually adapting, you can change language preferences through player.configure().  The player's language selection algorithm will be rerun and appropriate tracks will be selected, but adaptation will not be disabled.  In this way, you could provide a language-selection UI (show unique track.language values) instead of a track-selection UI.

Does this help?

-Joey

--
You received this message because you are subscribed to the Google Groups "Shaka Player Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to shaka-player-us...@googlegroups.com.
To post to this group, send email to shaka-pla...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/shaka-player-users/2aca6a0c-ab11-46d8-9a0c-6e35d8d31e71%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Grant Glendinning]

Hi Joey

That is perfect and should be enough to create the desired effect.

I had a brief read over the language detection and am unsure if this is an issue with how my MPD's are set up, the browser or the Shaka player. 

I created with the MPD and used the English audio as the first audio and Arabic as second using GPAC so are 2 letter language codes (en and ar). I set up a VM in Arabic and checked the language settings for the OS and browser do report 'AR' as the detected language. I therefore expected(?) the Arabic audio to play by default with this Arabic browser, but this isn't the case, the English audio continues to play. I tried to have a look over the language detection docs but this returns a virtually blank page here.

I'll have a read over the docs further to ensure I haven't missed anything obvious but in the meantime, thank you, I've no doubt I'll be back again soon!

Thanks again

Grant

[Joey Parrish]

Hi Grant,

The Shaka demo app populates the language settings using "navigator.language" in demo/main.js:

  var language = navigator.language || 'en-us';

  document.getElementById('preferredAudioLanguage').value = language;

  document.getElementById('preferredTextLanguage').value = language;

These values are passed to player.configure in demo/asset_section.js:

  config.preferredAudioLanguage =

      document.getElementById('preferredAudioLanguage').value;

  config.preferredTextLanguage =

      document.getElementById('preferredTextLanguage').value;

  player.configure(config);

The player will choose the audio track based on these settings.  Shaka understands both two- and three-letter language codes, so even if your preference is set to "ara" and your manifest says "ar" (or vice versa), it will still match.

So in your case, since English is playing in the VM, I would guess one of these is happening:

1. Your browser is not correctly reporting the OS's configured language

    (check the value of navigator.language in the JS console)

2. Your application is not using the browser language to configure the player

    (check player.getConfiguration().preferredAudioLanguage in the JS console)

Let me know how this turns out.

Thanks,

Joey

--

You received this message because you are subscribed to the Google Groups "Shaka Player Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to shaka-player-us...@googlegroups.com.
To post to this group, send email to shaka-pla...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/shaka-player-users/0638546f-1349-4a4b-b37f-f7ca0feead6e%40googlegroups.com.

[Grant Glendinning]

Hey Joey

I believe you have changed the way Shaka handles tracks in v2. getTracks now returns every type of track and I have (so far) been unable to achieve what I'm looking to do. Seems getAudioTracks() has been deprecated so am at a loss.

Essentially, I want to use something like this initially to identify the number of track types:-

var myAudio = player.getTracks('audio');
console.log(There are ' + myAudio + 'audio tracks');

// Expected output eg. 'There are 4 audio tracks'

var myText = player.getTracks('text');
console.log(There are ' + myText + 'text tracks');

This will hopefully allow me to be able to go on further and hide the select lists if there is only one audio track, or hide the text track select list if there are no text tracks, etc.


BTW, not sure where to log this, but I have found that when "The last segment should not end before the end of the Period" log is thrown, playback fails in OS-X. However, if I seek, say 1 minute into the video, playback automagically resumes. This is the only thing I was not expecting to see in the console, so suspect this caused the playback failure somehow. I fixed the problem causing this (audio tracks were 0.2 seconds longer than video track) and now have no issue in OS-X (Safari or Chrome), although it seems strange Chrome on Windows 7 did not have an issue and played the DASH just fine before and after the track length fix.

Thanks again.

On Monday, 23 May 2016 12:45:41 UTC+1, Grant Glendinning wrote:

[Joey Parrish]

Hi Grant,

Yes, we changed the API in v2.  Instead of getAudioTracks(), getVideoTracks() and getTextTracks(), we now offer getTracks() which returns all tracks.  This simplified away some unnecessary code that we had in v1.

You can easily filter the Array using Array.prototype.filter:

var tracks = player.getTracks();

var audioTracks = tracks.filter(function(t) { return t.type == 'audio'; });

var videoTracks = tracks.filter(function(t) { return t.type == 'video'; });

var textTracks = tracks.filter(function(t) { return t.type == 'text'; });

The code for populating the track lists in the demo app's UI is in demo/info_section.js, in the method shakaDemo.onTracksChanged_.  The 'trackschanged' event from Player is essential if you might need to handle multi-Period content, as the list of tracks will change when the Period changes.

Thanks,

Joey

--

You received this message because you are subscribed to the Google Groups "Shaka Player Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to shaka-player-us...@googlegroups.com.
To post to this group, send email to shaka-pla...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/shaka-player-users/bcc4e5c8-1c46-4aba-bef1-04add26b69f5%40googlegroups.com.