When VDH shows separate video & audio, it is a bug

[Wild Willy]

We first encountered VDH giving variants that were video without audio & audio without video about 3 years ago.  That was in connection with the free nightly opera streams the Metropolitan Opera was giving us while they were closed due to COVID-19.  But in 3 years I have learned some things.  That may surprise you but there it is.  I have actually learned some things.  I agree.  It's a miracle.

I have mentioned this in a number of other threads.  I think in order to catch Michel's attention, I need to make this a thread of its own.  I have come to the conclusion that the presentation of video & audio in separate variants is a bug in VDH.  I can describe when the bug manifests itself (pun intended).  I hope this makes it clear to Michel what he needs to fix in VDH to handle this situation properly.

It is common for a master manifest (HLS or DASH, doesn't matter) to be structured as I'm about to describe.  My terminology here is based on what ffprobe reports when I give it the master manifest.

Program 0
 Stream 0:0 - audio
 Stream 0:1 - video
Program 1
 Stream 0:2 - audio
 Stream 0:3 - video
Program 2
 Stream 0:4 - audio
 Stream 0:5 - video
Program 3
 Stream 0:6 - audio
 Stream 0:7 - video
Program 4
 Stream 0:8 - audio
 Stream 0:9 - video

With a simple structure like this, VDH correctly recognizes 5 variants & presents them in descending order of video resolution, which is not always the same sequence in which they appear in the manifest.  If a particular resolution happens to appear more than once, VDH will present those repeated resolutions in descending order of BANDWIDTH, a parameter that almost always occurs in a manifest, indicating descending order of quality within a given resolution.  But you can download each variant with VDH & it will give a result that has BOTH video AND audio.

But that is not the only structure of manifest that is common around the web.  You could also have something like this, which is very much like how the operas were:

Program 0
 Stream 0:0 - audio
 Stream 0:1 - video
Program 1
 Stream 0:2 - audio
 Stream 0:3 - video
Program 2
 Stream 0:4 - audio
 Stream 0:5 - video
Program 3
 Stream 0:4 - audio
 Stream 0:6 - video
Program 4
 Stream 0:4 - audio
 Stream 0:7 - video

In other words, video Streams 0:1 & 0:3 are associated with their own unique audio Streams 0:0 & 0:2 respectively.  But video Streams 0:5, 0:6, & 0:7 share the audio Stream 0:4.  VDH completely misinterprets this.  VDH gives us 5 variants of video without audio & one variant of audio without video.  Yes, just one audio variant even though there are THREE distinct audio variants in the manifest.  I have not figured out which audio variant VDH chooses.  Presenting us with video-only & audio-only variants is a bug in VDH.

Or this:

Program 0
 Stream 0:0 - audio
 Stream 0:1 - video
Program 1
 Stream 0:0 - audio
 Stream 0:2 - video
Program 2
 Stream 0:0 - audio
 Stream 0:3 - video
Program 3
 Stream 0:0 - audio
 Stream 0:4 - video
Program 4
 Stream 0:0 - audio
 Stream 0:5 - video

In other words, all 5 video Streams share a single Audio stream.  At least there is no confusion about which audio without video we're getting in the audio-only variant that VDH presents to us.  But once again, VDH completely fails to understand what it is being given here.  This is the same bug in VDH.

I've even seen this once:

Program 0
 Stream 0:0 - audio
 Stream 0:1 - video
 Stream 0:2 - video
 Stream 0:3 - video
 Stream 0:4 - video
 Stream 0:5 - video

I think VDH messed this one up completely.  I don't rightly remember.  I used ffmpeg to get this one.

Now, some videos really don't have any audio, like maybe old silent movies.  Some videos aren't videos at all, but rather just radio broadcasts or something of that nature.  These are cases in which VDH would be correct to offer variants that are video without audio or audio without video.

But the vast majority of content on the web has BOTH video AND audio.  If you can visit the web page, click the play button in the player there, & both see images & hear sound, then it is ALWAYS a bug in VDH if it shows any variants that are video without audio or audio without video.  These cases should ALWAYS get VDH to show us ONLY variants that have BOTH video AND audio.  That way, we can do a single download in VDH & get a complete video that actually has audio.  The video & audio ARE NOT SEPARATE.  You need to lose that train of thought.  EVERY video HAS audio.

Maybe YouTube confused the issue because they really do present their video & audio separately.  But YouTube is not the web in general.  YouTube is an anomaly, presenting a unique case.  No other web site does their content like YouTube.  No other web site that offers either HLS or DASH streaming offers anything that you should interpret as video-only & audio-only.  That is bogus.  All the video has audio.  Ffmpeg recognizes the true situation.  You're just not recognizing it.  It is a bug in VDH.

[mig]

Having a proper, non-Youtube, URL where to investigate would greatly help. Any free link not requiring registration ?

[Wild Willy]

I am VERY glad you are looking into this, Michel.

This recent thread had an example:

https://groups.google.com/g/video-downloadhelper-q-and-a/c/t4Bu_gQUH1w

The videos were on qvc.it. A link is in the thread.

This thread had something:

https://groups.google.com/g/video-downloadhelper-q-and-a/c/BpKHA_SQk1o

I came up with this URL of a master manifest:

https://ssaimanifest.prod.boltdns.net/us-east-1/playback/once/v1/hls/v4/clear/6150020952001/479edf58-3ecb-4c74-a651-e55972648e2d/0008b22c-bb15-4c3b-a854-9d780ce6f144/fbc4c449-c62b-485d-86c6-82557048259c/content.m3u8?bc_token=NjQ0MjU1OWRfOWRlYjI2YTY3NTNkMzMxNTU3YTJlODU5YWQ2OGI5MTdmN2I2OTcxODM5MTU3ZmE2OTgxNDdmN2Y1OWFjZDUxMQ%%3D%%3D&rule=discos-enabled

That won't help you. It's full of the random character strings that uniquely tie the
manifest to my session, so by the time you read this, it will have expired. I got this
URL from the Firefox Network Monitor on this video:

https://video.telequebec.tv/player/39736/stream?assetType=episodes&playlist_id=190

The manifest is one of those where most of the Programs have unique audio/video Stream
pairs, but the last 2 Programs include a shared audio Stream. I suspect this web site
has a number of videos you could hunt down that exhibit the pattern.

I hope that's enough to get you started. I found these threads by searching the forum
with "no audio" as my search key. There's a lot of threads that were not relevant, but
every so often you'll hit a useful one. That's how I came up with these 2. If you need
more, you could just as easily do the search as I could.

[Wild Willy]

Here's an interesting master manifest provided by a fellow user in this thread:

https://groups.google.com/g/video-downloadhelper-q-and-a/c/LJy7x8DQG28

I'm attaching the ffprobe report because I believe this is one of those expiring items.
The URL is full of the typical gibberish strings that associate a URL with a web session.
I think I was lucky to have found this thread before the URL expired, so I'm attaching
the ffprobe report on the assumption that by the time you read this, Michel, the URL
won't work.

[Wild Willy]

Here's one that looks to me like VDH should have handled it no problem:

https://groups.google.com/g/video-downloadhelper-q-and-a/c/fLjFBxS6Ka8

[Wild Willy]

This rather interesting case has just had some new activity:

https://groups.google.com/g/video-downloadhelper-q-and-a/c/4Apw_vv9-Io

It shows a case in which a video was dubbed into multiple languages.  These alternate languages are presented as multiple audio Streams within the Programs.  Each dubbed track appears in every Program.  This case also shows multiple subtitles, also repeated in every Program.  It would be nice if you could have VDH offer variants that clearly label the language of each dubbed audio.  That information is quite obviously carried in the master manifest in use in that case.  I know.  Not every manifest includes that kind of detail.  But this one does, and I've seen others that do.  So I'm recommending that you code conditional logic to take advantage of this sort of information when it's present.  It might not be present, but when it is, you should use it.  And then there's the issue of subtitles.  I know this has been brought up a lot of times before.  But that just means it would be an immensely popular enhancement to VDH.  Do please consider it well.