HLS Streaming option failing on every video, multiple link examples provided

[tom lang]

So for a while I've downloaded on this particular site with no issues but now for some reason the HLS streaming option just fails every single time no matter the video and that'll be the only option available for a lot of the videos. 

Windows 10 Professional 64 bit
Firefox 107.ob4 (also doesn't work on latest Google Chrome browser)
VDH 7.6.0 (no license on Firefox but license on Google Chrome where this same issue occurs)
VdhCoApp 1.6.3

Example link 1: https://d a f t s e x . c o m /watch/-175547611_456239018
Example link 2: https://d a f t s e x . c o m/watch/-214061752_456239103
Example link 3: https:// d a f t s e x . c o m/watch/-214061752_456239103

Basically any video on this site with the HLS streaming option will fail.

[tom lang]

Also thank you Wild Willy for letting me know my previously created thread was being marked as abuse for whatever strange reason. I would've never known had you not notified me thank you brother.

[Wild Willy]

You may think I am driven by higher principles, that I'm the epitome of altruism. Nah.
I'm just here trolling for content like yours to see if I'll like it. Sadly, this stuff
is not to my taste. But you can keep trying. I'll appreciate it. You might eventually
stumble upon something I'd like. I could share some of what I like with you but it's
behind a paywall. Oh well. Life is full of such tragedies.

So once I dropped my smart aleck persona, I did try to download the first one you
pointed to. I did find one HLS item listed on the VDH menu & I tried to download it. It
sat there at 0% for about 5 minutes & then failed with a 404 Not Found error. I even
tried turning on the HLS as M2TS setting in VDH. That sometimes helps. This time it
made 0 difference. Time for plan B:

https://groups.google.com/g/video-downloadhelper-q-and-a/c/sNfTCMYfiTU

Usually, HLS manifests show up in the Network Monitor as type x-mpegurl. Here, the
manifest showed up as type application/vnd.apple.mpegurl. Oh well. At least the file
name contained m3u8 so I could tell what it was. But running that through ffprobe showed
it reading every chunk in the stream, all 396 of them, & getting a 404 Not Found error on
every single one. I looked at the contents of the manifest using Copy Response. It
looked like a stream manifest. I was expecting a master manifest. In any case, it
looked like ffprobe was trying to read every chunk in this manifest & getting 404 on each
one. This was discouraging.

Up to this point, I had not clicked the play button in the video player on the web page.
I thought maybe that would give me something to work with in the Network Monitor. So
after I waded through & dismissed a few dozen ads, I managed to get the thing to start
playing in the web page. This did not add another manifest but it did start adding a
string of mp2t entries to the Network Monitor. I tried double clicking the first one &
that just made Firefox open a new tab in which I got the dialog asking me what viewer I
should use to open the object, or whether I wanted to download it. When I told it to use
Firefox to view it, it just gave me the same dialog box again. On a hunch, I told it to
open the file with VLC. To my huge surprise, VLC actually played the fragment. At that
point, I thought this has to be possible to download.

Looking carefully at the URLs of the mp2t fragments, I noticed that each one ended with
these items:

?extra=kO5DuIGRB3ziILndaMOStg&extra_info=1ssEFv6stfAx6HUBAP2XBxo

These strings are MISSING from the manifest. I am quite certain those gibberish letters
& digits will NOT work for anybody else. My assumption is that they are somehow randomly
generated by the site when I surf to the page. So the strings you will get will be
different when you do this. In any case, I figured I could manipulate the manifest as a
file on my system to correct the omission from the manifest. Since I had done the Copy
Response earlier, I just saved that as a file on my system. For my first step of repair,
I prefixed every line of the manifest with the beginning part of the URL I saw in the
Network Monitor for the chunks. This prefix part was:

https://psv158-1.crazycloud.ru/videos/-175547611/456239018/360/

The lines in the manifest for the chunks all looked like this:

seg-x-v1-a1.ts

The x in there started at 1 and ran to 396. So using the search/replace all function of
my text editor (Notepad++ in case anybody cares, but I think you can also do this in
Notepad & Wordpad), I modified those 396 lines to read this:

https://psv158-1.crazycloud.ru/videos/-175547611/456239018/360/seg-x-v1-a1.ts

The next step of repair involved suffixing each of these lines with the rest of the URL
shown in the Network Monitor, the gibberish part. Once again, using the search/replace
all function of Notepad++, I corrected each relevant line of the manifest to look like
this:

https://psv158-1.crazycloud.ru/videos/-175547611/456239018/360/seg-x-v1-a1.ts?extra=kO5DuIGRB3ziILndaMOStg&extra_info=1ssEFv6stfAx6HUBAP2XBxo

(That's a long line so it probably displays folded on 2 lines. But in reality, it's a
single line.)

There are other lines in the manifest so you have to be careful to change only the lines
that contain the partial URLs.

I saved this repaired manifest on my system. Then I fed it to ffmpeg. The download
began with Resource Monitor showing a rather poor download speed. But it gradually sped
up to about 1.25 million bytes per second cruising speed. After just under 3 minutes, I
had an MP4 on my system of size 150M, duration 32:55, resolution 640x360, frame rate
30fps. Crappy resolution. But it played in VLC just fine. Like I said, this isn't to
my taste so I just skimmed through it & it did appear to be all here, video + audio,
right to the end.

Interesting side note. This video contained timed_ID3 data, something I was unable to
discover ahead of time because the original manifest was broken & I didn't think to run
ffprobe on my saved manifest before using it to download the video. Fortunately, ffmpeg
has no idea how to handle timed_ID3 data & it simply ignores it. Timed_ID3 data is known
to cause all manner of problems for VDH so even if the manifest had been usable, you
probably would have had to revert to ffmpeg anyway to avoid the timed_ID3 data.

At that point, I noticed that every URL in the manifest has this qualifier:

/360/

I managed to eventually get the tool menu to open in the player on the web page. They've
got some very strange effects going on in their video window that make the controls go
out of sight just as your mouse cursor gets near the gear. But I managed to eventually
get the gear menu open & I saw that the highest resolution on offer here is 720p. That
would be 1280x720. So I change the /360/ in each line of the manifest to /720/. This is
a complete off the wall guess. There is no logic here, just intuition. Using this
second repair to the manifest I got a bunch of 404 Not Found errors. Not willing to give
up, I reloaded the page & set the resolution to 720. I got a new manifest & new mp2t
chunks. Turns out I was close. The URLs did all have /720/ in them but the suffix part
had different gibberish strings in it. With this second repaired manifest, I did get a
1280x720 video after a download lasting not quite 6 minutes. Of course, it played fine
as well. The image is noticeably sharper. The Windows Properties even show a higher bit
rate for the audio, although I'm not sure that's particularly noticeable. We're not
talking about a symphony or an opera here.

About my repairs to the manifest. Prefixing every partial URL with the site name (and
some other qualifiers) is something I had to do because the manifest was on my system.
Generally, when a manifest resides on a server, ffmpeg knows to prefix each partial URL
for a chunk with a section of the URL of the manifest. Many manifests include complete
URLs but many manifests include only partial URLs. If ffmpeg encounters a URL that
doesn't start with https://, it knows to take the extra step. But since the file was on
my system, I had to provide complete URLs in the manifest. So that step isn't so much a
repair as it is just something you have to do when you use a manifest that resides on
your system.

The suffix looks to me like a maneuver by the web site purposely intended to thwart
downloaders. Clearly, the video player embedded on their web pages knows about the
extra= & extra_info= tokens & appends them to the chunk URLs as necessary. In my
opinion, the manifest is purposely lying to the user expressly to prevent downloading.
That's why VDH failed with 404. The chunks really were not found. The URLs in the
manifest are bogus. I consider this way too much trouble to circumvent. I'm not even
thinking about trying your other 2 videos. I'm sure they're the same.

But now you'll tell me but it used to work. No doubt. Tough. They changed their way of
doing things. It's a regular occurrence around the web. This forum is littered with
people saying but it used to work. Used to work is ancient history, even if it's since
this morning. It doesn't matter in the least what used to work. You need to forget what
used to work. It only matters how things are now. I say there is no bug in VDH & Michel
should spend exactly 0 time investigating this. This site took measures to make it next
to impossible to download their content. You'll note that the site address in the URLs
shows .ru. Russia. I wouldn't expect anything from there to follow any sort of
standard. I'm actually a bit surprised that I was able to navigate the maze they
presented me. It was fun. But I don't think anybody but the most dedicated & desperate
user will actually go to the considerable trouble of following the steps I've outlined
here. If I were a regular visitor to this site, I would give up & move on.

[mjs]

I don't think I would have figured out how to do all of that but when I went to one of those links on the mobile phone there was a download

button on the video player. Viewing it from the computer there wasn't the button I saw on the mobile.

So to make it work on the computer, add an extension to change the user agent to an android device then go to the site.

It will then have a download button on the video player.