Recording the CBS TV livestream - HLS as M2TS

[Wild Willy]

I've just done a little experimenting with the CBS TV livestream. This is not CBS
Sports, just plain old CBS, the network you can get for free over the air with an
antenna. You don't need a cable subscription or a satellite dish, just a plain old TV &
an antenna. Nevertheless, when I tried looking at this, it did ask me to validate my
cable subscription, which seems a bit anomalous to me, but that's what I got. The URL is
this:

https://www.cbs.com/live-tv/stream/

I tried using my alternative technique to record the channel with ffmpeg. Despite
ffprobe on the HLS manifest telling me all about the streams on offer, when I tried to
record it with ffmpeg, I immediately got 403 Forbidden - access denied. VDH, on the
other hand, had no trouble. I would really like to know what magic Michel does to let
VDH record stuff that ffmpeg just errors out on.

In any case, when I recorded a few minutes with VDH, it seemed to be recording but the
file was not growing despite Resource Monitor showing plenty of line activity. I clicked
the stop button in the VDH status menu (the blue dot) but that just generated an error
about M2TS no data received. I thought that was odd but I've had trouble with that
button before. So I tried again but this time, instead of using VDH to end the
recording, I used my so-called safe technique. That is the one you can find over here:

https://groups.google.com/g/video-downloadhelper-q-and-a/c/BzPLK2YyL-s

Do a string search for "safely terminating" in there to see what I did. In any case,
when I did that, I still didn't get a usable file to play back. VDH ended the download
with the same error.

So I took my cue from the VDH error message & enabled the VDH Setting for HLS as M2TS.
This is disabled by default. Once I enabled this VDH Setting, VDH started recording the
CBS livestream. After about 3 minutes, I used the blue dot menu to stop the recording.
Even though this failed for me earlier, I tried it again just to cover the bases. I was
surprised when it seemed to work. VDH popped up its notification that my file was ready.
I had a small .m2ts file. Rather ominously, the Windows file properties were mostly
blank. OK, Windows doesn't have built-in support for that file type. It's not the only
such file type. So I ran the file through ffprobe. That did tell me I had a 1920x1080
video, 60fps, 10206 kb/s video bit rate (rather high quality video, I must say), with
audio properties 48kHz sampling, stereo, 128 kb/s (pretty standard quality audio). The
ffprobe information was much abbreviated compared to what I'm used to seeing with MP4s
but at least it showed something. It played fine in VLC. I want to emphasize that. VLC
plays M2TS files as is without conversion, although Firefox can't play them. There was
even a closed captions track that I could turn on.

I was surprised a while later to discover that I had a zombie process in VDH. I still
had a blue dot & VDH seemed to think it was still downloading something, even though
Resource Monitor showed no activity & by then, I had actually deleted the .m2ts file. I
had to do a reload extension function (string search on "reload the extension" in the
Table of Contents thread) to clear the condition.

I wish there were some way to know when you have to switch on the VDH Setting for HLS as
M2TS. Better yet, I wish VDH would figure it out & just do it automatically.

[Wild Willy]

I experimented some more with this & I have been able to use ffmpeg to record this livestream.  There's a couple of odd dodges you have to do.

It seems their master manifests expire.  When that happens, you get the 403 error I mentioned above.  But you can get a new master manifest by simply reloading the web page & clicking the play button again.  You will get a new master manifest.  You will also get an ever lengthening list of subordinate stream manifests.  Ignore those.  The master manifest is easy to identify because it is the first one in the Network Monitor & gee what a surprise it has the string "master" in its URL.  Just run the master manifest through ffprobe & look at those results.

Normally, you would run ffmpeg with -i containing the URL of the master manifest.  You would also supply -map parameters for the video & audio streams you want.  The master manifests I have looked at for this site usually contain several choices for video resolution.  Rather unusually, there are 2 choices for 1920x1080 resolution, one at 30fps, one at 60fps.  In addition, any time I've looked at this content, it has always included timed_ID3 data, something I have discussed several times elsewhere in this forum.  This is a type of stream you want to avoid.

So you don't want to do the usual thing.  Instead, carefully inspect the ffprobe output & decide which one of the several Programs you want to record.  Then look carefully at the earlier part of the ffprobe output, the part you would normally just skip over.  You need to select the URL of the second-level stream manifest corresponding to the Program you want to record.  Fortunately, ffprobe lists these URLs in the same order as it reports on them at the end of the output.  Pass that second-level stream manifest URL into ffprobe & get a new report.  This is partly to verify it's the right livestream, partly to verify the stream IDs you want to record.  You will usually want 0:0 & 0:1 but it's always good to check.  Since there is timed_ID3 data, you will need to code -map parameters in order to avoid this unwanted stream.

Finally, to record the stream, run ffmpeg with -i being the URL of your selected second-level stream manifest.  Supply -map parameters, usually 0:0 & 0:1, so you don't pick up the timed_ID3 data.  I believe ffmpeg would ignore timed_ID3 streams if you coded no -map parameters, but I like to be safe.

I have tested this with only short recordings, about 3 minutes long.  I assume this approach would work with something longer, like a 3-hour golf broadcast or a football game or something like that.  To be complete, I suppose I should reserve a small doubt in my mind until I do record something longer.  You never know what goes on with livestreams.  One manifest might hand off to a later one & ffmpeg might or might not smoothly handle the transition.  If I ever do record a more lengthy livestream from CBS, I will post again to report my experience.

[Wild Willy]

On further experimenting, it turns out you don't need to go through the deal with the
subordinate manifests.  You can do the usual ffmpeg approach with -i taking the master
manifest & -map values selecting the video & audio streams you want.  The only trick is
they are very prone to 403 Forbidden errors.  Just reload the page & get a new master
manifest until the 403 error goes away.  This may take you 2 or 3 extra page reloads
before you finally get a master manifest that stays valid long enough for you to launch
your recording.