Is VDH supposed to be able to record a live stream?
378 views
Skip to first unread message
Wild Willy
Jul 9, 2016, 1:05:10 AM7/9/16
to Video DownloadHelper Q&A
I ask this question because I've tried to do it a few times & it looks like it's working but in the end it doesn't. The latest example was this morning. I went to usga.org where they had a live stream of some bonus coverage of the US Women's Open Golf tournament. I started VDH downloading the stream & it sure looked like it was saving something on my hard drive. But after the stream ended after 2 hours, I had a file of about 14Meg on my hard drive. It was one of those part files. But when I tried to open it in VLC, it just sat there doing nothing. No error message, but no video playing either. VDH never gave me any error messages. I checked it a few times while the live stream was active, dropping down the VDH menu, and it always showed an entry in the list with the background shading implying it was in the process of downloading that stream. There was an elapsed time counter visible that increased every 10 seconds. And it had a little box on it like you normally get for downloading discrete videos (as opposed to live streams), the box being the way you could interrupt the download before it completes. But it all appeared to be a smokescreen because I didn't get what I wanted.
So I am just verifying whether I should expect VDH to be able to save a live stream on my hard drive as an MP4. And if the answer is that it should work, then I have to report that it doesn't.
So I am just verifying whether I should expect VDH to be able to save a live stream on my hard drive as an MP4. And if the answer is that it should work, then I have to report that it doesn't.
mig
Jul 9, 2016, 4:09:46 AM7/9/16
to Video DownloadHelper Q&A
I presume your live stream is using the HLS protocol, like Twitch.tv.
For those streams that have no natural ending, after you start recording, a .part file is being constructed containing the captured audio video data. But this file is not playable as it is missing important information to synchronize the streams. When you invoke the "Stop recording" action (this is the default action for the entry once you have started the capture), the file is finalized (synchronization data is written) and renamed to .mp4.
For those streams that have no natural ending, after you start recording, a .part file is being constructed containing the captured audio video data. But this file is not playable as it is missing important information to synchronize the streams. When you invoke the "Stop recording" action (this is the default action for the entry once you have started the capture), the file is finalized (synchronization data is written) and renamed to .mp4.
Wild Willy
Jul 9, 2016, 7:54:41 AM7/9/16
to Video DownloadHelper Q&A
Thanks for your response, Michel.
When the stream was done after 2 hours, I did click the little square on the line item in the VDH dropdown list. I assume that's the "Stop recording" action you are referring to. But nothing got finalized. My sense is that rather early on, when I was looking at the file size in Windows Explorer to see if it was increasing, it reached 14Meg and stopped. After only a few minutes, the file size stayed at 14Meg for over 90 minutes while I was trying to get VDH to record it, and it remained 14Meg after I stopped VDH from recording it further. Now I have about 6Gig of free space on that partition so VDH couldn't have stopped because it thought it didn't have enough disk space. Besides, I should hope VDH would generate an error message about running out of space if that's what happens, which it did not. I would also hope that VDH would notify me of any other reason it would have stopped recording, also which it did not.
If it is possible for you to do, I'd like you to try to record that stream. That golf tournament is on this afternoon starting at 15:00 EDT in the US eastern time zone. They are actually playing in the San Jose, California area, so that's noon local time where they are playing. Let's see . . . I believe that is actually 19:00 GMT. I have no idea where you are, as in time zone, so I'm hoping this tells you the time of day when it's going to be available in your time zone. To the best of my knowledge, usga.org will not open the live stream until that time. On the home page of that web site, there is an abbreviated score board over to the right near the top of the page. Just below the scores there's a little icon of a TV and the word "WATCH." If you click that, it will take you to the live stream page. But the stream is there only after they open it, which won't be until 19:00 GMT. I clicked it just now and it just takes you to a page of links to various news items, photo galeries, video clips, and so on. So if you can do it, I would greatly appreciate if you would try to record their live stream today. It will also be available tomorrow at the same time. According to the TV schedule, which is lower down on that live stream page, the stream will be available for 4 hours. It will probably be longer because it's a golf tournament & they always take longer than the allotted time to play, especially since this is a major & it's a particularly difficult course. I will also attempt to get VDH to record this stream today.
I have no idea if this stream is HLS. I have no idea how to determine that. In fact, I have no idea what HLS is. I have just now noticed that my VDH dropdown still shows that VDH thinks the stream is still in progress almost 24 hours after I closed it. The message on the item in the VDH menu says. "Finalizing... HLS streaming - 2:10:47 - 910 Kbps - MP4." I guess that means the stream is indeed HLS. There is also an X there. Now, I deleted the file from my hard drive yesterday when VLC wouldn't play it. So finalizing is just never going to happen.
So I clicked the X and now it says I have an orphan. No surprise. But what is surprising is that it looks like it is trying to download something. The background shows the shading moving from left to right as if it were trying to download something. Oh wait. It's gone now. That was there for only a few seconds. OK. It was just cleaning up. Very well. So I am going to try to record that stream this afternoon as an exercise, since I'm just going to watch the broadcast on TV. The live stream online will be, as far as I can tell, focused exclusively on the last threesome, whereas the broadcast will cover at least 8 or 10 players. So it's not something I'll be too displeased about if it doesn't work. But I would like to get recording of live streams to work just for future reference.
Again, thanks for responding.
When the stream was done after 2 hours, I did click the little square on the line item in the VDH dropdown list. I assume that's the "Stop recording" action you are referring to. But nothing got finalized. My sense is that rather early on, when I was looking at the file size in Windows Explorer to see if it was increasing, it reached 14Meg and stopped. After only a few minutes, the file size stayed at 14Meg for over 90 minutes while I was trying to get VDH to record it, and it remained 14Meg after I stopped VDH from recording it further. Now I have about 6Gig of free space on that partition so VDH couldn't have stopped because it thought it didn't have enough disk space. Besides, I should hope VDH would generate an error message about running out of space if that's what happens, which it did not. I would also hope that VDH would notify me of any other reason it would have stopped recording, also which it did not.
If it is possible for you to do, I'd like you to try to record that stream. That golf tournament is on this afternoon starting at 15:00 EDT in the US eastern time zone. They are actually playing in the San Jose, California area, so that's noon local time where they are playing. Let's see . . . I believe that is actually 19:00 GMT. I have no idea where you are, as in time zone, so I'm hoping this tells you the time of day when it's going to be available in your time zone. To the best of my knowledge, usga.org will not open the live stream until that time. On the home page of that web site, there is an abbreviated score board over to the right near the top of the page. Just below the scores there's a little icon of a TV and the word "WATCH." If you click that, it will take you to the live stream page. But the stream is there only after they open it, which won't be until 19:00 GMT. I clicked it just now and it just takes you to a page of links to various news items, photo galeries, video clips, and so on. So if you can do it, I would greatly appreciate if you would try to record their live stream today. It will also be available tomorrow at the same time. According to the TV schedule, which is lower down on that live stream page, the stream will be available for 4 hours. It will probably be longer because it's a golf tournament & they always take longer than the allotted time to play, especially since this is a major & it's a particularly difficult course. I will also attempt to get VDH to record this stream today.
I have no idea if this stream is HLS. I have no idea how to determine that. In fact, I have no idea what HLS is. I have just now noticed that my VDH dropdown still shows that VDH thinks the stream is still in progress almost 24 hours after I closed it. The message on the item in the VDH menu says. "Finalizing... HLS streaming - 2:10:47 - 910 Kbps - MP4." I guess that means the stream is indeed HLS. There is also an X there. Now, I deleted the file from my hard drive yesterday when VLC wouldn't play it. So finalizing is just never going to happen.
So I clicked the X and now it says I have an orphan. No surprise. But what is surprising is that it looks like it is trying to download something. The background shows the shading moving from left to right as if it were trying to download something. Oh wait. It's gone now. That was there for only a few seconds. OK. It was just cleaning up. Very well. So I am going to try to record that stream this afternoon as an exercise, since I'm just going to watch the broadcast on TV. The live stream online will be, as far as I can tell, focused exclusively on the last threesome, whereas the broadcast will cover at least 8 or 10 players. So it's not something I'll be too displeased about if it doesn't work. But I would like to get recording of live streams to work just for future reference.
Again, thanks for responding.
Wild Willy
Jul 9, 2016, 7:32:09 PM7/9/16
to Video DownloadHelper Q&A
I spent all afternoon trying to get VDH to capture that live stream without success. Here's what would happen. I would launch the capture in VDH and then periodically check in Windows Explorer the directory to which VDH was downloading to look at the file size of the part file. The size would tick upward by a couple hundred K perhaps every 4-5 seconds. But at some point, even though the live stream was continuing to play in the browser window, the file size would stop changing. It seemed like I saw it stop at 8.59Meg several times but it also stopped sometimes at smaller sizes and I did get it to go above 15Meg once. But every time, it would just stop at some point. The VDH dropdown menu would always say it was still downloading the file but the evidence in Windows Explorer would put the lie to that. And the live stream itself always continued to play. I would click the little box in VDH to stop the download. If I stopped a download that was still actually downloading, VDH would pop up a notification box saying my video was ready to play. And when I played those fragments in VLC, they were audio only, no video. Well that's pretty useless. They were the audio from the live stream, but where was the video? On the other hand, if I stopped a download that had stalled, then VDH would tell me it was finalizing the file, but that would never finish. I left some of them sitting for 10 minutes allegedly finalizing, but they never completed that step. I would then just delete the entry in VDH & delete the file off my hard drive, and launch my next attempt at recording the live stream.
So my original claim still stands. If VDH is meant to record live streams, I say it's not working. Most of the time, the VDH download simply stops without error message while claiming to be ongoing, despite the fact that the live stream is successfully still playing. And the few times I manually stopped a recording, the result was audio only, which to my way of thinking, is a failed recording. It's a live stream with BOTH audio AND VIDEO, so any recording that does not also include BOTH audio AND VIDEO is a failed recording.
So my original claim still stands. If VDH is meant to record live streams, I say it's not working. Most of the time, the VDH download simply stops without error message while claiming to be ongoing, despite the fact that the live stream is successfully still playing. And the few times I manually stopped a recording, the result was audio only, which to my way of thinking, is a failed recording. It's a live stream with BOTH audio AND VIDEO, so any recording that does not also include BOTH audio AND VIDEO is a failed recording.
mig
Jul 10, 2016, 4:46:59 AM7/10/16
to Video DownloadHelper Q&A
I haven't been able to watch (hence capture) a live stream from usga.org but capturing a pre-recorded video like http://www.usga.org/videos/2016/07/09/160708-unusual-seat-to-watch-golf-mp4-5029027904001.html gives a MP4 file where the video rate is 90000 frames per second, which is obviously a bug and likely to be the reason why the video stream does not display.
This has been added to the list of things to fix but for now, we must say that VDH does not work on usga.org.
This has been added to the list of things to fix but for now, we must say that VDH does not work on usga.org.
Wild Willy
Jul 10, 2016, 6:44:45 AM7/10/16
to Video DownloadHelper Q&A
I greatly appreciate your diligence in looking into this. I looked at that golf video myself just now, and it plays fine in my browser. It also downloads fairly quickly to my hard drive using VDH. But it's audio, no video. Is this the bug that you are referring to with the 90,000 fps problem? The video rate is impossible, therefore VDH saves something that is audio-only, no video? Where do you see this information? I looked at several things in the VLC player but didn't find anything that mentioned the video rate. I did find a place where it said Frame rate: 23.809524. So it thinks it's got a video. Is that not a valid number? What frame rates result in visible videos? I'm playing one of the videos I downloaded today from YouTube. Looking at the same information for that video, it does not quote a frame rate, like this golf video that is audio-only. This successful YouTube video says this:
Type: Video
Codec: Google/On2's VP9 Video (VP90)
Resolution: 864x480
Display resolution: 854x480
Decoded format: Planar 4:2:0 YUV
The unsuccessful golf video tells me this:
Type: Video
Codec: H264 - MPEG-4 AVC (part 10) (avc1)
Resolution: 1280x720
Display resolution: 1280x720
Frame rate: 23.809524
It tells me "frame rate" instead of "decoded format." So if somehow the information in the YouTube download implies a frame rate, I don't know how to figure that out, being the non-expert that I am.
How did you determine the frame rate of the golf video is 90,000 fps? I'm not doubting you; I just want to learn something.
As for a live stream, if you went looking an hour ago, there wouldn't be a live stream. An hour ago, it was 2:00 AM local time at the tournament site. Nobody is on the course golfing at that hour. Maybe some of the grounds crew is out there preparing the course, but that wouldn't be on a live stream. No, the live stream will be available in about 9 hours from now, as I write these words. If you could possibly look into that then, it might be a help. I accept there's a lot of variables involved in delivering a stream to a user, like line speed, connection reliabilty, software configuration at the receiving device, user expertise (as in dumb vs smart user). So a given download of a stream in VDH could stop for any number of reasons. I hope the fact that I have successfully downloaded some videos from YouTube proves that I can use VDH successfully so if it fails to download the live stream, I like to think it's not because I've done something wrong. I suppose that is still a possible explanation for my failing to get it to record that live video but I don't think that's the case. It seems to start OK & run for a while, but it always just stalls, without generating any error messages in VDH, which appears to think it is still recording. I would try it again this afternoon but I won't be home at that time. I'll just have to watch the golf tournament off my old-fashioned DVD recording.
Type: Video
Codec: Google/On2's VP9 Video (VP90)
Resolution: 864x480
Display resolution: 854x480
Decoded format: Planar 4:2:0 YUV
The unsuccessful golf video tells me this:
Type: Video
Codec: H264 - MPEG-4 AVC (part 10) (avc1)
Resolution: 1280x720
Display resolution: 1280x720
Frame rate: 23.809524
It tells me "frame rate" instead of "decoded format." So if somehow the information in the YouTube download implies a frame rate, I don't know how to figure that out, being the non-expert that I am.
How did you determine the frame rate of the golf video is 90,000 fps? I'm not doubting you; I just want to learn something.
As for a live stream, if you went looking an hour ago, there wouldn't be a live stream. An hour ago, it was 2:00 AM local time at the tournament site. Nobody is on the course golfing at that hour. Maybe some of the grounds crew is out there preparing the course, but that wouldn't be on a live stream. No, the live stream will be available in about 9 hours from now, as I write these words. If you could possibly look into that then, it might be a help. I accept there's a lot of variables involved in delivering a stream to a user, like line speed, connection reliabilty, software configuration at the receiving device, user expertise (as in dumb vs smart user). So a given download of a stream in VDH could stop for any number of reasons. I hope the fact that I have successfully downloaded some videos from YouTube proves that I can use VDH successfully so if it fails to download the live stream, I like to think it's not because I've done something wrong. I suppose that is still a possible explanation for my failing to get it to record that live video but I don't think that's the case. It seems to start OK & run for a while, but it always just stalls, without generating any error messages in VDH, which appears to think it is still recording. I would try it again this afternoon but I won't be home at that time. I'll just have to watch the golf tournament off my old-fashioned DVD recording.
mig
Jul 10, 2016, 7:31:06 AM7/10/16
to Video DownloadHelper Q&A
Using mplayer on linux:
$ mplayer Winery_Offers_Special_Vantage_Point_for_Golf.mp4
MPlayer 1.2.1 (Debian), built with gcc-5.3.1 (C) 2000-2016 MPlayer Team
...
VIDEO: [H264] 480x272 24bpp 90000.000 fps 0.0 kbps ( 0.0 kbyte/s)
Clip info:
major_brand: isom
minor_version: 512
compatible_brands: isomiso2avc1mp41
Load subtitles in ./
==========================================================================
Opening video decoder: [ffmpeg] FFmpeg's libavcodec codec family
libavcodec version 56.60.100 (external)
Selected video codec: [ffh264] vfm: ffmpeg (FFmpeg H.264)
==========================================================================
==========================================================================
Opening audio decoder: [ffmpeg] FFmpeg/libavcodec audio decoders
AUDIO: 48000 Hz, 2 ch, floatle, 66.8 kbit/2.17% (ratio: 8348->384000)
Selected audio codec: [ffaac] afm: ffmpeg (FFmpeg AAC (MPEG-2/MPEG-4 Audio))
==========================================================================
The MP4 format contains several redundant mechanisms to determine a stream rate. We have had several issues with this in the past, so i'm not really surprised to see some remaining glitches.
Now that we have a stable test case, it will be easier to fix the problem in a future release.
$ mplayer Winery_Offers_Special_Vantage_Point_for_Golf.mp4
MPlayer 1.2.1 (Debian), built with gcc-5.3.1 (C) 2000-2016 MPlayer Team
...
VIDEO: [H264] 480x272 24bpp 90000.000 fps 0.0 kbps ( 0.0 kbyte/s)
Clip info:
major_brand: isom
minor_version: 512
compatible_brands: isomiso2avc1mp41
Load subtitles in ./
==========================================================================
Opening video decoder: [ffmpeg] FFmpeg's libavcodec codec family
libavcodec version 56.60.100 (external)
Selected video codec: [ffh264] vfm: ffmpeg (FFmpeg H.264)
==========================================================================
==========================================================================
Opening audio decoder: [ffmpeg] FFmpeg/libavcodec audio decoders
AUDIO: 48000 Hz, 2 ch, floatle, 66.8 kbit/2.17% (ratio: 8348->384000)
Selected audio codec: [ffaac] afm: ffmpeg (FFmpeg AAC (MPEG-2/MPEG-4 Audio))
==========================================================================
The MP4 format contains several redundant mechanisms to determine a stream rate. We have had several issues with this in the past, so i'm not really surprised to see some remaining glitches.
Now that we have a stable test case, it will be easier to fix the problem in a future release.
mig
Oct 25, 2016, 2:24:12 AM10/25/16
to Video DownloadHelper Q&A
Issue with video http://www.usga.org/videos/2016/07/09/160708-unusual-seat-to-watch-golf-mp4-5029027904001.html has been fixed in beta version 6.2.0a4 https://groups.google.com/forum/#!topic/video-downloadhelper-5/J76KNU0gkJ8
Reply all
Reply to author
Forward
0 new messages