Playback starts in the middle of video

[Aleksey Ivanov]

Hello,

I would like to generate DASH/HLS for RTP input.

As we have already discussed currently direct RTP input is not supported, so I decided to use FFmpeg piping.

I decided to start from plain static mp4 file and not RTP.

So, the whole thing looks like this:
  - stream static mp4 with FFmpeg to UDP ports

  - use those UDP ports as input for Shaka Packager

Unfortunately, something goes wrong since playback always started from the middle of video.
I have noticed in browser console that segments start loading from #10 (or from #12) and not from #1.

If I omit FFmpeg and I directly use the file as input everything is fine: playback starts from the beginning and segments are loaded from #1.

Here are commands I use:

STREAM_IP=$(docker-machine ip default) # I run Shaka in Docker container
ffmpeg \
  -re -i samples/SampleVideo_1280x720_keyframe_24.mp4 \
  -c:v libx264 -an -f mpegts udp://$STREAM_IP:5004 \
  -c:a aac -vn -f mpegts udp://$STREAM_IP:5002

packager \
  'input=udp://0.0.0.0:5002,stream=audio,init_segment=dash-socket/audio_init.mp4,segment_template=dash-socket/audio_$Number$.m4s' \       'input=udp://0.0.0.0:5004,stream=video,init_segment=dash-socket/video_init.mp4,segment_template=dash-socket/video_$Number$.m4s' \
  --mpd_output dash-socket/manifest.mpd

For plain mp4 as input:

packager \             'input=samples/SampleVideo_1280x720_keyframe_24.mp4,stream=audio,init_segment=dash/audio_init.mp4,segment_template=dash/audio_$Number$.m4s' \
'input=samples/SampleVideo_1280x720_keyframe_24.mp4,stream=video,init_segment=dash/video_init.mp4,segment_template=dash/video_$Number$.m4s' \
--generate_static_mpd --mpd_output dash/manifest.mpd

One thing I have noticed is that MPD files differ.

Here is video section from dash-socket/manifest.mp4 (streamed with FFmpeg)

<AdaptationSet id="0" contentType="video" width="1280" height="720" frameRate="90000/3600" segmentAlignment="true" par="16:9">

<Representation id="0" bandwidth="995470" codecs="avc1.4d401f" mimeType="video/mp4" sar="1:1">

  <SegmentTemplate timescale="90000" initialization="video_init.mp4" media="video_$Number$.m4s" startNumber="1">

    <SegmentTimeline>

      <S t="126000" d="1166400"/>

      <S t="1292400" d="788400"/>

      <S t="2080800" d="946800"/>

      <S t="3027600" d="882000"/>

      <S t="3909600" d="900000"/>

      <S t="4809600" d="1162800"/>

      <S t="5972400" d="475200"/>

      <S t="6447600" d="802800"/>

      <S t="7250400" d="900000"/>

      <S t="8150400" d="856800"/>

      <S t="9007200" d="1299600"/>

      <S t="10306800" d="648000"/>

      <S t="10954800" d="874800"/>

      <S t="11829600" d="1044000"/>

      <S t="12873600" d="680400"/>

      <S t="13554000" d="950400"/>

    </SegmentTimeline>

  </SegmentTemplate>

</Representation>

</AdaptationSet>

and here is video section from dash/manifest.mpd (plain file as input)

<AdaptationSet id="0" contentType="video" width="1280" height="720" frameRate="12288/512" segmentAlignment="true" par="16:9">

<Representation id="0" bandwidth="1277207" codecs="avc1.64001f" mimeType="video/mp4" sar="1:1">

  <SegmentTemplate timescale="12288" initialization="video_init.mp4" media="video_$Number$.m4s" startNumber="1">

    <SegmentTimeline>

      <S t="1024" d="122880"/>

      <S t="123904" d="124928"/>

      <S t="248832" d="141312"/>

      <S t="390144" d="122880" r="2"/>

      <S t="758784" d="105984"/>

      <S t="864768" d="134144"/>

      <S t="998912" d="122880"/>

      <S t="1121792" d="117248"/>

      <S t="1239040" d="118272"/>

      <S t="1357312" d="138752"/>

      <S t="1496064" d="122880"/>

      <S t="1618944" d="104960"/>

      <S t="1723904" d="135680"/>

      <S t="1859584" d="122368"/>

      <S t="1981952" d="110080"/>

      <S t="2092032" d="9216"/>

    </SegmentTimeline>

  </SegmentTemplate>

</Representation>

</AdaptationSet>

As you can see they have different timescale. Maybe it causes the problem?

I have checked original file with ffprobe:

ffprobe -i samples/SampleVideo_1280x720_keyframe_24.mp4

ffprobe version 3.4 Copyright (c) 2007-2017 the FFmpeg developers

  built with Apple LLVM version 9.0.0 (clang-900.0.38)

  configuration: --prefix=/usr/local/Cellar/ffmpeg/3.4 --enable-shared --enable-pthreads --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-gpl --enable-ffplay --enable-libmp3lame --enable-libx264 --enable-libxvid --enable-opencl --enable-videotoolbox --disable-lzma

  libavutil      55. 78.100 / 55. 78.100

  libavcodec     57.107.100 / 57.107.100

  libavformat    57. 83.100 / 57. 83.100

  libavdevice    57. 10.100 / 57. 10.100

  libavfilter     6.107.100 /  6.107.100

  libavresample   3.  7.  0 /  3.  7.  0

  libswscale      4.  8.100 /  4.  8.100

  libswresample   2.  9.100 /  2.  9.100

  libpostproc    54.  7.100 / 54.  7.100

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'samples/SampleVideo_1280x720_keyframe_24.mp4':

  Metadata:

    major_brand     : isom

    minor_version   : 512

    compatible_brands: isomiso2avc1mp41

    encoder         : Lavf57.83.100

  Duration: 00:02:50.92, start: 0.000000, bitrate: 1693 kb/s

    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], 1304 kb/s, 24 fps, 24 tbr, 12288 tbn, 48 tbc (default)

    Metadata:

      handler_name    : VideoHandler

    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, 5.1, fltp, 383 kb/s (default)

    Metadata:

      handler_name    : SoundHandler

The timescale is 12288.

But FFmpeg streams it as 90000:

Output #0, mpegts, to 'udp://192.168.99.100:5004':

  Metadata:

    major_brand     : isom

    minor_version   : 512

    compatible_brands: isomiso2avc1mp41

    encoder         : Lavf57.83.100

    Stream #0:0(und): Video: h264 (libx264), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], q=-1--1, 24 fps, 90k tbn, 24 tbc (default)

    Metadata:

      handler_name    : VideoHandler

      encoder         : Lavc57.107.100 libx264

    Side data:

      cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1

So, maybe it is totally FFmpeg related issue. But after searching and "googling" I can't find how to set those tbn and tbc.

Could please help me to figure it out?

Thank you.

[KongQun Yang]

Hi Aleksey,

I don't think timescale matters.

It seems like a few frames were skipped. Do you start ffmpeg first or packager first? The behavior seen below would be expected if you started ffmpeg first since in that case shaka-packager missed the first key-frame, so it had to wait until next key-frame. If that is the case, can you start packager first?

Also, it won't be a problem for live RTP input since it only affects the first few frames when packaging starts.

-- KongQun Yang (KQ)

--
You received this message because you are subscribed to the Google Groups "Shaka Packager Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to shaka-packager-users+unsub...@googlegroups.com.
To post to this group, send email to shaka-packager-users@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/shaka-packager-users/adb7e230-72f2-42c4-b093-008b4abffa4a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Aleksey Ivanov]

Thank you for the answer.

Sorry, I forgot to mention that I definitely start packager first, it prints

[1208/082509:INFO:demuxer.cc(88)] Demuxer::Run() on file 'udp://0.0.0.0:5002'.

[1208/082509:INFO:demuxer.cc(160)] Initialize Demuxer for file 'udp://0.0.0.0:5002'.

[1208/082509:INFO:demuxer.cc(88)] Demuxer::Run() on file 'udp://0.0.0.0:5004'.

[1208/082509:INFO:demuxer.cc(160)] Initialize Demuxer for file 'udp://0.0.0.0:5004'.

and only then I run FFmpeg.

Yes, maybe for really live stream it wouldn't be so much problem, but with plain file it worries me a lot.

The file duration is 02:51. But playback starts only at ~01:50.
So, it is not about few seconds delay but even more that a half.

I would like to figure it out to better understand a process.
Maybe I am missing something important.

Is there any additional information I could check to investigate what is going on?

[KongQun Yang]

-- KongQun Yang (KQ)

On Fri, Dec 8, 2017 at 12:33 AM, Aleksey Ivanov <iale...@gmail.com> wrote:

Thank you for the answer.

Sorry, I forgot to mention that I definitely start packager first, it prints

[1208/082509:INFO:demuxer.cc(88)] Demuxer::Run() on file 'udp://0.0.0.0:5002'.

[1208/082509:INFO:demuxer.cc(160)] Initialize Demuxer for file 'udp://0.0.0.0:5002'.

[1208/082509:INFO:demuxer.cc(88)] Demuxer::Run() on file 'udp://0.0.0.0:5004'.

[1208/082509:INFO:demuxer.cc(160)] Initialize Demuxer for file 'udp://0.0.0.0:5004'.

and only then I run FFmpeg.

Yes, maybe for really live stream it wouldn't be so much problem, but with plain file it worries me a lot.

The file duration is 02:51. But playback starts only at ~01:50. 

So, it is not about few seconds delay but even more that a half.

Emm, that needs investigation. There may be multiple problems:

1.  Some frames are lost. If you started packager first, it shouldn't happen. We'll need to find out where the frames go - FFmpeg, network or packager's fault?

2. From the manifest, it appears only a few seconds were skipped. This does not match with what you are seeing "playback starts only at ~01:50. " Which player are you using? Could it be possible that the player just seeks to the live edge and there is extra some more contents available before the live edge?

 

Feel free to open an issue at https://github.com/google/shaka-packager/issues with detail re-pro instructions and we'll be happy to investigate what is going on.

I would like to figure it out to better understand a process.
Maybe I am missing something important.

Is there any additional information I could check to investigate what is going on?

--
You received this message because you are subscribed to the Google Groups "Shaka Packager Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to shaka-packager-users+unsub...@googlegroups.com.
To post to this group, send email to shaka-packager-users@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/shaka-packager-users/d8bbb141-b26a-437f-bdc4-e51923ba4594%40googlegroups.com.

[Aleksey Ivanov]

Thank you KQ.

Honestly, I am not familiar with client-side of adaptive streaming and at that time I couldn't spend much time on it.

So, I picked libraries that I first found and built quick & dirty solution like that:

<html lang="en">

<head>

  <link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/video.js/6.5.0/video-js.min.css">

</head>

<body>

  <video id=example-video width=600 height=300 class="video-js vjs-default-skin" controls> </video>

  <script src="https://cdnjs.cloudflare.com/ajax/libs/video.js/6.5.0/video.min.js"></script>

  <script src="https://cdnjs.cloudflare.com/ajax/libs/dashjs/2.6.3/dash.all.min.js"></script>

  <script src="https://cdnjs.cloudflare.com/ajax/libs/videojs-contrib-dash/2.9.2/videojs-dash.min.js"></script>

  <script>

    var player = videojs('example-video');

    player.src({ src: 'http://127.0.0.1:8887/dash-socket/manifest.mpd', type: 'application/dash+xml'});

         player.play();

  </script>

</body>

</html>

It definitely could be something wrong on client side.

What do you recommend?

Try to use Shaka Player and to look what result would be?
Then if Shaka Player would start playback in the middle too I would open issue on GItHub? 

[KongQun Yang]

Yes, please try with Shaka Player. You can use Shaka player demo: https://shaka-player-demo.appspot.com/demo/ then select "custom asset" and then put your manifest url in "Custom manifest" field. 

-- KongQun Yang (KQ)

--

You received this message because you are subscribed to the Google Groups "Shaka Packager Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to shaka-packager-users+unsub...@googlegroups.com.
To post to this group, send email to shaka-packager-users@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/shaka-packager-users/075072d1-42a5-4b08-a9bd-f6197849e344%40googlegroups.com.

[Aleksey Ivanov]

KQ,

I have uploaded DASH files to my server: https://194.67.209.117:89.

I use this link: https://shaka-player-demo.appspot.com/demo/#asset=https://194.67.209.117:89/manifest.mpd;lang=en-US

Browser downloads MPD, but it can't download any of segments. There are a lot of warnings in console and no playback starts at all.

However, I am able to download those file via curl.

Could you please take a look?

I use self-signed certificate, so you should to open https://194.67.209.117:89/manifest.mpd directly in browser first to add certificate exception.

Is it anything wrong with my configuration?

[Aleksey Ivanov]

With DASH generated directly from mp4 file it works pretty well:

https://194.67.209.117:89/dash-file/manifest.mpd

[KongQun Yang]

Hi,

See my replies below.

-- KongQun Yang (KQ)

On Fri, Dec 8, 2017 at 1:22 PM, Aleksey Ivanov <iale...@gmail.com> wrote:

KQ,

I have uploaded DASH files to my server: https://194.67.209.117:89.

I use this link: https://shaka-player-demo.appspot.com/demo/#asset=https://194.67.209.117:89/manifest.mpd;lang=en-US

Browser downloads MPD, but it can't download any of segments. There are a lot of warnings in console and no playback starts at all.

However, I am able to download those file via curl.

Are you uploading the contents while it is packaged? If you upload the contents after packaging completes - it is expected that the player is not able to download any of segments as you don't actually have data at the live edge - there are only a few minutes of data in the past.

Try uploading the contents immediately when packaging starts. You can then try seek back a little bit to see the beginning of the stream.

  

Could you please take a look?

I use self-signed certificate, so you should to open https://194.67.209.117:89/manifest.mpd directly in browser first to add certificate exception.

Is it anything wrong with my configuration?

--

You received this message because you are subscribed to the Google Groups "Shaka Packager Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to shaka-packager-users+unsub...@googlegroups.com.
To post to this group, send email to shaka-packager-users@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/shaka-packager-users/fe7fbf77-8f7f-4b00-99bc-67aff991ddb7%40googlegroups.com.

[Aleksey Ivanov]

Hi KQ,

For some season, I started to watch video only after it was completely packaged.
I forgot about it must be "live-like" stream. It is totally my bad, sorry.
Now I try to watch the video while it is being packaged and I am able to see it at ~00:08-00:09. 

But the playback is available only about 01:10-01:15 from packaging start. Is it the way it should be?

You have mentioned few times concept of "live edge". Could you please tell me more about that?
What is it? How is it calculated? Can one detect a position of that "live edge" in time?

Could you please also tell me why exactly playback starts at the position of ~00:08-00:09?
What does it depend on?
I wonder can I choose any video file, do some sort of calculations and tell exact time position that video would be started from?

[KongQun Yang]

Hi Aleksey,

See my replies inline.

-- KongQun Yang (KQ)

On Mon, Dec 11, 2017 at 6:26 AM, Aleksey Ivanov <iale...@gmail.com> wrote:

Hi KQ,

For some season, I started to watch video only after it was completely packaged.
I forgot about it must be "live-like" stream. It is totally my bad, sorry.
Now I try to watch the video while it is being packaged and I am able to see it at ~00:08-00:09. 

But the playback is available only about 01:10-01:15 from packaging start. Is it the way it should be?

There are some latency, but it shouldn't be that long. It is controlled by suggestedPresentationDelay, which is default to 10seconds in Shaka Player, I think. It is also configurable in Shaka Packager using --suggested_presentation_delay flag. There is another implicit one segment delay in packager and there is latency in FFmpeg as well. So the overall latency should be one_segment + suggested_presentation_delay + ffmpeg_latency. I am not sure what is the latency of ffmpeg for your stream. You may want to check if there is any drift in your clock as well.

 

You have mentioned few times concept of "live edge". Could you please tell me more about that?
What is it? How is it calculated? Can one detect a position of that "live edge" in time?

Here is a diagram from Shaka Player showing how live edge is defined: https://shaka-player-demo.appspot.com/docs/api/tutorial-architecture.html. (scroll to the bottom)

So basically, the live edge is calculated from the current timestamp and availabilityStartTime. availabilityStartTime is determined during packaging. If there is any drift in packaging server's clock and player's clock, you may additional latency or playback problems.

Could you please also tell me why exactly playback starts at the position of ~00:08-00:09?

That matches what we saw in the manifest. Looks like one key frame was missed, result in loss of frames. We'll need to figure out why that happens, which should not if you start packager before starting FFmpeg. Not sure if it has anything to do with docker. Can you open an issue out Github page with detail re-pro instructions and we will be happy to take a look.

What does it depend on?
I wonder can I choose any video file, do some sort of calculations and tell exact time position that video would be started from?

--
You received this message because you are subscribed to the Google Groups "Shaka Packager Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to shaka-packager-users+unsub...@googlegroups.com.
To post to this group, send email to shaka-packager-users@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/shaka-packager-users/634d65c4-d494-46d7-bc40-0ee2a1845465%40googlegroups.com.