Using WebM/VP8 for low latency streaming

[rohan]

Hi guys,

I'm trying to switch over from H.264 to VP8 in an application that
requires low latency (real time) streaming. I'm using FFMPEG on the
server to encode the video from a live source, and ffplay to play it
back on the client. However, I can't seem to get my latency down to
less than about 1 second.

I've tried a bunch of options from the video conferencing example on
the website and the "Real-time CBR Encoding and Streaming" example
parameters.

Does somebody have an idea of the best parameters to use for low
latency encoding?

Here are the ffmpeg encoding parameters I'm using right now (input
options cut to reduce size):

ffmpeg <input options> -loglevel debug -f webm -vcodec libvpx -r 30 -
skip_threshold 25 -speed 1 -quality realtime -b 900000 -maxrate 900000
-minrate 900000 -bufsize 900000 -rc_init_occupancy 900000 -
rc_lookahead 0 -qmin 20 -qmax 50 -an -

The mapping to VP8 encoder options (figured out via ffmpeg and vpxenc
source):
ffmpeg = vp8
skip_threshold = drop_frame
speed = cpu_used
quality = --rt
-b -maxrate -minrate = --end_usage=cbr --target-bitrate=
-bufsize -rc_init_occupancy = buf-sz, buf-initial, buf-optimal (with
current values, these will be set to 1000)
-qmin, -qmax = --min-q, --max-q

Would appreciate any help/ideas for other params to try.

Thanks!
Rohan

[Ralph Giles]

On 16 July 2011 19:29, rohan <hanr...@gmail.com> wrote:

> However, I can't seem to get my latency down to
> less than about 1 second.
>

> [...]

>
> ffmpeg <input options> -loglevel debug -f webm -vcodec libvpx

Off the top of my head, -f webm could well be the problem. You don't
say how you're streaming the format, but it's hard to get less than a
second of latency if you mux into space-oriented format like webm and
stream over (for example) http.

For realtime streaming one generally wants to use an RTP stream instead.

HTH,
-r

[James Zern]

On Sat, Jul 16, 2011 at 19:29, rohan <hanr...@gmail.com> wrote:
> [...]

> The mapping to VP8 encoder options (figured out via ffmpeg and vpxenc
> source):
>

This mapping is also available here [1].

> ffmpeg = vp8
> skip_threshold = drop_frame
> speed = cpu_used
> quality = --rt
> -b -maxrate -minrate = --end_usage=cbr --target-bitrate=
> -bufsize -rc_init_occupancy = buf-sz, buf-initial, buf-optimal (with
> current values, these will be set to 1000)
> -qmin, -qmax = --min-q, --max-q
>

[1]: http://ffmpeg.org/ffmpeg.html#SEC39

[rohan]

On Jul 17, 1:09 am, Ralph Giles <gi...@thaumas.net> wrote:

Cool, thanks. I was using a straight TCP socket for the transport, but
I'll try switching the format to see if that reduces the latency.

Something that confuses me though is why, in the low latency
"udpsample" application, the minimum, target and optimal buffer size
is set to 1000. Doesn't this cause the encoder to target a 1-second
latency?

[John Koleszar]

The rc buffer settings do have some impact on latency when you're
bandwidth constrained. I just merged a revised algorithm that makes
this case perform better and should be included in our next release.
Can you provide any details on how you're measuring latency? If you're
seeing 1 second on localhost, there's something else going on. There
may be buffering in the webm muxer, as mentioned by Raplh Giles.

[rohan]

On Jul 20, 10:51 am, John Koleszar <jkoles...@google.com> wrote:

> The rc buffer settings do have some impact on latency when you're
> bandwidth constrained. I just merged a revised algorithm that makes
> this case perform better and should be included in our next release.
> Can you provide any details on how you're measuring latency? If you're
> seeing 1 second on localhost, there's something else going on. There
> may be buffering in the webm muxer, as mentioned by Raplh Giles.

I was measuring latency just visually, comparing to the same
implementation using x264. Also, it wasn't over localhost, but over
the Internet with about 100ms of latency in the network itself. I'm
going to set up some more quantitative tests later, so I'll get back
to you with those numbers when I have them.
Switching from webm to ivf also seems to have helped.

[Vincent Autieri II]

One addition is that I'm also sending audio, and the audio lags as well.  So maybe its in the RTP/networking.

On Thursday, November 15, 2012 12:07:05 AM UTC-5, Vincent Autieri II wrote:

 I'm doing almost the same thing, trying VP8, coming from H264.  I'm seeing this issue using the webrtc project, working off their trunk.  I was hoping that group could help me figure out how to track down where the latency is coming from but it must be a question nobody their really knows where to start looking..  My link to the webrtc post is below.   Eventually I'll need to figure out why the video quality is not the greatest when using webrtc for faster moving thing as well, but right now I cannot move forward until I can get rid of the latency.  Can someone assist me in trying to narrow down where to start looking?  I have eliminated directshow and have my thread and process set to a high priority.  It just acts like a buffer n levels deep exists...piling up video frames.  

https://groups.google.com/forum/?fromgroups#!topic/discuss-webrtc/J9aas8iql3o

On Friday, October 26, 2012 4:27:09 PM UTC-4, hca...@gmail.com wrote:

Hi, Rohan! What was your conclusion, here? I'm tantalized by vp8 / webm , but also can't get less than 1.5 approx latency

[John Koleszar]

I don't have any specific advice, other than to instrument your
applications so that you can see the time a frame is generate to when
it is sent, and when it is received to when it's displayed. These
timers may be available in webrtc somewhere. Doesn't sound like
anything specific to VP8, given that you see the same behavior with
the audio.

Good luck,

John

> --
> You received this message because you are subscribed to the Google Groups
> "WebM Discussion" group.
> To view this discussion on the web visit
> https://groups.google.com/a/webmproject.org/d/msg/webm-discuss/-/mbFknTm4LBQJ.
>
> To post to this group, send email to webm-d...@webmproject.org.
> To unsubscribe from this group, send email to
> webm-discuss...@webmproject.org.
> For more options, visit this group at
> http://groups.google.com/a/webmproject.org/group/webm-discuss/?hl=en.

[Vincent Autieri II]

Thanks John,

 

I agree and thank you for responding.  I did disable the audio stream to make sure it doesn't enter the equation of trying to sync audio with video.  I also see they are compiling VP8 with the realtime flag so its buffer is only 1 level deep. Tweaking a few things it looks like I'm down to just under a second, but that means I still have around 750-900ms to go!  I'll start poking around at some of these statistics later today.  I do know I'm pushing in an RGB image instead of vp8's native (a todo for me), as well as using a negative height to flip the image.   Eventually I'll have those tasks pushed onto the GPU before passing into the webrtc framework.