Live streaming latency

[Luke]

Hi all,

I am investigating using crtmpserver as a low-latency solution for
streaming live output from an RTSP H.264 encoder (Axis M7001) to a
flash-based RTMP client (testing with Flowplayer but it'll be a custom
flash client eventually). For our system latency is absolutely
critical; the user can control the camera's position and orientation
through a separate data stream so the system must feel responsive. No
seek or pause ability is required.

We have a working system with Wowza which gives about 750ms of latency
(using rtp-live-lowlatency). This is good but still a bit laggy, and
wowza is not ideal for us for a number of reasons, so I am
investigating alternatives. Unfortunately, the crtmpserver solution
(using the flvplayback app) gives almost exactly five seconds of
latency, which is far too much. Clearly there is some significant
buffering going on in there somewhere.

I'll have a dig through the source code myself to see if I can find
out where and why this is happening, but does anyone know of any
config parameters that allow you to influence latency? Is there
another application better suited to live, low-latency streaming?
Also, any suggestions about where to start digging would be very much
appreciated.

Cheers,
Luke

[Marc Juul]

On Tue, Mar 1, 2011 at 4:00 PM, Luke <luke.g...@gmail.com> wrote:
> Hi all,
>
> I am investigating using crtmpserver as a low-latency solution for
> streaming live output from an RTSP H.264 encoder (Axis M7001) to a
> flash-based RTMP client (testing with Flowplayer but it'll be a custom
> flash client eventually).  For our system latency is absolutely
> critical; the user can control the camera's position and orientation
> through a separate data stream so the system must feel responsive.  No
> seek or pause ability is required.
>
> We have a working system with Wowza which gives about 750ms of latency
> (using rtp-live-lowlatency). This is good but still a bit laggy, and
> wowza is not ideal for us for a number of reasons, so I am
> investigating alternatives. Unfortunately, the crtmpserver solution
> (using the flvplayback app) gives almost exactly five seconds of
> latency, which is far too much.  Clearly there is some significant
> buffering going on in there somewhere.

Yeah I've experienced the exact same thing and I really don't know why
this happens. We're doing lectures with an associated chat and five
seconds is somewhat tolerable, but we are planning a feature to let
students ask questions via voice where five second delays will be
problematic.

I've tried using Adobe Flash Media Live Encoder and dvgrab+ffmpeg for
input, which didn't affect delay. I've used flvplayback and flowplayer
as clients, and it didn't affect delay. I hadn't tried a different
rtmp server though.

If you find any way of improving this, I'm very interested!

--
Marc Juul

[C++ RTMP Server]

This is happening due to packet re-ordering. I have to have a small buffer server side when transforming RTSP to RTMP.
While RTSP handles the audio and video as independent tracks (separate timelines), RTMP is quite disturbed if I do the same. So, after taking the packets from RTSP and before putting them on RTMP I have to re-order them and get the same timeline.
Example:

V - video
A - audio
This is the order in which the packets arrives from RTSP
A1 A2 A7 V5 V6 A9 V8 V12 A13

They are in sync, but out of order when considering them on the same timeline. This needs to be delivered on RTMP like this:

A1 A2 V5 V6 A7 V8 A9 V12 A13

For that to happen, I need to keep a buffer server side.

Right now the re-ordering is happening quite brutally (I keep a queue of 100 packets). I think (not sure) wowza is doing the same and they call that the "low-latency" feature.

To test this behavior, go to

sources/thelib/src/streaming/packetqueue.cpp at line 58 and lower from 100 to 10. The difference should be quite visible.

I will modify the algorithm of re-ordering to be event-driven (detecting transitions) instead of packets-count-driven (wait for 100 packets in the queue).

When that will be implemented, the queue will only contain minimum amount of packets (2 or 3).

To test what's happening to extremes, modify that number to 0 (no packet re-ordering). The flash player will start act weird

Cheers,
Andrei

> You received this message because you are subscribed to "C++ RTMP Server" mailing list.
> To post to this group, send email to c-rtmp...@googlegroups.com
> To unsubscribe from this group, send email to
> c-rtmp-serve...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/c-rtmp-server?hl=en

------
Eugen-Andrei Gavriloaie
Web: http://www.rtmpd.com

[Marc Juul]

On Wed, Mar 2, 2011 at 12:44 AM, C++ RTMP Server <crtmp...@gmail.com> wrote:
>
> To test what's happening to extremes, modify that number to 0 (no packet re-ordering). The flash player will start act weird

But the packet re-ordering is only needed for RTSP right?
Couldn't there be an option to disable packet re-ordering, which will
then disable RTSP but otherwise function normally?

--
Marc Juul

[Luke]

Andrei,

Thanks very much! I was excited to see your reply as we have no
interest in the audio stream. I hacked on PacketQueue::PushPacket to
make it drop audio packets and immediately send video packets, as
follows:

if(isAudio) return vector<Packet *>();
if(_startVideo==-1)
_startVideo = absoluteTimestamp;
absoluteTimestamp-=_startVideo;
Packet *pPacket = GetPacket(pData,dataLength,absoluteTimestamp,false);
vector<Packet *> result;
ADD_VECTOR_END(result,pPacket);
ADD_VECTOR_END(_free,pPacket);
return result;

As a result, round-trip latency dropped massively. I validated the
result by taking a number of measurements. I pointed the camera at an
onscreen timer and took screenshots, averaging about ten measurements
to get these results:

Wowza: 394ms (much faster than I'd thought, hmm)
CRTMP: 317ms
MJPG: 124ms

That last is the Motion-JPEG stream direct from the encoder's internal
live stream page. I'd have tested using VLC with the RTSP feed but it
requires a buffer of at least 100ms or the feed jags out. So it's
probably not faster than MJPG. I have no idea why Wowza was faster
today, although I suspect my previous hand-waving latency measuring to
be... less than accurate :)

As the latency figures include encoding (~70ms according to the
manufacturer) and decoding (who knows), I suspect there isn't much
more to trim from CRTMP. Having said that, it's now sufficiently
faster than Wowza for us to switch to it.

Andrei, thank you very much for your very fast and extremely helpful
reply! I will let you know if and when the system goes live to the
public. I'm very impressed with c-rtmp-server and will definitely
bear it in mind for future projects.

Thanks again,
Luke

>  smime.p7s
> 5KViewDownload

[Luke Gumbley]

Hi guys,

I was inspired enough to take this a step further and rewrite the PacketQueue for low latency performance. Revised files attached, hopefully they are useful.  I don't know what the exact latency figure is (too lazy to set up the test rig again) but it is comparable to the results from the previous dirty hack only audio is now supported as well.

Marc, you should be able to drop these in and recompile, let us know if it solves your problem!

Andrei, I tried to follow your coding style but the algorithm is possibly a bit clumsy. I won't be offended in the slightest if you do something different... I hereby relinquish all rights to the attached code etc. etc. if you need something more formal let me know.

Cheers,

Luke

2011/3/3 Luke <luke.g...@gmail.com>