example of unicast of a live stream using rtfmp

[bgpl]

hi,
I have gone over most of the posts, I think, and couldnt yet get an answer / recipe.

Could someone point me to some information / links etc., on how to stream av streams from a publisher (say from ffmpeg or gstreamer on the server) perhaps in flv format, to cumulus server and the flash client can view it ? In other words, I am more interested in the rtmfp unicast features (publisher using ffpmeg containing av streams using h264 + aac on flv) to a client, rather than p2p.

Best regards,
bgpl

[Thomas Jammet]

Like it is said in the post bellow VOD is not supported yet : 

https://groups.google.com/d/msg/openrtmfp-cumulus/6HkJhlzPN3s/Hb0kri_eTOsJ 

[bgpl]

And I am not looking at VOD, I am looking at live streaming as I mentioned.

I did see posts in this forum including one from cumulus-dev which mentioned that live streaming is supported:
https://groups.google.com/forum/#!searchin/openrtmfp-cumulus/live$20streaming/openrtmfp-cumulus/pZYUKMgtTaI/gtfuaLXe7qMJ

By VOD here, I assume you are referring to streaming a video file in some format.
In my case I generate streams from gstreamer on the fly.




 

[bgpl]

 Ah, apologies, I see that in my first email, I did not make it clear that I am looking at live streaming, not VOD.

[Mathieu Poux]

If you generate video on the fly from gstreamer, you have to find a way to "share" this video packets with Cumulus, and give it to a "virtual publication" in Cumulus.

You can create one "virtual" publication from server application with "cumulus:publish" method (see cumulus object description), it will return you a "publication" object, on which you could push your video/audio packets (see publication object description). But how send your packet FROM gstreamer TO this server application?

Many options are possible for that:

- Use sockets (you create a TCP or UDP server which receives the gstreamer video/audio packet to distribute it to your "publication" object).

- Create a LUA extension for gstreamer and encapsulates its API to get it available from a server application.

- Search a LUA extension already existing (I have just seen "https://github.com/katcipis/luagstreamer" but I have not tester ;-)).

Hope to have answering you :-)

Le 11 novembre 2013 18:05, bgpl <badri....@gmail.com> a écrit :

 Ah, apologies, I see that in my first email, I did not make it clear that I am looking at live streaming, not VOD.

--
Vous recevez ce message, car vous êtes abonné au groupe Google Groupes OpenRTMFP Cumulus.
Pour vous désabonner de ce groupe et ne plus recevoir d'e-mails le concernant, envoyez un e-mail à l'adresse openrtmfp-cumu...@googlegroups.com.
Pour plus d'options, visitez le site https://groups.google.com/groups/opt_out .

[bgpl]

Thank you Mathieu. I will read your post in greater detail.

Perhaps I should update on what I have done so far, where I am stuck, and ask for help... Let me attempt to do that here.
Please forgive any stupidity as I am pretty new to rtmfp / cumulus. It looks like a great piece of software.

Background:
-------------------
My use case is: I create live streams on server side containing video in h.264 and audio (aac), wrapped in an flv container. I am capable and flexible in streaming either the flv container, or streaming the independent video / audio streams to cumulus on server side (anything possible, to get cumulus working). Client is a regular flash player connecting to the rtmfp stream.

I already have it working with an rtmp mediaserver (crtmpserver) and it works well. My requirement is to lower the (network congestion related) latency even further by using udp, so I am really using it as a live unicast from server to client, no p2p requirements. My goal therefore is lowest possible latency.

What I have tried so far:
-----------------------------------
(1) I started with just video sterams just to make sure. I wrote a main.lua (attached to this message) to receive video data on port 6666 (TCP).
In that I tried to create a publication on client connection.

testpublication = nil
function onConnection(client,...)
    -- if first time, start off publication
    if not testpublication then
        NOTE("Created a new publication")
        testpublication = cumulus:publish("testpublication")
    end
end


(2) When the client connects, I am trying to add it to the publication. I think I am doing it wrong, and dont think I can set the publisher of the client this way (is llisterner.publication.publisher "read only" ?)

function onSubscribe(client, listener)
    NOTE("Added listener to publication")
    listener.publication.publisher = testpublication
end

(3) I started a simple gstreamer test stream sending video h264 data to port 6666 (TCP).

For completeness, the stream is as follows. Removing the flvmux below will send the raw stream, without flvmux wrapper.
 % gst-launch  videotestsrc is-live=true do-timestamp=true ! queue ! video/x-raw-yuv,framerate=30/1,width=640,height=480 ! timeoverlay font-desc="Courier 40px" color=0x00ff00 shadow=false halignment=right valignment=bottom ! tee name=t1 ! queue ! xvimagesink sync=false t1. ! queue ! videoscale ! video/x-raw-yuv,framerate=30/1,width=640,height=480,format="(fourcc)I420" ! videoflip name=vflip ! x264enc tune=zerolatency speed-preset=ultrafast profile=baseline key-int-max=240 bitrate=1000 ! flvmux !  tcpclientsink port=6666

(4)
I push the video data received by the TCP server to the publication. Maybe I am doing this wrong.

function server:clientHandler(client)
    function client:onReception(data)
        if (testpublication) then
            NOTE("Pushing video packet..")
            testpublication:pushVideoPacket(cumulus.epochTime, data)
            testpublication:flush()
        end
        return 0 -- return rest (all has been consumed here)
    end
end

(5)
I see client from flashplayer connecting, and tcp server receiving streams from gstreamer, and pushing video packets to flash client. But nothing on flashplayer video.

Where I am stuck:
---------------------------
- Video packets forwarding but nothing seen on flashplayer. I know flashplayer connects since to cumulus since i get the messages printed on connection.
- SHould I be sending flv or h264. I saw pushVideoPacket and pushAudioPacket, which seems to indicate that it needs raw H264 elementary streams. If it is raw H264, then in the case of audio + video will the stream not get out of avsync due to network latencies on the way to the client ?

Note that this is only a prototype attempt and I know it is inefficient, but i just want to get a stream playing first and then clean it up. I am completely ok to dig into the C++ code of cumulus, but thought someone will know better than me since that is a very slow path.

Sorry for the long mail, and appreciate your help.

regards
Badri

[bgpl]

 hi Mathieu,
 after reading your post in more detail, I have concluded that I am doing the same as you said, using the sockets option.
 My problem is that I am probably doing something wrong (maybe not correctly connecting the listener from the flashplayer client to the "virtual" publication in the cumulus server main.lua), because I see no video coming on flashplayer, though i am pushing video packets to it. I have posted what I have done, so knowledgeable people on this forum may comment / correct me.

 thanks,
 bgpl

[Mathieu Poux]

Hi,

Yes, you are on the good way :-)

Excepts it:

function onSubscribe(client, listener)
    NOTE("Added listener to publication")
    listener.publication.publisher = testpublication
end

This piece of code is useless (onSubscribe event is here just to say you that there is a new subscriber for the publication, it's already subscribed).

Otherwise, yes, pushVideoPacket takes a raw H264 packet. And about audio/video synchronisation, you have to get the time value of gstreamer (certainly sending in its packets), and your client will synchronise it in using this time value. In other terms, I mean that you should not use the "cumulus.epochTime" thing, but really use the time value sent by gstreamer.

I have never used gstreamer, but it seems here that it sends a FLV file... Maybe you can change configurations of gstreamer to send raw packet (and read the time value on first frame and use it), or you have to manipulate the gstreamer packets to unpack raw content and give it to Cumulus.

Certainly it will require you a little effort, but you are on the good way ;-)

good luck!

Mathieu

2013/11/11 bgpl <badri....@gmail.com>

--

[bgpl]

 hi Mathieu,

 I have tried with Raw h264 packets as well, but it had the same result.  I have also changed new lua file passing a correct time, not epochTime, but that didnt help this either. (new file attached, this time using udp instead of tcpserver).

 My question:
 How do I subscribe the flash client as a listener to the "virtual" publication ? You said that the below code:

function onSubscribe(client, listener)
    NOTE("Added listener to publication")
    listener.publication.publisher = testpublication
end

was useless, but I am using it to make the flash client subscribe to the virtual publication. Am I doing this wrong ?

PS: the new gstreamer pipeline is without the flv, and send raw x264 streams..

gst-launch  videotestsrc is-live=true do-timestamp=true ! queue ! video/x-raw-yuv,framerate=30/1,width=640,height=480 ! timeoverlay font-desc="Courier 40px" color=0x00ff00 shadow=false halignment=right valignment=bottom ! tee name=t1 ! queue ! xvimagesink sync=false t1. ! queue ! videoscale ! video/x-raw-yuv,framerate=30/1,width=640,height=480,format="(fourcc)I420" ! videoflip name=vflip ! x264enc tune=zerolatency speed-preset=ultrafast profile=baseline bitrate=1000 !  udpsink port=6666

[Mathieu Poux]

Your subscriber client subscribes for the publication from flash side, with a simple _netStream.play("testpublication"), really your onSubscribe code here is completly useless (onSubscribe allows to reject the subscriber if need... but when the event callback the subscriber is already playing the publication).

You are certainly very close of your goal, maybe few bytes are not correctly set on the header of every packets, I don't know... normally video packet starts with a 0x09 byte following by 4 timestamp bytes. For audio, 0x08 following by timestamp.

2013/11/11 bgpl <badri....@gmail.com>

[bgpl]

 Ahh.. I was not writing a subscriber client to _netstream.play("testpublication").
let me try that.

[Mathieu Poux]

Detailed solution here:

https://groups.google.com/d/msg/openrtmfp-cumulus/DWc8YZ4xw90/ZwXsRchWbbgJ

[Ram Natarajan]

Mathieu,
How can we monitor the qualityOfService from the testpublication.listener.videoQOS object?
-Ram