VideoRoom replay.

[Horsetopus]

Hi,

So, I know the subject has been talked a lot about, but I just want to clarify and summarize a few things.

So, if you have a videoroom, by using the configure request, you can save the mjr video and audio files in the folder you want with the name you want.

It can even happen while publishing, resending the configure request to the publishing handle with only the recording options can start and stop the recordings, each start generating new files.

But then all you have are some files. You can't just replay them, you have to do some modifications either for recordplay or for streaming

For recordplay:

  - modifications:

       You need to keep track of the mjr files that should have been created, and generate the nfo files.

  - replay:

       You cannot seek or pause, you can just play the video. It's basically like a virtual webcam.

For streaming:

  - modifications:

       You need to re-encode the files to another video format with an external tool.

  - replay:

       You can play the files as on demand streaming for one client, or have a stream synchronously sent to multiple clients.

       

Questions:
  - Do I have this right?

  - With recordplay, can I set a different folder for a plugin instance or do I need to move the files to the one set in the configuration file?

  - With streaming, I can pause, but can I seek?

  - I'm guessing already that streaming is the only valid solution for my needs, is there a good, complete example somewhere?

Thanks.

[Lorenzo Miniero]

These assumptions are right if you want to replay via WebRTC. Anyway, none of our plugins does seeking. That needs to be implemented at a plugin level, and is quite problematic when you have multiple publishers/sessions being played together at the same time (you want them to advance together, and what if you go beyond the end of a recording, or before one started and another ended?). That said, WebRTC is not mandatory for what is an on-demand kind of application, and there are easier alternatives.

You can still post-process each contribution to a web-playable format and reproduce the resulting files together in a synchronous way (assuming you know when each medium "appeared"/"disappeared" in the room) using plain HTML5: this is what we do, for instance, for web based recordings at the IETF, https://ietf97.conf.meetecho.com/index.php/Recordings

This has the same considerations related to synchronous playback and the fact that by seeking you have to be sure recordings that are not currently active might be at the seeking target: anyway, it's in theory easier to to do with media files and HTML5, as there are JavaScript libraries that can help with that, and you have fixed info on lenghts and things like that. Another alternative is post-processing the files and then mixing them all together in a single video file, still taking into account the proper synchronization of all involved streams. Harder to generate, but much easier to render, as you just play a file that has all the synchronization stuff rendered already. Again, this is what we also do for the IETF, when publishing recordings on YouTube: https://www.youtube.com/playlist?list=PLC86T-6ZTP5gtLuoSjpTGO_mS5Ly2pfIS

Hope this helps,

Lorenzo

[Horsetopus]

It does help a lot, thank you.

I will do the same thing as you for replay.

But I still need to be able to play video synchronously for all client during sessions, and I don't want to handle it from the client side. I want identical frame of every clients.

I am trying a few things regarding rtp.

Here is my test:

With VLC, I open a file and stream it as rtp.

On another one I open the rtp stream.

It works fine as far as I can test. I can pause, seek, it's fluid and all.

VLC works great as command line tool and allow for a lot of commands, among witch pause and seek.

Tomorrow, I will convert the video to the appropriate codecs and test the streaming plugin.

But do you think it's worse pursuing, or can you already see a reason it would fail?

[Lorenzo Miniero]

I don't think the Streaming plugin would be enough. Besides, as I said, it doesn't seek.

You'd probably have to write a new plugin that is aware of multiple mjr files or mountpoints at the same time and of their place in the timeline, and can receive commands to then force a seek/pause/etc. on all the PeerConnections a user has created for the same recording context. Seeking/pausing/etc. would likely require timestamp and sequence number rewriting.

L.

[Horsetopus]

Either I really don't understand rtp (which is very likely) or you didn't get how I want to command the flux.

If rtp just is a stream of packets, all I want from the streaming plugin is to broadcast them throw WebRTC to the clients.

The manipulation of the video, I want to do by keeping an handle to the VLC process and sending it the commands for seeking and pausing. So of course I have to implement a communication protocol throw my app, but I shouldn't have to modify the plugin.

So I'm giving it a try anyway...

I converted to a vp8 opus webm video, and now I do this from vlc:

cvlc -I rc  /home/user/Documents/video.webm --sout='#rtp{dst=127.0.0.1,port-audio=5004,port-video=5005,name=test4,sdp=file:///home/user/Desktop/test4.sdp}' --sout-keep --loop

And it starts doing so... no complains.

I get the following info from the console:

[ Stream 1 ] Sample rate: 48000 Hz      Codec: Opus Audio (Opus)     Bits per sample: 32   Type: Audio
[ Stream 0 ] Display resolution: 320x180   Resolution: 320x180   Codec: Google/On2's VP8 Video (VP80)   Frame rate: 24.000384   Type: Video

And this SDP file:

v=0
o=- 15851819386715259645 15851819386715259645 IN IP4 virtual-box
s=test4
i=N/A
c=IN IP4 127.0.0.1
t=0 0
a=tool:vlc 2.2.2
a=recvonly
a=type:broadcast
a=charset:UTF-8
m=audio 5004 RTP/AVP 96
b=RR:0
a=rtpmap:96 opus/48000/2
a=fmtp:96 sprop-stereo=1
m=video 5005 RTP/AVP 96
b=RR:0
a=rtpmap:96 VP8/90000
a=rtcp:5006

Now I cannot seem to stream it through VLC. 1st it doesn't accept the audio codec, but also 2nd when I remove the audio from the SDP, it fails after 10 seconds telling me it didn't receive anything.

But rtp streams usually don't use those codecs... so I'm thinking maybe the streaming plugin will accept although VLC doesn't.

That's where I am at. I still hope this will work, although you are doubtful.

But I have problems with the streaming plugin configuration and/or communication.

Here is my janus.plugin.streaming.cfg:

[vlc-test]
type = rtp
id = 1
description = Opus/VP8 live stream coming from vlc
audio = yes
video = yes
audioport = 5004
audiopt = 111
audiortpmap = opus/48000/2
videoport = 5005
videopt = 100
videortpmap = VP8/90000

But once my room is created, I have the following dialogue with Janus:

>> {"janus":"attach","transaction":"bvbE77HpalLA","session_id":3638022464488655,"plugin":"janus.plugin.streaming"}

<< {"janus":"success","transaction":"bvbE77HpalLA","data":{"id":4085036045478945}}

>> {"janus":"message","transaction":"3NGa4wml8jCl","session_id":3638022464488655,"handle_id":4085036045478945,"body":{"request":"list"}}

<< {"janus":"success","transaction":"3NGa4wml8jCl","plugindata":{"plugin":"janus.plugin.streaming","data":{}}} 

As you can see, the list returns empty data.

Am I doing something wrong?

It's always a bit hard to figure out the right request to send Janus... a lot of trial and error.

[Horsetopus]

Scratch that about the listing request, it was my mistake.

[Horsetopus]

Ok, so it's kinda working.

For anybody trying to do the same, and because it's hard to find some examples of the communication protocol for the streaming plugin, here are my logs ( it's very similar to listening to a feed in the videoroom ):

http://pastebin.com/dwKdU88a

It's very similar to the listening part of the videoroom plugin so I copied passed a lot, there might be some unecesary variables.

So, yeah, it works but with a few issues for now.

I connect to the stream without issues, but I get some frame freezing and then skipping (every 5 seconds or so).

But pausing and resuming works.
Even seeking ahead, but not seeking back (although reopening the file and seeking seems to work).

The different clients are not synchronized though.

I think the frame skipping comes from some configuration issue.

I don't transcode the file, and neither my CPU remains very low, so maybe it has to do with the treaming.cfg file?
The two parameters I don't get are audiopt and videopt aka <audio/video RTP payload type>

what are they?

Should they vary depending on the video imputed? on the SDP?

Also, I noticed in the janus.plugin.streaming.cfg file that you support rtsp, and other codecs.

1/ rtps contains extra information regarding playback, right? could it help the with the synchronisation issue?

2/ is there a complete list of what codecs are supported by the streaming plugin?

Thanks.

[Lorenzo Miniero]

Ah ok, so you meant having VLC as the "source" of a Streaming mountpoint, and VLC itself to be responsible for pausing, resuming, seeking, etc. In that case it should work, as long as VLC generates a consistent RTP stream even when those controls are used (e.g., when you pause, it sends you a still image, timestamps and sequence numbers increase as expected, etc.), all things you would have needed to do yourself if you wanted to do the same on an .mjr source or a static RTP source, for instance.

As to frame skipping, synchronization issues and things like that, you'll have to look at what traffic VLC is generating, and if it is indeed keeping the RTP stream consistent as explained above, if key frames are being sent regularly, and things like that. RTP was conceived to do exactly that, synchronize streams, so it's likely something wrong in the RTP stream, which the Streaming plugin simply forwards as is.

I don't get the question on codecs. Janus doesn't care about codecs, and neither does the Streaming plugin. The pt stuff you set in the configure are the payload types the Streaming plugin will put in the SDP, so that the browser knows, for instance, that a pt=96 in RTP will be VP8.

L.