Generation of live CMAF content (using sockets)

359 views
Skip to first unread message

MV

unread,
Nov 20, 2017, 2:17:32 PM11/20/17
to Shaka Packager Users

Hello,

I'm working with TS segments whose properties (resolution or bitrate) change sligthly within a same quality with the aim of saving bandwith. My goal is to send several Transport stream segment buffers in N qualities over N different sockets to generate the corresponding live CMAF content using Shaka Packager.

I'm having a weird behaviour, because I have another program (within a pipeline) that sends the memory buffers of the segments over a datagram socket, Shaka packager listens in those ports and warns us with the following messages:

[1120/202246:WARNING:h26x_byte_to_unit_stream_converter.cc(91)] Seeing varying NAL unit of type 7. You may need to set --strip_parameter_set_nalus=false during packaging to generate a playable stream.
[1120/202246:WARNING:es_parser_h264.cc(133)] H.264 decoder configuration has changed.

Shaka generates MP4 segments, but I see a lot of errors when I try to play the playlist (a MPD manifest with GPAC Osmo player).
Also I have tried to change the value of  "strip_parameter_set_nalus" to false in h26x_byte_to_unit_stream_converter.cc, but I don't know well the implications of this modification and the playback is not correct yet.

While if I write those segment in disk before and then an independent program reads those files from disk (e.g. segment_0001.tz) and sends its content over the sockets (instead of passing its memory buffer directly), Shaka packager doesn't print those messages and the playback is correct.

I have checked that the number of bytes sent in each case is equal to the length of the corresponding segment buffer and segments from different qualities are sent over different ports.

I know that it is difficult to understand my problem (neither I understand what it is happening), but maybe someone has some suggestion.

Thanks in advance.

KongQun Yang

unread,
Nov 20, 2017, 7:34:16 PM11/20/17
to MV, Shaka Packager Users
Hi,

I don't understand why the result for, (1) Using memory buffer and (2) Writing to disk first, are different. It sounds like you may have sent different qualities of content in the same socket in (1) since you are only seeing the configuration change warning message in (1). Can you check again? You can change your code to package only one quality to confirm.

If the configuration does change, try change 'true' to 'false' in https://github.com/google/shaka-packager/blob/05a5a419693b84fb6a8a0f1808327c08fce9958e/packager/media/codecs/h26x_byte_to_unit_stream_converter.cc#L19 and recompile the library then try again. This will only work if there is minor changes in the configuration otherwise the player will still reject it.





-- KongQun Yang (KQ)

--
You received this message because you are subscribed to the Google Groups "Shaka Packager Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to shaka-packager-users+unsub...@googlegroups.com.
To post to this group, send email to shaka-packager-users@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/shaka-packager-users/f678943d-2bb7-4e9d-9259-61a558217f2d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

MV

unread,
Nov 29, 2017, 10:11:41 AM11/29/17
to Shaka Packager Users
Hello,

I have a new doubt about live streaming with Shaka packager.
This is my situation: on the one hand, I have my program which generates Transport-Stream segments (.ts), then I send those buffers over a datagram socket (with a port) to Shaka Packager (I do in this way, because it is the only alternative that I found out) and Shaka Packager listens at that port and generates the corresponding CMAF segments.

Now I want to get low latency in live streaming with CMAF and I'm working with chunks. I see that the input parameters related to chunks are: segment duration and subsegment duration. I have tested to generate the content with subsegments and it looks like correct: a styp box followed by a sidx box that contains the list of chunks (their duration, size and flags like if it starts with IDR, etc) and then, a moof-mdat pair per each subsegment. But I need that the segment to be available from the first chunk (I don't know if I'm explaining clearly), because I understand that the CMAF segment is written in disk when its subsegments are available. In conclusion,  I need that the segment start to be written from its first chunk in order to the player can play it and doesn't have to wait for the whole segment. Is this possible with Shaka packager?

Thanks a lot.

-- KongQun Yang (KQ)

To post to this group, send email to shaka-pack...@googlegroups.com.

KongQun Yang

unread,
Nov 29, 2017, 2:04:19 PM11/29/17
to MV, Shaka Packager Users
Hi,

My understanding on CMAF's low latency live streaming is that 

(1) Packager generates subsegments in separate files (packager does not support it yet)
(2) The subsegments are concatenated by streaming server or at CDN, so it is transparent to player, which is not aware of and does not need to be aware of the subsegments.

My understanding seems to be a little bit different to what you have in mind. See my replies below.

-- KongQun Yang (KQ)

On Wed, Nov 29, 2017 at 7:11 AM, MV <v.martin....@gmail.com> wrote:
Hello,

I have a new doubt about live streaming with Shaka packager.
This is my situation: on the one hand, I have my program which generates Transport-Stream segments (.ts), then I send those buffers over a datagram socket (with a port) to Shaka Packager (I do in this way, because it is the only alternative that I found out) and Shaka Packager listens at that port and generates the corresponding CMAF segments.

Now I want to get low latency in live streaming with CMAF and I'm working with chunks. I see that the input parameters related to chunks are: segment duration and subsegment duration. I have tested to generate the content with subsegments and it looks like correct: a styp box followed by a sidx box that contains the list of chunks (their duration, size and flags like if it starts with IDR, etc) and then, a moof-mdat pair per each subsegment.
 
But I need that the segment to be available from the first chunk (I don't know if I'm explaining clearly), because I understand that the CMAF segment is written in disk when its subsegments are available. In conclusion,  I need that the segment start to be written from its first chunk in order to the player can play it and doesn't have to wait for the whole segment. Is this possible with Shaka packager?

I am not sure if I understand correctly. Are you suggesting that: whenever a new chunk is available, create a new segment if it is the first chunk otherwise append to the segment created earlier? Do we need streaming server's assistance so player can handle the update in segments correctly?


Thanks a lot.

On Tuesday, November 21, 2017 at 12:34:16 AM UTC, KongQun Yang wrote:
Hi,

I don't understand why the result for, (1) Using memory buffer and (2) Writing to disk first, are different. It sounds like you may have sent different qualities of content in the same socket in (1) since you are only seeing the configuration change warning message in (1). Can you check again? You can change your code to package only one quality to confirm.

If the configuration does change, try change 'true' to 'false' in https://github.com/google/shaka-packager/blob/05a5a419693b84fb6a8a0f1808327c08fce9958e/packager/media/codecs/h26x_byte_to_unit_stream_converter.cc#L19 and recompile the library then try again. This will only work if there is minor changes in the configuration otherwise the player will still reject it.





-- KongQun Yang (KQ)

On Mon, Nov 20, 2017 at 7:17 PM, MV <v.martin....@gmail.com> wrote:

Hello,

I'm working with TS segments whose properties (resolution or bitrate) change sligthly within a same quality with the aim of saving bandwith. My goal is to send several Transport stream segment buffers in N qualities over N different sockets to generate the corresponding live CMAF content using Shaka Packager.

I'm having a weird behaviour, because I have another program (within a pipeline) that sends the memory buffers of the segments over a datagram socket, Shaka packager listens in those ports and warns us with the following messages:

[1120/202246:WARNING:h26x_byte_to_unit_stream_converter.cc(91)] Seeing varying NAL unit of type 7. You may need to set --strip_parameter_set_nalus=false during packaging to generate a playable stream.
[1120/202246:WARNING:es_parser_h264.cc(133)] H.264 decoder configuration has changed.

Shaka generates MP4 segments, but I see a lot of errors when I try to play the playlist (a MPD manifest with GPAC Osmo player).
Also I have tried to change the value of  "strip_parameter_set_nalus" to false in h26x_byte_to_unit_stream_converter.cc, but I don't know well the implications of this modification and the playback is not correct yet.

While if I write those segment in disk before and then an independent program reads those files from disk (e.g. segment_0001.tz) and sends its content over the sockets (instead of passing its memory buffer directly), Shaka packager doesn't print those messages and the playback is correct.

I have checked that the number of bytes sent in each case is equal to the length of the corresponding segment buffer and segments from different qualities are sent over different ports.

I know that it is difficult to understand my problem (neither I understand what it is happening), but maybe someone has some suggestion.

Thanks in advance.

--
You received this message because you are subscribed to the Google Groups "Shaka Packager Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to shaka-packager-users+unsubscrib...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Shaka Packager Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to shaka-packager-users+unsub...@googlegroups.com.

MV

unread,
Nov 30, 2017, 5:31:48 AM11/30/17
to Shaka Packager Users
Hello,

Thanks for your answer.
I have been searching about how to implement low latency with CMAF, but I haven't found out anything that provide enough detail.

Taking into account your answer, I imagine that the subsegments will be generated in separate files and when all subsegments of a particular segment will have been generated, the whole segment will be built in fragmented MP4 format and the subsegments will be delete from disk. The streaming server (I think) should be able to deliver the latest chunk of the segment that is being requested by the player.

Regards.

-- KongQun Yang (KQ)


-- KongQun Yang (KQ)

To unsubscribe from this group and stop receiving emails from it, send an email to shaka-packager-users+unsub...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Shaka Packager Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to shaka-packager-users+unsub...@googlegroups.com.
To post to this group, send email to shaka-pack...@googlegroups.com.

KongQun Yang

unread,
Nov 30, 2017, 4:22:22 PM11/30/17
to MV, Shaka Packager Users
Hi,

Here is what I think:

> when all subsegments of a particular segment will have been generated, the whole segment will be built in fragmented MP4 format and the subsegments will be delete from disk

The segment can be built by simply concatenating all subsegments. This step is optional depends on how it is handled in the streaming server.

> The streaming server (I think) should be able to deliver the latest chunk of the segment that is being requested by the player. 

Here is one possible implementation:

The streaming server is responsible for sending the subsegments/chunks continuously to the player. For example, when player requests Segment_N, initially there is only one chunk available, Segment_N_Chunk_0; streaming server sends it to the player; it should not close the connection, but wait for Segment_N_Chunk_1 to be available and send it to the player in the same connection; it goes on until the last chunk in the segment is sent to the player.

This requires work in both Shaka Packager and streaming server. I don't expect a lot of work though.


-- KongQun Yang (KQ)


-- KongQun Yang (KQ)


-- KongQun Yang (KQ)

To unsubscribe from this group and stop receiving emails from it, send an email to shaka-packager-users+unsubscrib...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Shaka Packager Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to shaka-packager-users+unsubscrib...@googlegroups.com.

To post to this group, send email to shaka-pack...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "Shaka Packager Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to shaka-packager-users+unsub...@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages