Chunksize IOBuffer and Aggregate messages
Alexandru
I'm finishing the implementation of handling aggregate messages. The
concrete type is 0x16. I'm doing this because FMS always send audio/
video like an aggregate type. My issue is like that: when a message
arrives and is bigger than 4096 bytes (the minimum size for IOBuffer),
when I read/Ignore the buffer, something wrong happens. The buffer
became corrupted and leave one byte somewhere , the structure of the
message is not being the same and the comparation between the size
that says back-buffer and the message header is not the same. I have
to say that I didn't follow the normal implementation reading the
chunk size as maximum size. If I detect that the message has 5000
bytes length, I read 5008/5011(depends on message header). This stuff
should work, even that I don't respect the max chunksize (because FMS
doesn't respect me and send me more than chunksize :-) ).
My questions are:
¿This is normal to corrupt the IOBuffer ignoring/reading an IOBuffer a
quantity of bytes more than 4096?
¿It's normal that FMS send me an Aggregate more that 4096 if he told
me that the chunk size is 4096 and I've reply him with the same?
I've seen that there's no mutex on IOBuffer implementation. I'm very
curious to know the technique of IOBuffer an which benefits/drawbacks
has, in the way that you do. That's why I've never seen a buffer
handled by a producer/consumer without any kind of carrier protection
(mutex, semaphore, etc) and your implementation seems to be very fast,
because I've test it in an 800 Mhz powerpc and crtmpserver can handle
160 connections like nothing :-)
Thank you.
King regard,
Alexandru.
C++ RTMP Server
I don't know exactly how you do it, but here are some facts about IOBuffer
1. IOBuffer is rock solid. It has been used in so many places with so many scenarios, that leaking is almost impossible. If IOBuffer crashes your code, it is its way of telling you that you are doing something terribly wrong! :)
2. NEVER cache/store "uint8_t *" pointers extracted from an IOBuffer instance outside the scope of simple function/block. Those are most likely changing and you end up accessing either zombie pointers or garbage data. Always use GETIBPOINTER(buffer). Same goes for the size of available data. Always use GETAVAILABLEBYTESCOUNT(buffer). So, whenever you alter the IOBuffer by either using read/write/ignore functions, ALWAYS refresh your local pointers with GETIBPOINTER(buffer) and GETAVAILABLEBYTESCOUNT(buffer)
Regarding chunk size... FMS always respects that chunk size. Otherwise, all the RTMP channel becomes screwed up beyond recognition
On Feb 12, 2012, at 1:06 PM, Alexandru wrote:
> Hello Sirs.
> I'm finishing the implementation of handling aggregate messages. The
> concrete type is 0x16. I'm doing this because FMS always send audio/
> video like an aggregate type. My issue is like that: when a message
> arrives and is bigger than 4096 bytes (the minimum size for IOBuffer),
> when I read/Ignore the buffer, something wrong happens. The buffer
> became corrupted and leave one byte somewhere , the structure of the
> message is not being the same and the comparation between the size
> that says back-buffer and the message header is not the same. I have
> to say that I didn't follow the normal implementation reading the
> chunk size as maximum size. If I detect that the message has 5000
> bytes length, I read 5008/5011(depends on message header). This stuff
> should work, even that I don't respect the max chunksize (because FMS
> doesn't respect me and send me more than chunksize :-) ).
>
> My questions are:
>
> ¿This is normal to corrupt the IOBuffer ignoring/reading an IOBuffer a
> quantity of bytes more than 4096?
NO! If that is happening, you do something wrong
> ¿It's normal that FMS send me an Aggregate more that 4096 if he told
> me that the chunk size is 4096 and I've reply him with the same?
No, is not normal. If that is "apparently" happening, that translates into a bug inside rtmpd :(
> I've seen that there's no mutex on IOBuffer implementation. I'm very
> curious to know the technique of IOBuffer an which benefits/drawbacks
> has, in the way that you do. That's why I've never seen a buffer
> handled by a producer/consumer without any kind of carrier protection
> (mutex, semaphore, etc) and your implementation seems to be very fast,
> because I've test it in an 800 Mhz powerpc and crtmpserver can handle
> 160 connections like nothing :-)
Everything is single threaded because rtmpd is solely and utterly I/O bound. It doesn't make absolutely no sense at all to have multiple threads when a single thread can saturate one 1Gbps network card. If more is needed, than spawn another rtmpd instance. Clustering is 100% available on the commercial evostream version (www.evostream.com). This is why rtmpd is so fast: it doesn't do any context switching and the code cache hit rate on the CPU is very high! :)
Best regards,
Andrei
>
> Thank you.
>
> King regard,
> Alexandru.
>
> You received this message because you are subscribed to "C++ RTMP Server" mailing list.
> To post to this group, send email to c-rtmp...@googlegroups.com
> To unsubscribe from this group, send email to
> c-rtmp-serve...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/c-rtmp-server?hl=en
Alexandru Ionut Grama
Well I'm trying to handle aggregate message. For now, FMS doesn't send aggregate message because them aren't send. Even if in the code is a parameter for the factory that generates the connect, when you build the variant for connecting, the capabilities are not assigned. That fact makes FMS set chunk size to 1024 and FMS doesn't send aggregates message. This sould not be like that: the capabilities should be send, because FMS is completly closed, and in some future version maybe FMS don't send you H.264 (who knows), like is happening with old style handshake. If you perform an old style handshake, the server don't send you H.264.
The point is that you're right. It seems that IOBuffer to be solid like a rock :-). My first thought when I see what was happening was a carrier condition provocated by a body size message higher than chunk size. The point is the next:
- An aggregate message is type 0x16 and it can has inside whatever the server wants.
- After the header of aggregate it becomes a special header telling the type (first byte), size (3 consecutives bytes), timestamp + extended timestamp (4 bytes) and the stream ID. That makes increases headers, even if Adobe says the opposite.
- After the message headers comes the message payload
- After the message payload comes a backpointer (4 bytes) that it's the same like the size of the message.
- Repeat this scheme (msg header + msg payload + backpointer) till you fill the chunk size
The algorithm of handling this is adding 2 new states to channel. MSG_H (when you read the header of msg) and MSG_PAYLOAD ( when you read the message payload). Next you have to handle the backpointer (4 bytes).After that you have to return to CS_HEADER if you read the whole body of chunk or MSG_H if you have to read more message in the aggregate.
When you detect an aggregate you have to store the header of the aggregate and left the current_header for the header of the message, changing only the the size, timestamp and type.
It works well with aggregate less than 4096, but when it comes a bigger aggregates, handle each message correctly except the last, because the last backpointer is not what has to be.
I've check that I work with GETIBPOINTER and GETAVALIBLEBYTES and yeah, I do it every time that I ignore a part.
¿Any clue of what it's happening? ¿May I send you the part of the code?
King regards,
Alexandru.
--
---------------------------------------------------------------
Alexandru Ionut Grama
email: gramaalex...@gmail.com