Re: KristalliProtocol work-in-progress specification]

[Tommi Laukkanen]

Hi

I have been following Kirstalli-thread in your list but have not yet
found time to read through the spec and participate. I believe that
merge of Kristalli and MXP transport layer would launch the virtual
world protocol development to useful direction by combining the
strengths of our projects. We could start by creating a simple feature
level comparison chart between MXP 0.5, planned MXP 0.6, Kristalli and
ENET. This would give us high level view on the situation and a
starting point for merge considerations. From our point of view this
is a good time as we have managed to create pretty solid 0.6
specification with features we wanted to put in but the implementation
has not been started yet. It is good time to evaluate the feature set
and see what kind of improvements can be done by merge of these two
independent approaches.

I will try and read through the specification by next Monday and
produce skeleton of the feature set comparison matrix. We can also add
list of protocol level innovations which should be specifically noted
when considering the final product.

Apologies about cross posting but I thought this would be interesting
for readers in both projects.

Yours sincerely,
Tommi Laukkanen

[Tommi Laukkanen]

Hello

Here is link to the initial comparison matrix:

http://www.bubblecloud.org/wiki/-/wiki/Main/Comparison

Please notify me of any errors or ambiguities. You can also register and modify the wiki yourself and copy the matrix to your own materials.

We could also add SLUDP there but I would need someone with experience to contribute that column or help me with the analysis. 

Regards,

Tommi

--
You received this message because you are subscribed to the Google Groups "Metaverse eXchange Protocol" group.
To post to this group, send email to metaverseexc...@googlegroups.com.
To unsubscribe from this group, send email to metaverseexchangep...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/metaverseexchangeprotocol?hl=en.

[jukka....@ludocraft.com]

> Hello
>
> Here is link to the initial comparison matrix:
>
> http://www.bubblecloud.org/wiki/-/wiki/Main/Comparison

...

> Regards,
> Tommi
>

I'll say it does not do well to reduce the features to a simple binary
on/off comparison, since even when supported, the details may vary wildly.
Perhaps add footnotes or similar to add clarifications where needed?

Oh well, here goes anyway:

SLUDP
-----
Supported:

Connectivity
- Partial. No connection establishment - only keepalive and disconnection
specified
Optional guaranteed delivery
Separated transport layer
Open Source License
UDP support
C++ Windows support
C++ Linux support
C++ Mac support
Mature and widely adopted implementation
Open Specification
Native C# support

Not supported:

Message assembly
Message fragmentation
Sequencing
Channels
Prioritization
Congestion avoidance
Encryption
TCP support

Unknown:

C++ Solaris support
Native Java support
Native Python support
Binding Python support

Raknet
------
Supported:

Connectivity
Optional guaranteed delivery
- Uses both ACK and NAK schemes.
Message assembly
Message fragmentation
Sequencing
Channels
Prioritization
Congestion avoidance
Encryption
- http://www.jenkinssoftware.com/raknet/manual/secureconnections.html
Separated transport layer
Open Source License
- OS-like for indie development, commercial for business use
UDP support
TCP support
C++ Windows support
C++ Linux support
C++ Mac support
Binding Python support
- pyraknet
Mature and widely adopted implementation

Unknown:

C++ Solaris support

No:

Open Specification
- documented on a high-level, can dig up details from sources. No
official detailed spec available, and I doubt the company will welcome
free implementations along their commercial one.

AFAIK, none of the following officially supported:
Native C# support
- Apparently being ported to a managed assembly, but not official
Native Java support
Native Python support

---------------

Other thoughts:

- Compression: Of course at message payload level anyone can do anything
he wishes with any of these protocols. At whole datagram level,
implementing compression is cumbersome. I wonder what kind of compression
features you were seeking for here? SLUDP has a form of compression
(zero-encoding) and raknet has a form of compression (VLE'd 0/1
truncation) - would these apply? We employ RLE and zlib at a message level
in Kristalli, but there is no ready option for "set this flag to true and
all data is compressed".

- Multiplexing: Can you elaborate on this? I'm not really sure about this
one, and I left it out completely from above.

- KristalliProtocol does support multiple channels. In fact, the message
ordering feature that it supports is more general than just N separate
communication channels.

- Also, KristalliProtocol over UDP implements Congestion Avoidance pretty
much in the same way as TCP does. With this respect, I find the sentence
'It is not enough to allow application to manually limit the bandwidth
according to metrics as bad behaving application may jam the network.' a
bit contradicting. I'd change this to say 'It is not enough to leave it as
a burden of the application level to manually limit the bandwidth
according to metrics, as it requires library users to always implement
their own.' Having the *option* of managing the flow control is a
necessity e.g. for streaming audio/video applications, where the
application must be able to react quickly to network conditions, say, by
varying bitrate. In these cases, a single fixed specified black-box
congestion control mechanism, like what TCP offers, is not ideal. This is
why Kristalli over UDP basically says "here's the default method, but
override it whatever you want if necessary."

- You put in C++ Mac support for Kristalli. I don't think such exists, or
were you thinking of boot camp ;) or something similar? Or has someone
been really busy working on a mac implementation?

Thumbs up for the interesting work,

Jukka

[Ryan McDougall]

You should include the sirikata guys, or maybe use the kyoryoku list,
to get a wider audience.

Cheers,

[Tommi Laukkanen]

Hi

Big thanks for all this data and good comments. I have updated the matrix wiki page accordingly. Raknet is interesting addition as commercial benchmark.

Does SLUDP have separated transport layer so that you can use it independently from SLUDP message layer? It is interesting for us (MXP team) if we could use some already existing OS library as transport layer.

I added practical example of the multiplexing to Wiki for clarification.

-tommi

[jukka....@ludocraft.com]

> Hi
>
> Big thanks for all this data and good comments. I have updated the matrix
> wiki page accordingly. Raknet is interesting addition as commercial
> benchmark.
>
> Does SLUDP have separated transport layer so that you can use it
> independently from SLUDP message layer? It is interesting for us (MXP
> team)
> if we could use some already existing OS library as transport layer.

Yes. SLUDP has a message ID field for each message. A few years ago when
the RealXtend project was in its inception, we extended our own messages
using unused message IDs. However, due to practical difficulties with the
protocol libraries, some of these extensions were later on scrapped, and
we stopped adding new messages this way. SLUDP also has a single message
called GenericMessage, inside which you can serialize whatever you want.
Each GenericMessage has an ascii string to identify what the message
actually is, and RealXtend uses custom application messages based on this
structure.

The specifics can be seen here
http://wiki.secondlife.com/wiki/GenericMessage. As one can see,
serializing a whole message protocol inside this is very ineffective.

>
> I added practical example of the multiplexing to Wiki for clarification.
>
> -tommi
>

Ah, now it makes more sense. None of Kristalli, SLUDP or raknet support
multiplexing.

Jukka

[Tommi Laukkanen]

Lets continue on this thread to include Sirikata and other interested
parties:

http://groups.google.com/group/kyoryoku/browse_thread/thread/8bf0f57e888480f5

[Daniel Reiter Horn]

I can't figure out how to edit the page, so I've included it in an HTML email to your list--
You guys forgot a row that's really important: native javascript support, because there's no magical "binding layer" in javascript that you can just paste in C++ code, and javascript does not allow for truly binary strings, so you have to base128 or base64 encode things or use firefox extensions for binary unicode.
This is why simply TCP support is insufficient for javascript support

Transport Layer Comparison Matrix

Feature	MXP0.5	MXP0.6	ENET	Kristalli	SLUDP	Raknet	TCPSST(Sirikata)
Connectivity	+	+	+	+	+	+	+
Optional guaranteed delivery	+	+	+	+	+	+	+
Message assembly	+	+	+	+		+	+
Message fragmentation	+	+	+	+		+	+
Multiplexing	+	+					+
Sequencing		+	+	+		+	+
Channels		+	+	+		+	+
Prioritization				+		+
Congestion avoidance		*	+	+		+	+
Compression					+	+
Encryption						+
Separated transport layer		+	+	+	+	+	+
Open Source License	+	+	+	?	+		+
Open Specification	+	+		+	+		+(WebSockets)
UDP support	+	+	+	+	+	+
TCP support		*		+		+	+
C++ Windows support	+	*	+	+	+	+	+
C++ Linux support	+	*	+		+	+	+
C++ Solaris support	^	*	+		?	?	+
C++ Mac support	+	*	+		+	+	+
Native C# support	+	*			+	^	(easy)
Native Java support	+	*			?
Native Javascript Support							+ (webSockets)
Native Python support	^	*			?
Binding Python support			+		?	+	+

Mature and widely adopted implementation

+

+

+

'+' Supported, ' ' Not supported, '^' Developed, '*' Planned

[Ryan McDougall]

It seems to me that the differences between implementations are
relatively minor (excluding say SLUDP or RakNet). Is it not possible
to just pick one of them and use them uniformly? Or are those minor
differences worth it?

Cheers,

On Sun, Jan 31, 2010 at 11:28 PM, Daniel Reiter Horn
<dani...@graphics.stanford.edu> wrote:
> Tommi wanted me to take this discussion to the kyoryoku list, replies
> inline about the new protocol
>
> On Sun, 2010-01-31 at 22:02 +0200, Tommi Laukkanen wrote:
>> Hi Daniel
>>
>>
>> Thank you for the TCPSST data. It looks very well shaped. Is TCPSST
>> your existing protocol you use in Sirikata currently or a new thing
>> being developed?
> It is currently in use in sirikata! We've revved the master branch to
> include the websocket specification
>>
>>
>> Some clarifications:
>>
>>
>> Optional guaranteed delivery is something you can do only with UDP as
>> you need to be able to mark some of the messages unreliable so that
>> those packets can be dropped by the network when they are in flight.
> The API lets you mark packets as optional and the plan was to
> potentially favor dropping them if the send queues are full and new data
> needs to be added on the application side, but none of that has been
> codified, and so it's probably not fair for us to check that--- but
> there are some queues for double buffering during async send, so you
> could imagine dropping there optionally. Right now we do have a
> simulation for this functionality so you can develop against it, but
> it's not good ;-)
>
>>  This is important for rapid resolving of congestion in the
>> connection. TCP just keeps retransmitting all data which has been send
>> to the wire. I need to clarify this to the wiki page.
> We work around the head of line blocking issue by optionally allowing
> more than 1 TCP connection to a place, so that in the case of loss at
> least other streams can send.  Then packets marked unordered will be
> placed on the queues of less congested streams.  So we have some
> functionality but it's not directly comparable to the UDP analogs.
>
> A reminder that we chose the TCP route so we could support all languages
> including javascript (which does not have access to UDP for the
> foreseeable future). We are developing a UDP component to the TCPSST
> protocol that will allow for higher performing access to the same API,
> but the implementation is still in its very early stages and right now
> limited to our research code only.
>
>>
>>
>> Could you provide me with link to WebSockets specification so I can
>> add it to the wiki page?
> http://tools.ietf.org/html/draft-hixie-thewebsocketprotocol-70
>
>>
>>
>> You have ported your entire network library to pure JavaScript?
> As of last night (I think) yes.  If you need immediate access to the
> code I can point it to you, but the webserver is my dev machine so I
> wouldn't want to post my IP to a public list for everyone to hammer.
> We'll be uploading in less than a week to github and it's already under
> the BSD license, so if you ask me for early access and host it yourself
> you can redistribute.
>>
>>
>> I would appreciate if you replied here and quoted my question so we
>> can move this discussion to kyoryoku list:
> done
>>
>>
>>
>> Another thing which came to my mind: What kind of encoding you are
>> using for in wire format as you need to be able to decode it with
>> javascript? Do you have some json/xml like explicit encoding syntax or
>> message/packet structure contract (like TCP/UDP packet structure for
>> instance)?
>> http://groups.google.com/group/kyoryoku/browse_thread/thread/8bf0f57e888480f5
>>
>
> So TCPSST, like websockets has null/0xff delimited packets and length
> delimited packets.  If a TCPSST compliant server receives a null/0xff
> delimited packet then TCPSST expects items to be base64 encoded
> (otehrwise it's raw binary)--Either way we use the exact google protocol
> buffers format.  We wrote a native javascript protobuf parser protojs
> ( http://github.com/sirikata/protojs ) that we use to encode these items
> directly to base64.
>
> The protobuf spec is well documented and open source BSD Licensed
> implementations exist for all major languages that we know of (with our
> contribution there being the notably missing javascript implementation)
>
>
> Hope that answers your questions!
> -Daniel
>>
>> Best regards,
>> Tommi Laukkanen
>>
>>
>
>
> --
> http://groups.google.com/group/kyoryoku