MXP for hi-speed gaming?

[kripken]

Hello everyone,

I am interested in possibly using MXP as the network protocol for a
future version of my virtual worlds/3D gaming platform, the Intensity
Engine [1]. A clean, standard protocol like MXP would be a good thing
for us, if it fulfills the basic condition of not degrading
performance. Unlike say Second Life, the Intensity Engine allows
people to build virtual worlds with elements of first person shooters
and other hi-speed games, and our currently network protocol is highly
optimized due to that.

Here are the main issues that are requirements for the Intensity
Engine, which our existing protocol already offers:

1. Both reliable and unreliable messages, where 'reliable' means that
if not received the message is resent, and messages are guaranteed to
be received in the correct order, and 'unreliable' means that no
resending is done, and messages sent earlier that arrive later are
silently ignored (in other words, a "Quake-like protocol"). (All of
this in UDP, of course.)

2. Separate 'channels', where the ordering relevant in point #1 is
maintained only within a channel (this lets the other channel continue
to push data).

3. Protocol compression.

4. A good C/C++ implementation. I recommend using ENet [2], which
supports reliable and unreliable messages, channels, etc., and is MIT
licensed. The only problem with it is that it uses exponential backoff
like TCP (this is disabled in the Intensity Engine).

5. Easy ability to define new message types, with some sort of message
template file, code autogeneration, etc.

6. Some messages, in particular position updates, should have the
ability to append additional world-specific data, that needs to arrive
at the same speed as position updates (i.e., fast - this might be
relevant for animations in some applications, etc.).

I'd appreciate feedback regarding the status of these points in MXP.

Best,
Kripken

[1] http://www.intensityengine.com
[2] http://enet.bespin.org/

[Tommi Laukkanen]

Hello Kripken

Thank you for composing this requirements list. I included my comments
below between quotes:

> 1. Both reliable and unreliable messages, where 'reliable' means that
> if not received the message is resent, and messages are guaranteed to
> be received in the correct order, and 'unreliable' means that no
> resending is done, and messages sent earlier that arrive later are
> silently ignored (in other words, a "Quake-like protocol"). (All of
> this in UDP, of course.)

This ordered delivery seems to be relevant requirement which is
missing from our requirements list. I am still a bit uncertain whether
we should feed messages to higher level engine which will then deduce
when to wait for earlier messages and when to apply them
independently. If it is implemented in network level entirely then the
messages in the reliable channel will all stall if one message is
delayed.

> 2. Separate 'channels', where the ordering relevant in point #1 is
> maintained only within a channel (this lets the other channel continue
> to push data).

I tend to agree that we need to add some kind of channel concept so
that we can provide reliable messages their own indexing which can be
used whether some message in between is delayed. Whether we should
allow for multiple channels or just 2 namely reliable and unreliable
is something we need to think about carefully.

> 3. Protocol compression.

I am uncertain how good compression levels can be acquired inside one
message. As good levels can maybe be acquired by hand optimizing key
messages. It could be good to allow for compression of extensions at
least as they may be considerably longer than the messages itself.

> 4. A good C/C++ implementation. I recommend using ENet [2], which
> supports reliable and unreliable messages, channels, etc., and is MIT
> licensed. The only problem with it is that it uses exponential backoff
> like TCP (this is disabled in the Intensity Engine).

I agree that we do need good c++ implementation and Enet certainly
looks like a robust library. The reservations I have is that it would
need to be well documented to be viable for standard like
specification. In addition I would think ports to languages like Java,
C# and python at least would be required. Native library bindings
being a pain when applying any library. On the other hand maybe using
network library like ENET could make MXP more desirable as network
library candidate for projects requiring high quality and proven
operation in heavy production use.

> 5. Easy ability to define new message types, with some sort of message
> template file, code autogeneration, etc.

MXP approach is to provide sort of transport layer messages which can
be extended with google protocol buffers to cover application specific
needs. Instead of ending up with a jungle of hundreds of custom
messages like SLUDP we have well defined extensible vocabulary
(message set).

> 6. Some messages, in particular position updates, should have the
> ability to append additional world-specific data, that needs to arrive
> at the same speed as position updates (i.e., fast - this might be
> relevant for animations in some applications, etc.).

We have in a requirement list a custom object modification message
which would carry this kind of application specific data. Our approach
is currently to optimize tightly key signals like movements in their
own messages. We will likely need to do something similar for free
style gesturing where you need to transmit skeletal animation signal
in similar manner as movement.

-tommi

[Hurliman, John]

This requirements list sounds like a description of SCTP: http://en.wikipedia.org/wiki/Stream_Control_Transmission_Protocol

John

[Kripken]

Yes, it is somewhat like a streamlined/specialized version of SCTP.

- Kripken

[Tommi Laukkanen]

John, do you think SCTP could serve as transport layer for network
virtual environment interoperation protocol? Is there native
implementations for required platforms like different windows versions
and linuxes. How about language ports to Java/C# and python? If
implementation is not too complex maybe missing ports could be done.

-tommi

[Hurliman, John]

My understanding is that the SCTP driver in the Linux kernel is very stable at this point. There is also a Windows driver released by a Japanese team, but I do not know the status of SCTP in OSX. The API for both platforms is a C header file that looks similar to a normal TCP socket interface but with some more options. That interface would need a C# wrapper written. Not sure about the Java status, I think the only other language I saw wrappers for was Python (even that I would have to double check). SCTP seemed a bit ahead of its time when it was created and has mostly been treated as an experimental research project, but it seems to be a perfect fit for state melding in virtual worlds. You still need to define messages on top of it using something like Google Protobufs, but SCTP takes care of everything else (marking things as reliable, unreliable, messages, streams, channels, ordered, unordered, etc).

John

>-----Original Message-----
>From: metaverseexc...@googlegroups.com
>[mailto:metaverseexc...@googlegroups.com] On Behalf Of Tommi
>Laukkanen
>Sent: Tuesday, July 07, 2009 11:29 PM
>To: metaverseexc...@googlegroups.com
>Subject: Re: MXP for hi-speed gaming?
>
>