Benchmarks in other languages

[sauerbrat]

The benchmarks here look very impression:

http://google.github.io/flatbuffers/flatbuffers_benchmarks.html

I'm assuming these are using the C++ flatbuffers implementation?

Have benchmarks been done on other languages? In particular I am interested in Python and JavaScript performance. Do those languages by their nature lose the "zero-allocation" feature that flatbuffers uniquely has?

Also, a tangential question:

Suppose I'm writing a web-based first-person shooter type game where I want the server to provide real-time updates to the clients at 33fps over JS websockets, using a Python server. 

With 100 players, this would mean the server receives and needs to parse 100*33 packets per second. Each structure would need to include the player's keys pressed or other similar information. Then, the server needs to process that information, update its game state (apply physical calculations, update positions of players and objects based on velocity and inputs), and serialize that and send out the packets.

I think the server performance would be most important here, since the server needs to process much more information than the client, since it needs to communicate with all the ~100+ clients.

Based on the Python JSON benchmarks here: http://artem.krylysov.com/blog/2015/09/29/benchmark-python-json-libraries/ (each test is over 100 runs, I think)

It seems that all deserializations and serializations for a tick could take on the magnitude of 20ms (less with better hardware). If my budget is only ~30ms, (for a 33 tickrate server), then this is eating up a large chunk of time.

How much advantage would using flatbuffers have over JSON?

[Wouter van Oortmerssen]

Yup, they're for C++ (made that obvious in the docs just now).

Other languages lose zero allocation to some extent, as it requires accessor objects to read the data, and usually conversions to native strings, but depending on the data and access pattern it can still be a lot more efficient than converting it all to objects. But yes, FlatBuffers was designed for languages like C++, and other languages may not benefit quite as much.

If you are so performance bound that you wish to serve a certain amount of players in a certain time, then even if FlatBuffers is a lot faster than JSON, you're going to lose it all by using a language like Python. Using a faster language there will give you bigger gains than using a faster serialization format, unless of course you're bottlenecked by network speed :)

--
You received this message because you are subscribed to the Google Groups "FlatBuffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email to flatbuffers+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Andre]

Thanks for the response.

If flatbuffers were designed for languages like C++, how would you describe the purpose of all the integrations with other languages? Is it just so that less CPU-bound applications (possibly like a single JS client parsing data 33 times a second to update a 3D WebGL scene), can interoperate well with a server where serialization/deserialization performance is critical?

To confirm your last point: are you saying that you think that the penalty in using Python for game logic would dwarf JSON serialization/deserialization overhead?

To unsubscribe from this group and stop receiving emails from it, send an email to flatbuffers...@googlegroups.com.

[Wouter van Oortmerssen]

The first motivation for supporting languages is interop (projects are often composed of many languages, a serialization system that supports just one is useless). FlatBuffers was designed for games, where often high performance and low memory usage is desired mostly on the client, where at least the engine (and sometime parts of the gameplay) are in C++.

That said, we're hoping that reducing garbage collector churn in other languages is a win. For example, in C#, all accessor objects are by-value structs, so all FlatBuffer access that are not strings is entirely allocation free. Java may do escape analysis. JS compilers are getting pretty impressive.

Python is just a really slow language, compared to all the alternatives. C++ is very fast, and C#/Java/Go/JS are decently fast. Python and PHP are really slow. It of course depends on your gameplay logic, number of clients, and networking, and all sorts of factors, but if you have performance goals, it is dangerous starting with Python. Certainly a fast JSON implementation in a fast language will beat FlatBuffers in Python. Of course, FlatBuffers in a fast language is ideal :)

To unsubscribe from this group and stop receiving emails from it, send an email to flatbuffers+unsubscribe@googlegroups.com.

[Andre]

One question, why do you say that high performance and low memory usage "is desired mostly on the client"? It seems that the server connected to many clients would need to do more serialization/deserialization than an individual client.

[Wouter van Oortmerssen]

This is specific to games, especially mobile games, that are very performance sensitive. Server hardware is not nearly as constrained, and people seem to use all sorts of (not so fast) languages on the server.

To unsubscribe from this group and stop receiving emails from it, send an email to flatbuffers+unsubscribe@googlegroups.com.

[Andre]

Got it, thank you. I was thinking about a server-heavy multiplayer game engine like the Source Engine by Valve; I think in that architecture it's possible for the server be more strained.

(source)

By the way, I really enjoyed playing Sauerbraten back in the day :)

Cheers

[Wouter van Oortmerssen]

On Thu, Aug 17, 2017 at 3:03 PM, Andre <an...@andrehaas.com> wrote:

Got it, thank you. I was thinking about a server-heavy multiplayer game engine like the Source Engine by Valve; I think in that architecture it's possible for the server be more strained.

Yeah, for a fast action game, for sure. I thought they don't make those anymore, though :P

 

By the way, I really enjoyed playing Sauerbraten back in the day :)

Hah, small world! Thanks :)