Game hiscores: flat files or database?

[Felix E. Klee]

Somewhat as a training, I am developing a puzzle game. A first version
was entered into the js13kGames competition:

http://js13kgames.com/entries/rotogamesq

In that version, hiscores are simply stored in the browser. For the next
version, my idea is to store hiscores on a server running Node.js /
express, hosted on Nodejitsu. How it could work:

Server starts: Hiscore lists for all puzzles are loaded from disk into
server memory.

User opens game in browser:

1. Browser loads game. Via Socket.IO the hiscore lists for all puzzles
are transferred to the browser.

2. User finishes one puzzle, gets into hiscore list, and enters name.

3. Via Socket.IO, data is sent to the server: puzzle, moves, name

4. The server verifies that the moves are correct, using a background
process.

5. Name and new hiscore are entered into puzzle's hiscore list.

6. The puzzle's hiscore list is written to disk.

7. Using Socket.IO, the updated hiscore list is broadcasted.

Now I wonder what is the simplest way to perform step 6. I see two
options:

* (A) Store hiscore list in a file `hiscores.json`, associated with
the puzzle. Problems:

- There *may* be concurrency issues. If two users finish the same
puzzle near simultaneously, then `fs.writeFile` will be called
in quick succession. I am not sure whether this could lead to
file corruption.

- For creating a backup, it doesn't seem possible to *download*
files from Nodejitsu, unless I write my own software for that.

- Although unlikely, suppose I scale up, and two Node.js are
serving the game. In this case, there are further concurrency
issues due to the hiscore lists held in memory not being shared
among the processes.

* (B) Store hiscore list in database, e.g. Mongo. Problems: I don't
see any, except that the solution feels fat, a database needs to be
set up, and I prefer individual files for simplicity.

After writing this email, I tend to option (B), but still:

Which option would you chose? Something else? Suggestions?

[greelgorke]

b) redis: http://redis.io/topics/introduction
its less puzzling than files, has replication and pubsub, and atomic operations

[Felix E. Klee]

On Mon, Oct 1, 2012 at 12:55 PM, greelgorke <greel...@gmail.com>
wrote:
> b) redis: http://redis.io/topics/introduction

Thanks for the suggestion. However, if using a database, then I prefer
Mongo. Performance is not an issue. Persistence, on the other hand, is a
big issue: If someone spent hours to crack the hiscore, then he will be
disappointed if his entry goes away due to a server crash.

[Pedro Teixeira]

http://redis.io/topics/persistence

-- 

Pedro

--

Job Board: http://jobs.nodejs.org/

Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines

You received this message because you are subscribed to the Google

Groups "nodejs" group.

To post to this group, send email to nod...@googlegroups.com

To unsubscribe from this group, send email to

nodejs+un...@googlegroups.com

For more options, visit this group at

http://groups.google.com/group/nodejs?hl=en?hl=en

[Felix E. Klee]

On Mon, Oct 1, 2012 at 1:50 PM, Pedro Teixeira
<pedro.t...@gmail.com> wrote:
> http://redis.io/topics/persistence

Too complicated and/or not reliable enough.

At *any* point in time, I want to have the database mirrored to disk, in
its entirety. Mongo seems perfect. I wonder why people recommend Redis.

[Pedro Teixeira]

Did you read the append-only:true AOF part of the link I sent you?

-- 

Pedro

[Arnout Kazemier]

Then why ask the question if you already made your mind? You clearly don't want to use anything else then mongodb because

you seem to know better and the only thing that redis can do is crash anyways, mongo on the other hand is web scale and build

to solve all the issues in the universe that people throw at it.

As for A) there a couple of "decent" databases written in Node, but using a database is better, because at some point you are going

to scale out of process, or use multiple apps that are not hosted on the same process.. Simplicity is nice and all but it's not always

the best answer on all the questions.

[Felix E. Klee]

On Mon, Oct 1, 2012 at 3:09 PM, Arnout Kazemier <in...@3rd-eden.com>
wrote:

> Then why ask the question if you already made your mind?

I'm far from having made up my mind. I just wonder why people recommend
Redis when performance is not an issue here. After, all solving a puzzle
may take some minutes, and only once in a while someone will get into
the hiscores table.

[Felix E. Klee]

On Mon, Oct 1, 2012 at 3:00 PM, Pedro Teixeira
<pedro.t...@gmail.com> wrote:
> Did you read the append-only:true AOF part of the link I sent you?

Yes. I also now read that from time to time Redis rewrites the log so
that - I assume - entries that have been deleted will not appear
anymore. Sounds good!

However, it is recommended to not use AOF alone, possibly making things
more complicated than when using Mongo:

"There are many users using AOF alone, but we discourage it since to
have an RDB snapshot from time to time is a great idea for doing
database backups, for faster restarts, and in the event of bugs in the
AOF engine."

Also, I don't know what happens when the Redis server is restarted. Will
it automatically read the log to recreate the last database status?

I really don't understand what is the particular advantage of using
Redis vs. Mongo for the use case. Redis looks more complicated to set up
and possibly to maintain.

[Brian Kardell]

Brian Kardell :: @bkardell :: hitchjs.com

> --
> Job Board: http://jobs.nodejs.org/
> Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
> You received this message because you are subscribed to the Google
> Groups "nodejs" group.
> To post to this group, send email to nod...@googlegroups.com
> To unsubscribe from this group, send email to
> nodejs+un...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/nodejs?hl=en?hl=en

If the only reason you would even consider a db over a file is a very, very obscure case then why not just set a semaphor and retry after a second or two if it is still open?

[Felix E. Klee]

On Mon, Oct 1, 2012 at 3:41 PM, Brian Kardell <bkar...@gmail.com>
wrote:

> If the only reason you would even consider a db over a file is a very,
> very obscure case then why not just set a semaphor and retry after a
> second or two if it is still open?

True. However, that only solves the concurrency issue when only one
instance is running. With several instances running (though unlikely)
then things become more complicated, especially since there is no memory
shared between instances.

[Brian Kardell]

Brian Kardell :: @bkardell :: hitchjs.com

> --
> Job Board: http://jobs.nodejs.org/
> Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
> You received this message because you are subscribed to the Google
> Groups "nodejs" group.
> To post to this group, send email to nod...@googlegroups.com
> To unsubscribe from this group, send email to
> nodejs+un...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/nodejs?hl=en?hl=en

Note that I said "if the only reason" and it was specifically WRT your initial question and (what seemed to be) your desire to keep it really simple.  Of course, there are any number of solutions and tradeoffs (short term and long) that you can make depending on what your comfort and goals are.

[Daniel Rinehart]

I'd recommend using Redis. Its sorted set data type makes a high score
list a breeze to maintain. As mentioned using a combination of RDB and
AOF you get great durability with Redis.

-- Daniel R. <dan...@neophi.com> [http://danielr.neophi.com/]

[Dan Milon]

Then you should read again about Mongo.

danmilon.

[Felix E. Klee]

On Mon, Oct 1, 2012 at 5:39 PM, Daniel Rinehart <dan...@neophi.com>
wrote:

> I'd recommend using Redis.

Redis, Redis, Redis. Nobody recommending anything else. So, I'm ready to
give it a try.

As I want to use it together with Socket.IO, could I just use the Redis
store built into Socket.IO, or is that strictly temporary?

Remember that, naturally, hiscores should persist a server restart.
Also, when a client connects, it should get all hiscores. So, that's
fundamentally different to, say, an IRC-like chat system.

> Its sorted set data type makes a high score list a breeze to maintain.

Interesting. Note, however, that hiscore table entries are not sorted by
score only: If there are two entries with the same score, then the entry
that was added last should appear first. So sorting is by score plus
time.

[Felix E. Klee]

I am now using Redis 2.6 with LUA scripting, so far with great joy!

The hiscores I store in sorted sets, with floating point scores:

* Integer part: game score (between 0 and 99)

* Fractional part: timestamp (46 bits)