In that version, hiscores are simply stored in the browser. For the next
version, my idea is to store hiscores on a server running Node.js /
express, hosted on Nodejitsu. How it could work:
Server starts: Hiscore lists for all puzzles are loaded from disk into
server memory.
User opens game in browser:
1. Browser loads game. Via Socket.IO the hiscore lists for all puzzles
are transferred to the browser.
2. User finishes one puzzle, gets into hiscore list, and enters name.
3. Via Socket.IO, data is sent to the server: puzzle, moves, name
4. The server verifies that the moves are correct, using a background
process.
5. Name and new hiscore are entered into puzzle's hiscore list.
6. The puzzle's hiscore list is written to disk.
7. Using Socket.IO, the updated hiscore list is broadcasted.
Now I wonder what is the simplest way to perform step 6. I see two
options:
* (A) Store hiscore list in a file `hiscores.json`, associated with
the puzzle. Problems:
- There *may* be concurrency issues. If two users finish the same
puzzle near simultaneously, then `fs.writeFile` will be called
in quick succession. I am not sure whether this could lead to
file corruption.
- For creating a backup, it doesn't seem possible to *download*
files from Nodejitsu, unless I write my own software for that.
- Although unlikely, suppose I scale up, and two Node.js are
serving the game. In this case, there are further concurrency
issues due to the hiscore lists held in memory not being shared
among the processes.
* (B) Store hiscore list in database, e.g. Mongo. Problems: I don't
see any, except that the solution feels fat, a database needs to be
set up, and I prefer individual files for simplicity.
After writing this email, I tend to option (B), but still:
Which option would you chose? Something else? Suggestions?
b) redis: http://redis.io/topics/introduction its less puzzling than files, has replication and pubsub, and atomic operations Am Montag, 1. Oktober 2012 12:10:50 UTC+2 schrieb Felix E. Klee:
> In that version, hiscores are simply stored in the browser. For the next > version, my idea is to store hiscores on a server running Node.js / > express, hosted on Nodejitsu. How it could work:
> Server starts: Hiscore lists for all puzzles are loaded from disk into > server memory.
> User opens game in browser:
> 1. Browser loads game. Via Socket.IO the hiscore lists for all puzzles > are transferred to the browser.
> 2. User finishes one puzzle, gets into hiscore list, and enters name.
> 3. Via Socket.IO, data is sent to the server: puzzle, moves, name
> 4. The server verifies that the moves are correct, using a background > process.
> 5. Name and new hiscore are entered into puzzle's hiscore list.
> 6. The puzzle's hiscore list is written to disk.
> 7. Using Socket.IO, the updated hiscore list is broadcasted.
> Now I wonder what is the simplest way to perform step 6. I see two > options:
> * (A) Store hiscore list in a file `hiscores.json`, associated with > the puzzle. Problems:
> - There *may* be concurrency issues. If two users finish the same > puzzle near simultaneously, then `fs.writeFile` will be called > in quick succession. I am not sure whether this could lead to > file corruption.
> - For creating a backup, it doesn't seem possible to *download* > files from Nodejitsu, unless I write my own software for that.
> - Although unlikely, suppose I scale up, and two Node.js are > serving the game. In this case, there are further concurrency > issues due to the hiscore lists held in memory not being shared > among the processes.
> * (B) Store hiscore list in database, e.g. Mongo. Problems: I don't > see any, except that the solution feels fat, a database needs to be > set up, and I prefer individual files for simplicity.
> After writing this email, I tend to option (B), but still:
> Which option would you chose? Something else? Suggestions?
Thanks for the suggestion. However, if using a database, then I prefer
Mongo. Performance is not an issue. Persistence, on the other hand, is a
big issue: If someone spent hours to crack the hiscore, then he will be
disappointed if his entry goes away due to a server crash.
On Monday, October 1, 2012 at 12:48 PM, Felix E. Klee wrote:
> On Mon, Oct 1, 2012 at 12:55 PM, greelgorke <greelgo...@gmail.com (mailto:greelgo...@gmail.com)>
> wrote:
> > b) redis: http://redis.io/topics/introduction
> Thanks for the suggestion. However, if using a database, then I prefer
> Mongo. Performance is not an issue. Persistence, on the other hand, is a
> big issue: If someone spent hours to crack the hiscore, then he will be
> disappointed if his entry goes away due to a server crash.
On Monday, October 1, 2012 at 1:56 PM, Felix E. Klee wrote:
> On Mon, Oct 1, 2012 at 1:50 PM, Pedro Teixeira
> <pedro.teixe...@gmail.com (mailto:pedro.teixe...@gmail.com)> wrote:
> > http://redis.io/topics/persistence
> Too complicated and/or not reliable enough.
> At *any* point in time, I want to have the database mirrored to disk, in
> its entirety. Mongo seems perfect. I wonder why people recommend Redis.
Then why ask the question if you already made your mind? You clearly don't want to use anything else then mongodb because
you seem to know better and the only thing that redis can do is crash anyways, mongo on the other hand is web scale and build
to solve all the issues in the universe that people throw at it.
As for A) there a couple of "decent" databases written in Node, but using a database is better, because at some point you are going
to scale out of process, or use multiple apps that are not hosted on the same process.. Simplicity is nice and all but it's not always
the best answer on all the questions.
On Monday 1 October 2012 at 14:56, Felix E. Klee wrote:
On Mon, Oct 1, 2012 at 3:09 PM, Arnout Kazemier <i...@3rd-eden.com>
wrote:
> Then why ask the question if you already made your mind?
I'm far from having made up my mind. I just wonder why people recommend
Redis when performance is not an issue here. After, all solving a puzzle
may take some minutes, and only once in a while someone will get into
the hiscores table.
<pedro.teixe...@gmail.com> wrote:
> Did you read the append-only:true AOF part of the link I sent you?
Yes. I also now read that from time to time Redis rewrites the log so
that - I assume - entries that have been deleted will not appear
anymore. Sounds good!
However, it is recommended to not use AOF alone, possibly making things
more complicated than when using Mongo:
"There are many users using AOF alone, but we discourage it since to
have an RDB snapshot from time to time is a great idea for doing
database backups, for faster restarts, and in the event of bugs in the
AOF engine."
Also, I don't know what happens when the Redis server is restarted. Will
it automatically read the log to recreate the last database status?
I really don't understand what is the particular advantage of using
Redis vs. Mongo for the use case. Redis looks more complicated to set up
and possibly to maintain.
Brian Kardell :: @bkardell :: hitchjs.com
On Oct 1, 2012 9:29 AM, "Felix E. Klee" <felix.k...@inka.de> wrote:
> On Mon, Oct 1, 2012 at 3:09 PM, Arnout Kazemier <i...@3rd-eden.com>
> wrote:
> > Then why ask the question if you already made your mind?
> I'm far from having made up my mind. I just wonder why people recommend
> Redis when performance is not an issue here. After, all solving a puzzle
> may take some minutes, and only once in a while someone will get into
> the hiscores table.
> You received this message because you are subscribed to the Google
> Groups "nodejs" group.
> To post to this group, send email to nodejs@googlegroups.com
> To unsubscribe from this group, send email to
> nodejs+unsubscribe@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/nodejs?hl=en?hl=en
If the only reason you would even consider a db over a file is a very, very
obscure case then why not just set a semaphor and retry after a second or
two if it is still open?
On Mon, Oct 1, 2012 at 3:41 PM, Brian Kardell <bkard...@gmail.com>
wrote:
> If the only reason you would even consider a db over a file is a very,
> very obscure case then why not just set a semaphor and retry after a
> second or two if it is still open?
True. However, that only solves the concurrency issue when only one
instance is running. With several instances running (though unlikely)
then things become more complicated, especially since there is no memory
shared between instances.
Brian Kardell :: @bkardell :: hitchjs.com
On Oct 1, 2012 10:04 AM, "Felix E. Klee" <felix.k...@inka.de> wrote:
> On Mon, Oct 1, 2012 at 3:41 PM, Brian Kardell <bkard...@gmail.com>
> wrote:
> > If the only reason you would even consider a db over a file is a very,
> > very obscure case then why not just set a semaphor and retry after a
> > second or two if it is still open?
> True. However, that only solves the concurrency issue when only one
> instance is running. With several instances running (though unlikely)
> then things become more complicated, especially since there is no memory
> shared between instances.
> You received this message because you are subscribed to the Google
> Groups "nodejs" group.
> To post to this group, send email to nodejs@googlegroups.com
> To unsubscribe from this group, send email to
> nodejs+unsubscribe@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/nodejs?hl=en?hl=en
Note that I said "if the only reason" and it was specifically WRT your
initial question and (what seemed to be) your desire to keep it really
simple. Of course, there are any number of solutions and tradeoffs (short
term and long) that you can make depending on what your comfort and goals
are.
I'd recommend using Redis. Its sorted set data type makes a high score
list a breeze to maintain. As mentioned using a combination of RDB and
AOF you get great durability with Redis.
> In that version, hiscores are simply stored in the browser. For the next
> version, my idea is to store hiscores on a server running Node.js /
> express, hosted on Nodejitsu. How it could work:
> Server starts: Hiscore lists for all puzzles are loaded from disk into
> server memory.
> User opens game in browser:
> 1. Browser loads game. Via Socket.IO the hiscore lists for all puzzles
> are transferred to the browser.
> 2. User finishes one puzzle, gets into hiscore list, and enters name.
> 3. Via Socket.IO, data is sent to the server: puzzle, moves, name
> 4. The server verifies that the moves are correct, using a background
> process.
> 5. Name and new hiscore are entered into puzzle's hiscore list.
> 6. The puzzle's hiscore list is written to disk.
> 7. Using Socket.IO, the updated hiscore list is broadcasted.
> Now I wonder what is the simplest way to perform step 6. I see two
> options:
> * (A) Store hiscore list in a file `hiscores.json`, associated with
> the puzzle. Problems:
> - There *may* be concurrency issues. If two users finish the same
> puzzle near simultaneously, then `fs.writeFile` will be called
> in quick succession. I am not sure whether this could lead to
> file corruption.
> - For creating a backup, it doesn't seem possible to *download*
> files from Nodejitsu, unless I write my own software for that.
> - Although unlikely, suppose I scale up, and two Node.js are
> serving the game. In this case, there are further concurrency
> issues due to the hiscore lists held in memory not being shared
> among the processes.
> * (B) Store hiscore list in database, e.g. Mongo. Problems: I don't
> see any, except that the solution feels fat, a database needs to be
> set up, and I prefer individual files for simplicity.
> After writing this email, I tend to option (B), but still:
> Which option would you chose? Something else? Suggestions?
On Mon, Oct 1, 2012 at 5:39 PM, Daniel Rinehart <dani...@neophi.com>
wrote:
> I'd recommend using Redis.
Redis, Redis, Redis. Nobody recommending anything else. So, I'm ready to
give it a try.
As I want to use it together with Socket.IO, could I just use the Redis
store built into Socket.IO, or is that strictly temporary?
Remember that, naturally, hiscores should persist a server restart.
Also, when a client connects, it should get all hiscores. So, that's
fundamentally different to, say, an IRC-like chat system.
> Its sorted set data type makes a high score list a breeze to maintain.
Interesting. Note, however, that hiscore table entries are not sorted by
score only: If there are two entries with the same score, then the entry
that was added last should appear first. So sorting is by score plus
time.
