Implementation for a weekly leaderboard. Help please!

[Prasad Ramdas]

A Newbie here, so please bear with my query.

Wanted to check out possible implementations for a weekly leaderboard.

1. At the beginning of the week, update the scores to 0, set the dirty score to true and wait for the ranker to update the scores. A mapreduce job this should be i guess for the amount of writes.

2. Create a new ranker every week, and manually delete the older ranker from the datastore? How should i do this?

or any other method?

Thanks in advance.

Prasad.

[Bartholomew Furrow]

Hi Prasad,

It's a little tricky to answer for sure without knowing a little more about how else you're storing your data. A few thoughts:

1. If you just want to display the top N, you don't need the ranklist: just a query.

2. If you don't mind "leaking" the old ranklist, you can just leave it in your datastore. Storage costs are, if I'm reading correctly, 0.18usd per Gigabyte per month, and ranker nodes are probably about 1kb each. A ranker with a million nodes would cost two dollars a year. Even the time you spend reading this email is probably worth more than the amount you will ever spend on storage for leaked ranklists, so I'd strongly advocate for creating a new ranker every week and just leaving the old one untouched. Don't be penny-wise and pound-foolish, spending at least an hour of your time to save what will probably never add up to more than a dollar (and there's nothing saying you can't go back and delete these things once your app hits it big).

3. To manually reduce the older ranker, which -- again -- I strongly recommend against, you can do an ancestor query on the root key, and just delete everything that's retrieved. Unless your ranklist gets really big, that will probably run in under 30 seconds.

Bartholomew

--
You received this message because you are subscribed to the Google Groups "Google App Engine Ranklist" group.
To unsubscribe from this group and stop receiving emails from it, send an email to google-app-engine-r...@googlegroups.com.
To post to this group, send email to google-app-en...@googlegroups.com.
Visit this group at https://groups.google.com/group/google-app-engine-ranklist.
For more options, visit https://groups.google.com/d/optout.

[Prasad Ramdas]

Hi Bartholomew,

Thanks for taking the time out to reply. Just some details about how i am storing my data, your insight will probably help optimise it.

Datastore - WeeklyPlayerStats (Ill probably change this to append the week no and the year to make it WeeklyPlayerStats_09_2016).

id,

player name,

score

I have a datastore created for the weekly leaderboard which i had planned to reset at the beginning of the week. Now ill probably let it be and handle it when i have to. Does app engine have any limits on the number of datastores that i could create? I need the rank of the player and his details and the twenty five players above him and twenty five players below his score to display on the leaderboard on every play. So on every request i get, i run 3 queries on the datastore for the leaderboard entries one each for the player details, for the 25 above and the last one for the 25 below him. One Rank query for the player.

Thanks,

Prasad.

On Saturday, 12 March 2016 00:51:50 UTC+5:30, Bartholomew wrote:

Hi Prasad,

It's a little tricky to answer for sure without knowing a little more about how else you're storing your data. A few thoughts:

1. If you just want to display the top N, you don't need the ranklist: just a query.

2. If you don't mind "leaking" the old ranklist, you can just leave it in your datastore. Storage costs are, if I'm reading correctly, 0.18usd per Gigabyte per month, and ranker nodes are probably about 1kb each. A ranker with a million nodes would cost two dollars a year. Even the time you spend reading this email is probably worth more than the amount you will ever spend on storage for leaked ranklists, so I'd strongly advocate for creating a new ranker every week and just leaving the old one untouched. Don't be penny-wise and pound-foolish, spending at least an hour of your time to save what will probably never add up to more than a dollar (and there's nothing saying you can't go back and delete these things once your app hits it big).

3. To manually reduce the older ranker, which -- again -- I strongly recommend against, you can do an ancestor query on the root key, and just delete everything that's retrieved. Unless your ranklist gets really big, that will probably run in under 30 seconds.

Bartholomew

On Fri, Mar 11, 2016 at 9:59 AM Prasad Ramdas <prasad...@gmail.com> wrote:

A Newbie here, so please bear with my query.

Wanted to check out possible implementations for a weekly leaderboard.

1. At the beginning of the week, update the scores to 0, set the dirty score to true and wait for the ranker to update the scores. A mapreduce job this should be i guess for the amount of writes.

2. Create a new ranker every week, and manually delete the older ranker from the datastore? How should i do this?

or any other method?

Thanks in advance.

Prasad.

--
You received this message because you are subscribed to the Google Groups "Google App Engine Ranklist" group.

To unsubscribe from this group and stop receiving emails from it, send an email to google-app-engine-ranklist+unsub...@googlegroups.com.

[Bartholomew Furrow]

Hi Prasad,

I think you've done a great job thinking this through, and I do have a suggestion. First though, to clear up a difference in vocabulary, "datastore" is the term I'm using for the service that stores data. What you're referring to as a datastore I would call an "Entity kind", or (I think somewhat inaccurately) a "Table".

In any case, here's what I would suggest:

Entity Kind: WeeklyPlayerStats

Entries:

- Player ID (presumably Google App Engine ID)

- Player Nickname

- Week

- Best Score this week

So an entry will look like:

player_id='123456'

player_nickname='Ruberik'

week=53

best_score=9999999

Then there will be a different WeeklyPlayerStats Entity representing my score in week 54. All you need to do is manually specify a "composite index" in index.yaml for [week, best_score].

For bonus speed -- I recommend this, and it makes the composite index unnecessary -- specify a "parent" for each WeeklyPlayerStats, so your key looks like:

parent_key = ndb.Key(pairs=[('AllWeeklyPlayerStats', 'all'), ('WeekGroup', 'week=%d' % this_week)])

user_key = ndb.Key(pairs=parent_key.pairs() + [('WeeklyPlayerStats', 'u=%s' % user_id)])

Your procedure for fetching data sounds about right, though ties make it trickier. I'd consider:

1. Retrieve the user's row, which you can get without a query -- just construct the key as above and do user_key.get(), which is faster than a query.

2. Ask the ranker for the rank for the user's score. It will also tell you how many are tied at that rank.

3. Ask the ranker for the scores for ranks user_rank - 25 and user_rank + 25.

4. Do some stuff to figure out how to display things in case of ties. :-) The ranker should return enough information for you to do that.

Good luck!

Bartholomew