Optimazation for indeces

[de Herdt Arne]

Hey,

I already pushed once a commit back to the project to add indexes for performance, but today we ran into another bottleneck on Rubycas.

Seems you need to add indexes on the date columns as well because the CAS runs into bottleneck issue shere with table look ups.

Just a tid bit for people running into performance issues. Will provide more info once I have it available.

--

Arne

[Robert Mitwicki]

Thanks for reporting that.

Right now I work on some improvements in rubycas-server to provide
DataMapper adapter.
This will allow to use much more efficiency solution like some nosql databases.

Could you also provide some numbers about your setup and some
statistics (server resources, amount of users, connection per second
etc). It will help to find out the weakest points and provide some
input for stress tests.

Thanks a lot
Best regards

> --
> You received this message because you are subscribed to the Google Groups
> "RubyCAS" group.
> To post to this group, send email to rubycas...@googlegroups.com.
> To unsubscribe from this group, send email to
> rubycas-serve...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/rubycas-server?hl=en.



--
Robert Mitwicki
Senior Software Engineer
FXI Technologies

[de Herdt Arne]

Sure,

We run a modified RubyCAS under passenger on a FreeBSD CARP set up. 2
boxes, each with around 40'ish Passenger spawns running the RubyCAS system.
If we run the RubyCAS for about 1 day on live, we can look at easily up to
3-6 million different tickets being generated. We tuned the deletion of
tickets so the database tables are kept reasonable.

The issue we discovered was the look up of tickets based on the ticket
parameter. On our implementation this now takes an average of 0.8ms in
general. We were looking at up to1-2 seconds without the proper indeces.
Adding the date and (ticket, consumed) index sped things up even further
on larger sets.
When we launched one of our new games, we were looking at 25000
connections every 5 minutes to give a rough estimate.

Our database back end is currently a mySQL cluster with master-slave
replication.
As we speak, we are rewriting the code base to swap the storage of tickets
etc to a Redis cluster instead of an ActiveRecord mapped database.


--
Arne

[Matt Zukowski]

Redis makes a lot of sense for ticket storage. As would memcached. That sounds like something that should be at least an easy config option in RubyCAS.

On Thu, Nov 22, 2012 at 3:04 AM, de Herdt Arne <Arne.d...@prosiebensat1games.com> wrote:

rough estimate.

[de Herdt Arne]

Unfortunatly it's not THAT easy…

We currently created a new base class to map the calls to Redis and retrieve the data to construct the classes. Works fine, and we use the Redis TTL to automatically delete tickets after a grace period:

• 5 minutes for LT
• 2 days for non-consumed ST
• 2 weeks for consumed ST
• 2 days for PGT and TGT

This is working for us, but the problem is associations. Redis has no native support for this and we plan on building our own framework to keep the associations between tickets alive.

We're still "CAS" at the moment protocol wise, but the implementation doesn't feel rubyCAS anymore for us. We plan to refactor this codebase into something that's more RubyCAS like and then we will push this to the github branch when we can.

Arne De Herdt               

Software Engineer

 

 

ProSiebenSat.1 Games GmbH

A company of ProSiebenSat.1 Media AG

 

Medienallee 19

D-85774 Unterföhring

Tel. +49 (89) 9507-8808

Fax +49 (89)9507-98934

Arne.D...@ProSiebenSat1Games.com

Executive Board: Markus Büchtmann, 
Andreas Heyden, Achim Kaspers

Company Residence: Unterföhring

HRB 119657 AG Munich

VAT number DE 182196509
TAX no. 9143/122/60981

[Matt Zukowski]

Ha, it never is :) I'm just thinking that, in general, totally moving RubyCAS' internal storage off SQL/ActiveRecord onto something like redis or memcached might make a lot of sense. The data RubyCAS stores is mostly transient and doesn't really need fancy querying. Although if we did go that route, we'd probably need a default or fallback option that doesn't require an external server; so probably plain files or in-memory, neither of which are ideal.

Then again, aren't there ActiveRecord adapters for Redis that would let you just switch the storage backend without having to change the API? (I've never tried, so I dunno)

[Robert Mitwicki]

This is why we should switch to datamapper or at least add it as default.

I work right now to achieve that. 

With datamapper (as real ORM) we have much more options then with active record,

especial noSql solutions.

I will keep inform you about progress.

Best regards

[Matt Zukowski]

Sounds good Robert.