ruote-cassandra
John Mettraux
on IRC:
> 2011-05-11 18:37:43 utc hartog: hi all
> 2011-05-11 19:24:41 utc hartog: what are the types a storage could encounter?
> 2011-05-11 20:47:17 utc hartog: i think i just created a draft version of Cassandra storage ... *yikes*
> 2011-05-11 20:49:57 utc hartog: but there is work left to be done I see... ;-)
> 2011-05-11 21:19:13 utc hartog: just loaded a process definition with Cassandra storage in place :-)
> 2011-05-11 21:19:32 utc hartog: tomorrow there is another day
> 2011-05-11 21:22:17 utc hartog: https://github.com/coffeeaddict/ruote-cassandra
Hello Hartog,
types are :
variables
msgs
expressions
errors
schedules
configurations
workitems
I still have to find the time to finish this document :
http://ruote.rubyforge.org/implementing_a_storage.html
Cassandra is a very very interesting backend, but ruote needs, for "msgs" and "schedules" to be able to reserve/delete those objects in an exclusive way. Thus ruote-cassandra is limited to 1 worker.
The ruote workers pick msgs in the storage and reserve them. Using vanilla Cassandra, you have no way to let one and only one worker pick/reserve a msg. If two or more workers process the same msg, chaos. The same applies for schedules.
Best regards,
--
John Mettraux - http://jmettraux.wordpress.com
Hartog C. de Mik
>
> on IRC:
> > 2011-05-11 18:37:43 utc hartog: hi all
> > 2011-05-11 19:24:41 utc hartog: what are the types a storage could encounter?
> > 2011-05-11 20:47:17 utc hartog: i think i just created a draft version of Cassandra storage ... *yikes*
> > 2011-05-11 20:49:57 utc hartog: but there is work left to be done I see... ;-)
> > 2011-05-11 21:19:13 utc hartog: just loaded a process definition with Cassandra storage in place :-)
> > 2011-05-11 21:19:32 utc hartog: tomorrow there is another day
> > 2011-05-11 21:22:17 utc hartog: https://github.com/coffeeaddict/ruote-cassandra
>
> Hello Hartog,
>
> types are :
>
> variables
> msgs
> expressions
> errors
> schedules
> configurations
> workitems
Almost had them all then... Just missed workitems (see :
https://github.com/coffeeaddict/ruote-cassandra/blob/master/dev/schema.cql)
> I still have to find the time to finish this document :
>
> http://ruote.rubyforge.org/implementing_a_storage.html
Are these documents on github? I would not mind adding more info to
them as I fight my battle with this storage thingy...
> Cassandra is a very very interesting backend, but ruote needs, for
> "msgs" and "schedules" to be able to reserve/delete those objects in
> an exclusive way. Thus ruote-cassandra is limited to 1 worker.
>
> The ruote workers pick msgs in the storage and reserve them. Using
> vanilla Cassandra, you have no way to let one and only one worker
> pick/reserve a msg. If two or more workers process the same msg,
> chaos. The same applies for schedules.
I already guessed that when reading ruote-redis and
ruote-sequel. Perhaps this might prove to be useful:
http://code.google.com/p/cages/
I also added a README to the github repo, prooving there is still lots
of ground to cover (even without the ACID issues you mentioned above)
Grtz,
Hartog.
John Mettraux
On Thu, May 12, 2011 at 06:29:26AM +0000, Hartog C. de Mik wrote:
>
> > I still have to find the time to finish this document :
> >
> > http://ruote.rubyforge.org/implementing_a_storage.html
>
> Are these documents on github? I would not mind adding more info to
> them as I fight my battle with this storage thingy...
Hello Hartog,
https://github.com/jmettraux/ruote_website
Thanks in advance !
There are a few notes about testing at
https://github.com/jmettraux/ruote/tree/master/test
Basically, testing a storage goes like
ruby test/unit/storage.rb -- --cassandra
and then
ruby test/functional/storage.rb -- --cassandra
(it expects to find ruote-cassandra in adjacent to ruote/
somedir/
|-- ruote/
|-- ruote-cassandra/
> > Cassandra is a very very interesting backend, but ruote needs, for
> > "msgs" and "schedules" to be able to reserve/delete those objects in
> > an exclusive way. Thus ruote-cassandra is limited to 1 worker.
> >
> > The ruote workers pick msgs in the storage and reserve them. Using
> > vanilla Cassandra, you have no way to let one and only one worker
> > pick/reserve a msg. If two or more workers process the same msg,
> > chaos. The same applies for schedules.
>
> I already guessed that when reading ruote-redis and
> ruote-sequel. Perhaps this might prove to be useful:
> http://code.google.com/p/cages/
>
> I also added a README to the github repo, prooving there is still lots
> of ground to cover (even without the ACID issues you mentioned above)
Have fun !