Rhino ServiceBus & DHT Saga storage

[Ayende Rahien]

Right now I am working on the ESB parts of the port, and I am thinking hard again about what should and shouldn't be in there.

On the one hand, one of the major reasons that I created RSB is that I wanted to make something that is developer friendly and easy to get started.

On the other hand, there are some things where we do want to provide extensibility and customization for the users.

For the most part, I think we managed to do that by using the container in some clever ways, but with the DHT saga storage I think I really messed it up.

It is complex, both to set it up and to make use of it and to understand how it works.

I have tentatively removed it from the project.

I would like to provide a saga storage that is easy to use and fit the bill for most of the operations that you need, without bringing undue burden for the administrator or developer.

Last week I had several discussions with Udi about that, and he pointed out that the most commonly used and easiest to reason about is a locked saga state. That is, during the execution of a transaction, the state of the saga is locked. A common example would be using a DB to handle that while using serializable transactions.

I still want to enable the "let us just use this" mode, and I still want to avoid dependencies on infrastructure that isn't xcopy deployable.

We can support this easily if we will utilize only the PHT. But that will work for local mode only. We can make use of the DHT, but then we need to provide a solution for farm wide locking. A lot of the design behind the DHT is based on always on system, because I have a requirement to keep the system going while nodes are coming and going. Locking is... interesting in this scenario. I would love to hear options about that.

Or, we could just provide a simple DB saga state and let the DB handle that and clustering to handle fail over. 

[Tyler Burd]

I'm currently using a custom saga persister that I'd be happy to share, but it uses NH and I doubt you want to make that a dependency of RSB.  I found it was simple to write (5 minutes), simple to understand, and it just works.  I didn't need *extreme* throughput, though, and I expect that's the case for the vast majority of projects, so my vote is +1 for a simple db persister.  It's still going to be a hell of a lot more scalable than a traditional thread hungry ASP NET app.

-tyler

[Jason Meckley]

I dug into the code and this finally clicked and I understand the
problem of balancing configuration & extensibility.
I spend most of my time in the previous revision since it's all in
tack.
in looking through it I can understand why you want to re-design how
DHT works. Why remove the persister strategies and local DHT Client
though? that all seems to work without issue.

I think a db persister would be straight forward. were talking about a
single table with 6 columns. wrap ADO.Net with a simple facade and
call it a day. you could add a deploy action to build the schema.
have it pull from the config file and add the table under another
schema. similar to Rhino.Security.

tyler i would be interested in your database implementation. for my
immediate need I ported Local DHT Client and OptimisticStatePersister
to my project to work against the latest RSB build.

On Sep 3, 9:36 am, Tyler Burd <tylerb...@gmail.com> wrote:
> I'm currently using a custom saga persister that I'd be happy to share, but
> it uses NH and I doubt you want to make that a dependency of RSB.  I found
> it was simple to write (5 minutes), simple to understand, and it just
> works.  I didn't need *extreme* throughput, though, and I expect that's the
> case for the vast majority of projects, so my vote is +1 for a simple db
> persister.  It's still going to be a hell of a lot more scalable than a
> traditional thread hungry ASP NET app.
>
> -tyler
>
> On Wed, Sep 2, 2009 at 4:43 PM, Ayende Rahien <aye...@ayende.com> wrote:
> > Right now I am working on the ESB parts of the port, and I am thinking hard

> > again about what should and shouldn't be in there.On the one hand, one of

[Ayende Rahien]

The problem is that then you introduce a dependency on NHibernate.

Not a biggie, but I have a fondness for zero dependencies here, just plug & go.

[Ayende Rahien]

Jason,

Because they brought a lot of complexity to the table.

I actually think that the local DHT + optimistic is something that I would like to end up with.

We can specify a local, self deployed, version for development, and scale up for a remote one for farm scenario and a full DHT cluster for reliability.

[Tyler Burd]

I completely agree that an NH dependency would be bad.  I think a vanilla ADO persister would be a good thing to include.

NSB used to have the DbBlobSagaPersister.  It would be nice if there was something similar in RSB, but using an NTEXT field and the message serializer instead of binary serialization.

[Ayende Rahien]

What are the advantages of that vs. using PHT/DHT?

[Tyler Burd]

It's really easy to understand, and there is generally good tooling around dbs so people can use their existing tools to inspect saga state.

Admittedly, I have not explored the PHT/DHT stuff at all, since I simply converted an existing saga persister from an NSB project to RSB.  I will spend some time with them this evening.  My opinion may very well change!

[Udi Dahan]

The reason we moved to the NHibernateSagaPersister is that we could
use it to automatically generate a schema for the user's saga objects
including the appropriate indexable columns. This means users don't
need to understand more than to point it at a given database and it'll
install and go.

Hope that makes sense.

-- Udi Dahan

On Sep 5, 10:05 pm, Tyler Burd <tylerb...@gmail.com> wrote:
> I completely agree that an NH dependency would be bad.  I think a vanilla
> ADO persister would be a good thing to include.
>
> NSB used to have the DbBlobSagaPersister.  It would be nice if there was
> something similar in RSB, but using an NTEXT field and the message
> serializer instead of binary serialization.
>
> On Sat, Sep 5, 2009 at 4:00 AM, Ayende Rahien <aye...@ayende.com> wrote:

> > Jason,Because they brought a lot of complexity to the table.

> > I actually think that the local DHT + optimistic is something that I would
> > like to end up with.
> > We can specify a local, self deployed, version for development, and scale
> > up for a remote one for farm scenario and a full DHT cluster for
> > reliability.
>

[Ayende Rahien]

Oh you are talking about saga finders

[Tyler Burd]

Ayende, Assuming you're referring to me, I am talking about saga persisters, not finders.  I think a db-based saga persister with explicit row-locking would cover 90% of needs, but I fully accept that a DHT/PHT implementation would be better.  I spent the evening exploring the load balancer, so I didn't quite make it to saga persistence with the DHT.  And heck, you probably know better anyways!

Udi, the reason I stuck with a simple one-table approach is that I didn't want to create NH objects and mappings for every saga state class.  I have about a hundred saga state classes in the project I am involved in at the moment, and some of those classes have fairly deep relations.  Dumping them into a single table with Xml Serialization suited my needs perfectly and saved me a LOT of development time.

-tyler

[Ayende Rahien]

I was talking to Udi, actually.

NH Saga Persister makes sense if you want to do things like lookup by something else (saga finders).

While, key/value based solutions are ideal if you don't want that.

[Steve Wagner]

Whats about providing the code for the hashtable in a Contrib or
Extensions subdirectory?

Ayende Rahien schrieb:

> Right now I am working on the ESB parts of the port, and I am thinking hard

> again about what should and shouldn't be in there.On the one hand, one of

[Ayende Rahien]

We don't have a contrib project.
The problem that I have with this is that I do no want to have to support complex stuff.