Using rabbitmq within a transaction

[Andrei Alecu]

I'm evaluating mass transit for a project and we need to ensure that
any messages published within an ambient transaction do not get
published until the transaction is committed.

We use DTC already, but we do not emply a message queue yet. I believe
MSMQ might be able to roll back a message posted to a queue if the
transaction isn't committed, but does RabbitMQ do it?

If it doesn't - am I wrong in thinking that it should be easy to add
by subscribing to the ambient transaction and only publishing to the
queue when it is committed?

Thanks.

[Henrik Feldt]

Hello Andrei,

If the publish-on-commit is your only concern with regards to transactions, then yes, all you have to do is to do:

Transaction.Current.EnlistVolatile( ... )
http://msdn.microsoft.com/en-us/library/system.transactions.transaction.enlistvolatile.aspx

You simply then do the publish on commit.

That said, what transactions are mostly used for is the fact that they are 2PC, and in conjunction with queues, that you have once-and-only-once semantics on writes in conjunction with consume from the queue.

RabbitMQ does support distributed transactions, but only through the XA-based API. I have never used it, and there's no support currently in MT for it; that said; it should be the same interfaces as what MSMQ transport uses since the LTM (lightweight transaction manager in .net) only is usable through the System.Transactions API. As such, you could pretty IN THEORY much copy-n-paste the transactional behaviour from the MSMQ transport, add the API enlistment calls to the durable resource that is the RMQ Broker and off you go.

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.
To post to this group, send email to masstrans...@googlegroups.com.
To unsubscribe from this group, send email to masstransit-dis...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/masstransit-discuss?hl=en.

[Travis Smith]

So implementing transactions around RabbitMQ shouldn't be difficult.
Distributed Transactions will likely require more magic. You can do
this unquestionably with MSMQ, though configuring all the pieces for
DTC makes everyone cry. It's really a huge pain to debug.

EnlistVolatile seems like a good choice here, otherwise you could
handle something similar in your own code. Collect messages to be
published and just publish at the last step of your transaction. That
would require pretty much zero MT knowledge and could be done within
your application.

-Travis

[Andrei Alecu]

Thank you for replying.

It pretty much needs to be done via EnlistVolatile because I'm
planning on publishing to the queue from within a WCF service that
gets a distributed transaction from its caller. The initial caller
then decides whether to commit the transaction or not.

I already wrote an email sender library that I'm using for this which
works similarly, keeping all messages in a queue until the transaction
is committed. Only at that point the emails are being actually sent
out.

So, in order to get the publish to happen on commit, I basically need
to make a wrapper around the Bus that keeps all the messages in memory
until the transaction completes.

Or - I could patch MT somewhere, and perhaps allow it as a
configuration option when setting up the Bus, something like:


Bus.Initialize(sbc =>
{
sbc.UseTransactionalPublishing()
});

Does that look worthwhile? If it does, I could submit it as a pull
request later on.

-Andrei

On Apr 12, 8:27 pm, Travis Smith <tra...@legomaster.net> wrote:
> So implementing transactions around RabbitMQ shouldn't be difficult.
> Distributed Transactions will likely require more magic. You can do
> this unquestionably with MSMQ, though configuring all the pieces for
> DTC makes everyone cry. It's really a huge pain to debug.
>
> EnlistVolatile seems like a good choice here, otherwise you could
> handle something similar in your own code. Collect messages to be
> published and just publish at the last step of your transaction. That
> would require pretty much zero MT knowledge and could be done within
> your application.
>
> -Travis
>
>
>
>
>
>
>
> On Thu, Apr 12, 2012 at 12:53 PM, Henrik Feldt <logi...@gmail.com> wrote:
> > Hello Andrei,
>
> > If the publish-on-commit is your only concern with regards to transactions,
> > then yes, all you have to do is to do:
>
> > Transaction.Current.EnlistVolatile( ... )

> >http://msdn.microsoft.com/en-us/library/system.transactions.transacti...

>
> > You simply then do the publish on commit.
>
> > That said, what transactions are mostly used for is the fact that they are
> > 2PC, and in conjunction with queues, that you have once-and-only-once
> > semantics on writes in conjunction with consume from the queue.
>
> > RabbitMQ does support distributed transactions, but only through the
> > XA-based API. I have never used it, and there's no support currently in MT
> > for it; that said; it should be the same interfaces as what MSMQ transport
> > uses since the LTM (lightweight transaction manager in .net) only is usable
> > through the System.Transactions API. As such, you could pretty IN THEORY
> > much copy-n-paste the transactional behaviour from the MSMQ transport, add
> > the API enlistment calls to the durable resource that is the RMQ Broker and
> > off you go.
>

[Andrei Alecu]

From my tests it seems that masstransit is a 'real time' pub/sub
framework. If a consumer isn't available, the framework immediately
moves the message into an error queue.

What I was mainly interested in was to be able to publish a message,
and later on when a subscriber started up (even hours later), it would
be able to process all the messages that have been sent since then.

I couldn't seem to figure out how to do this, if a subscriber isn't
available immediately while publishing, I see a "Message retry limit
exceeded" error logged, and the message moves into the error queue. I
believe I might need to create a permanent subscription, but either
the documentation is outdated or I couldn't figure out how to make
one.

Also, I wasn't able get UseSubscriptionService() to work at all. When
using it, my test project just hangs on the
ServiceBusFactory.New( ... ) line indefinitely. I was able to confirm
this with the PublishSubscrible.sln sample, which hangs as well.

What am I doing wrong? UseMulticastSubscriptionClient seems to work
properly.

[Andrei Alecu]

Alright, I figured out the UseSubscriptionService() problem. Looks
like I need to run/configure MassTransit.RuntimeServices.exe.

[Travis Smith]

UseSubscriptionService is only for MSMQ. Rabbit takes care of
subscription internally with exchange bindings. The
SubscriptionService is just a small exe that runs a service to persist
subscription data for MSMQ.

I'm not sure about doing the wrapping for messages; it should work but
I don't understand the app. You could intercept the messages with your
wrapper, use the distributor and replace the IWorkerSelectionStrategy
with one that holds messages until the tx is commited or released, or
use the pipeline and intercept messages.

At this point, depending on the complexity of doing so, I would lean
towards just doing it in your app until you understand how to make it
happen in within MT after some more experience. However, you're
welcomed to do whatever you feel most comfortable with :)

-Travis