MassTransit (over RabbitMQ) IServiceBus.Publish(...) hangs unexpectedly

[usar...@gmail.com]

Hello,

I'm developing web based solution using MassTransit + RabbitMQ as transport. Some time ago I noticed that when I run my integration tests some of them start unexpectedly and very inconsistently hanging for a very loooong time.

After short investigation I found out that my code hangs somewhere inside ServiceBus.Publish<T>(T message) method.

Unfortunately MassTransit documentation and blog posts that I found do not answer what is the reason and how to solve this problem. Hopefully here somebody could help to resolve it.

Here is how I initialize my service bus:

ServiceBusFactory.New(sbc =>

      {

        sbc.ReceiveFrom(busAddress);

        sbc.UseRabbitMq(r =>

        {

          r.ConfigureHost(new Uri(busAddress),

            cfg =>

            {

              if (!string.IsNullOrEmpty(busUser) && !string.IsNullOrEmpty(busPass))

              {

                cfg.SetPassword(busPass);

                cfg.SetUsername(busUser);

              }

            });

        });

        sbc.UseControlBus();

        sbc.SetCreateMissingQueues(true);

        sbc.SetCreateTransactionalQueues(true);

        sbc.SetNetwork("workgroup");

        sbc.UseBsonSerializer();

        sbc.SetDefaultRetryLimit(2);

        sbc.SetDefaultTransactionTimeout(new TimeSpan(0, 0, timeoutSec));

        

        sbc.Validate();

      });

timeoutSec value is 10

Service bus is initialized once and registered in Autofac container.

Publishing happens by calling IServiceBus.Publish(...) method.

One of the solutions is to  wrap Publish(...) method into Task and use Task.Wait(...) method to enforce timeout but this does not look like a good solution.

I would be really appreciated for any kind of help!

Thank you!

[Travis Smith]

So you're using RabbitMQ? 

You shouldn't need any of

>        sbc.UseControlBus();

>        sbc.SetCreateMissingQueues(true);

>        sbc.SetCreateTransactionalQueues(true);

>        sbc.SetNetwork("workgroup");

Unless you're specifically doing something with one of them. I doubt that's the issue though, unneeded configuration is just ignored. 

How long is a long time for the timeout? Normally when RabbitMQ won't respond quickly, it's an IO problem. If you open up the RabbitMQ dashboard, there's likely a red light on it somewhere telling you what's going on. Sadly, RabbitMQ just blocks the connection rather than failing if there's a problem like this. 

-Travis

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-dis...@googlegroups.com.
To post to this group, send email to masstrans...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/masstransit-discuss/53a8c813-72a7-4158-bd08-7267690f1c63%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[usar...@gmail.com]

Thank you for the hint! I will remove those redundant config options.

How long is a long time for the timeout?

It will take all time that I set up for HTTP timeout. It can take 2-3 minutes easily.

Looks like Masstransit is waiting on response from RabbitMQ but never receives it.

If you open up the RabbitMQ dashboard, there's likely a red light on it somewhere telling you what's going on

Nope, no red lights. I know the trick with insufficient RAM and disk space, when RabbitMQ just stops responding, but that was not the case. Masstransit will start working properly again as soon as I restart IIS process.

I thought that it could be because there are lots of unprocessed messages in the queue but my suggestion did not confirm. It can happen with zero messages in the queue as well.

-Travis

To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-discuss+unsub...@googlegroups.com.

[Tim Bailey]

I had a similar problem where it would work fine in a console app, but not in a web app.

Seemed to be relegated to SyncronisationContext, in the end, like you, I used a threadpool/task but that is a bit naff.

https://groups.google.com/forum/#!msg/masstransit-discuss/oC1FOe6KsAU/VdoTrkcdHS0J

Tim

[usar...@gmail.com]

Travis,

one quick off-topic question regarding those configuration options. so, you say they are not needed but accordingly to the documentation I need at least those two:

sbc.UseControlBus();
sbc.SetCreateMissingQueues(true);

Control bus is recommended to use accordingly to this article: http://masstransit.readthedocs.org/en/latest/overview/controlbus.html

Create missing queues also does not seem to be a bad idea.

Transactional queues and network key are not supported by RabbitMQ.

So, are you sure I need to remove all four configuration options? Do you use it in this way in your projects?

On Monday, April 27, 2015 at 2:18:44 PM UTC+2, Travis Smith wrote:

-Travis

To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-discuss+unsub...@googlegroups.com.

[usar...@gmail.com]

I guess, for now that's the only solution. 

But then I run into another problem when message was actually published (if I understand correctly this bug) and error is also logged. Basically, I need to check every error and not re-run manually some actions that were already executed. Did you have this kind of issues?

[Travis Smith]

Travis,

one quick off-topic question regarding those configuration options. so, you say they are not needed but accordingly to the documentation I need at least those two:

sbc.UseControlBus();
sbc.SetCreateMissingQueues(true);

UseControlBus is really only the RuntimeServices, which is used for MSMQ. SetCreateMissingQueues() only applies to MSMQ as well, I think the RabbitMQ behaviour binds queues no matter what. Chris, can you verify?