MassTransit array bounds exception during subscribe

[Geoff]

We are (very rarely) seeing MT throw the array bounds exception below. Once the exception does occur though, all subsequent subscription attempts suffer the same fate. The same error is repeated throughout our logs for around 14 hours. I had to stop/start our app to resolve it.  99.9% of the time, the issue does not occur.

Any suggestions, even a way to work around, would be appreciated as we are 2 weeks out from go-live date (eeip!)

App is .NET 4, sitting on RabbitMQ 2.8.2 cluster (w/HA queue). We're using the NuGet packages for masstransit: MassTransit 2.1.1, MassTransit.Log4Net 2.1.1, MassTransit.RabbitMQ 2.1.1.

System.IndexOutOfRangeException: Index was outside the bounds of the array.

   at System.Collections.Generic.HashSet`1.AddIfNotPresent(T value)

   at MassTransit.Subscriptions.Coordinator.BusSubscription.OnSubscribeTo(SubscribeTo added) in d:\BuildAgent-03\work\8d1373c869590c5b\src\MassTransit\Subscriptions\Coordinator\BusSubscription.cs:line 73

   at MassTransit.Subscriptions.Coordinator.BusSubscriptionCache.OnSubscribeTo(SubscribeTo message) in d:\BuildAgent-03\work\8d1373c869590c5b\src\MassTransit\Subscriptions\Coordinator\BusSubscriptionCache.cs:line 49

   at MassTransit.Subscriptions.Coordinator.BusSubscriptionEventListener.Subscribe[TMessage](String correlationId) in d:\BuildAgent-03\work\8d1373c869590c5b\src\MassTransit\Subscriptions\Coordinator\BusSubscriptionEventListener.cs:line 69

   at MassTransit.Subscriptions.Coordinator.BusSubscriptionEventListener.SubscribedTo[TMessage]() in d:\BuildAgent-03\work\8d1373c869590c5b\src\MassTransit\Subscriptions\Coordinator\BusSubscriptionEventListener.cs:line 43

   at MassTransit.Pipeline.Configuration.InboundPipelineConfigurator.<>c__DisplayClass2`1.<SubscribedTo>b__1(ISubscriptionEvent x) in d:\BuildAgent-03\work\8d1373c869590c5b\src\MassTransit\Pipeline\Configuration\InboundPipelineConfigurator.cs:line 56

   at System.Collections.Generic.List`1.ForEach(Action`1 action)

   at MassTransit.Util.RegistrationList`1.<>c__DisplayClass8.<Each>b__7(List`1 x) in d:\BuildAgent-03\work\8d1373c869590c5b\src\MassTransit\Util\RegistrationList.cs:line 51

   at Magnum.Threading.ReaderWriterLockedObject`1.<>c__DisplayClass1.<ReadLock>b__0(ReaderWriterLockContext x) in d:\Builds\MassTransit-Magnum\src\Magnum\Threading\ReaderWriterLockedObject.cs:line 56

   at Magnum.Threading.ReaderWriterLockContext.ReadLock(Action`1 action) in d:\Builds\MassTransit-Magnum\src\Magnum\Threading\ReaderWriterLockContext.cs:line 52

   at Magnum.Threading.ReaderWriterLockedObject`1.ReadLock(Action`1 action) in d:\Builds\MassTransit-Magnum\src\Magnum\Threading\ReaderWriterLockedObject.cs:line 56

   at MassTransit.Util.RegistrationList`1.Each(Action`1 action) in d:\BuildAgent-03\work\8d1373c869590c5b\src\MassTransit\Util\RegistrationList.cs:line 51

   at MassTransit.Pipeline.Configuration.InboundPipelineConfigurator.SubscribedTo[TMessage]() in d:\BuildAgent-03\work\8d1373c869590c5b\src\MassTransit\Pipeline\Configuration\InboundPipelineConfigurator.cs:line 56

   at MassTransit.SubscriptionConnectors.HandlerSubscriptionConnector`1.Connect(IInboundPipelineConfigurator configurator, HandlerSelector`1 handler) in d:\BuildAgent-03\work\8d1373c869590c5b\src\MassTransit\Configuration\SubscriptionConnectors\HandlerSubscriptionConnector.cs:line 41

   at MassTransit.HandlerSubscriptionExtensions.<>c__DisplayClass1`1.<SubscribeHandlerSelector>b__0(IInboundPipelineConfigurator x) in d:\BuildAgent-03\work\8d1373c869590c5b\src\MassTransit\Configuration\HandlerSubscriptionExtensions.cs:line 104

   at MassTransit.Pipeline.InboundMessagePipeline.Configure[TResult](Func`2 configureCallback) in d:\BuildAgent-03\work\8d1373c869590c5b\src\MassTransit\Pipeline\InboundMessagePipeline.cs:line 54

   at MassTransit.ServiceBus.Configure(Func`2 configure) in d:\BuildAgent-03\work\8d1373c869590c5b\src\MassTransit\ServiceBus.cs:line 204

   at MassTransit.HandlerSubscriptionExtensions.SubscribeHandlerSelector[T](IServiceBus bus, HandlerSelector`1 handler) in d:\BuildAgent-03\work\8d1373c869590c5b\src\MassTransit\Configuration\HandlerSubscriptionExtensions.cs:line 104

   at MassTransit.HandlerSubscriptionExtensions.SubscribeHandler[T](IServiceBus bus, Action`1 handler, Predicate`1 condition) in d:\BuildAgent-03\work\8d1373c869590c5b\src\MassTransit\Configuration\HandlerSubscriptionExtensions.cs:line 90

   at Sanitized.BaseFeeder.Subscribe() in x:\sanitized\BaseFeeder.cs:line 136

   at Sanitized.FeedManager.<>c__DisplayClass1.<SubscribeUnsubscribed>b__0(KeyValuePair`2 x) in x:\sanitized\FeedManager.cs:line 75

Thanks,

Geoff

[Chris Patterson]

This -might- be a threading issue, are you subscribing multiple handlers at the same time to the same message type? Perhaps via Request/Response? If so, I may have overlooked this race condition...

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.
To view this discussion on the web visit https://groups.google.com/d/msg/masstransit-discuss/-/glti9jj9HHsJ.
To post to this group, send email to masstrans...@googlegroups.com.
To unsubscribe from this group, send email to masstransit-dis...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/masstransit-discuss?hl=en.

[Geoff Steffens]

Hi Chris,

Yes, we are infact doing exactly that. I'll build a unit test on our side to try and verify, but it certainly sounds plausible.

Cheers,

Geoff

---

Sent from a mobile device