Timeout waiting for response from the subscription service after upgrade to 2.0.1.0

[Paul Cox]

I've recently upgraded from old installation of MassTransit (0.8) to
the latest NuGet package and have hit a snag where any service
publishing or subscribing to messages throws an exception at startup
because it times out while waiting for the subscription service to
respond.

I have an application service that receives messages from a couple of
web applications and a couple of console applications. Only the
application service subscribes to messages at the moment and all other
services just send with the web applications publishing messages while
the console applications grab an endpoint from the endpointcache and
send the message directly.

Previously I was using the DefaultMassTransitContainer from the
WindsorIntegration package with the following configuration on the web
app side

<configuration>
<facilities>
<facility id="masstransit">
<bus id="client" endpoint="msmq://localhost/icompile_web">
<subscriptionService endpoint="msmq://localhost/
mt_subscriptions" />
<managementService heartbeatInterval="3" />
</bus>
<transports>
<transport>MassTransit.Transports.Msmq.MsmqEndpoint,
MassTransit.Transports.Msmq</transport>
</transports>
</facility>
</facilities>
</configuration>

that has been replaced with

public class MassTransitInstaller : IWindsorInstaller
{
public void Install(IWindsorContainer container,
IConfigurationStore store)
{
// TODO: Queues need to be configured
container.Register(
Component.For<IServiceBus>()
.UsingFactoryMethod(() =>
ServiceBusFactory.New(sbc =>
{
sbc.UseMsmq();
sbc.ReceiveFrom("msmq://localhost/
icompile_web");
sbc.UseSubscriptionService("msmq://localhost/
mt_subscriptions");
}))
.LifeStyle.Singleton);
}
}

and similarly on the application server side replacing

<configuration>
<facilities>
<facility id="masstransit">
<bus id="poller" endpoint="msmq://localhost/icompile_server">
<subscriptionService endpoint="msmq://localhost/
mt_subscriptions" />
<managementService heartbeatInterval="3" />
</bus>
<transports>
<transport>MassTransit.Transports.Msmq.MsmqEndpoint,
MassTransit.Transports.Msmq</transport>
</transports>
</facility>
</facilities>
<components>

with

public class MassTransitInstaller : IWindsorInstaller
{
public void Install(IWindsorContainer container,
IConfigurationStore store)
{
// TODO: Queues need to be configured
container.Register(
Component.For<IServiceBus>()
.UsingFactoryMethod(() =>
ServiceBusFactory.New(sbc =>
{
sbc.UseMsmq();
sbc.ReceiveFrom("msmq://localhost/
icompile_server");
sbc.UseSubscriptionService("msmq://localhost/
mt_subscriptions");
//sbc.Subscribe(subs =>
subs.LoadFrom(container));
}))
.LifeStyle.Singleton);
}
}

I can see the following 2 rows in the SubscriptionClientSaga table and
all other tables are empty.

CorrelationId CurrentState ControlUri DataUri
42E31EC7-342C-4FBC-B9C5-9F9D012150E3 Active msmq://paulcox-pc/icompile_web?tx=false
msmq://paulcox-pc/icompile_web?tx=false
98CE849A-2E07-4C26-9DFA-9F9D01276750 Active msmq://paulcox-pc/icompile_server?tx=false
msmq://paulcox-pc/icompile_server?tx=false

The log from the application service shows the following at startup
(truncated at start of stacktrace)

Topshelf v2.2.2.0, .NET Framework v4.0.30319.239
Detected the operating system: 'win'
Running as a console application, using the console host
Starting up as a console application
[Topshelf] Before starting services
[Topshelf] Before starting services complete
[Service] Create
MassTransit v2.0.1.0, .NET Framework v4.0.30319.239
Starting SubscriptionClient using msmq://localhost/mt_subscriptions
Creating MessageQueue: FormatName:DIRECT=OS:localhost\private$
\mt_subscriptions
SEND:FormatName:DIRECT=OS:localhost\private$
\mt_subscriptions:MassTransit.Services.Subscriptions.Messages.AddSubscriptionClient,
MassTransit:61427d02-28db-4bec-8b59-00d796e52767\115725
SEND:msmq://paulcox-pc/mt_subscriptions?
tx=false:AddSubscriptionClient:
Waiting for response from the subscription service
An exception occurred
Topshelf.Exceptions.TopshelfException: All services were not started
within the specified timeout
at
Topshelf.Model.ServiceCoordinator.WaitUntilServicesAreRunning(IEnumerable`1
services, TimeSpan timeout) in d:\BuildAgent-01\work\799c08e77fef999d
\src\Topshelf\Model\ServiceCoordinator.cs:line 226
at Topshelf.Model.ServiceCoordinator.Start() in d:
\BuildAgent-01\work\799c08e77fef999d\src\Topshelf\Model
\ServiceCoordinator.cs:line 118

I am using RuntimeServices because I want permanent subscriptions but
I am not averse to moving to RabbitMQ if that may make my life easier.

Sorry for the long message but I wanted to get everything in that I
could think of.

Thanks for any help,

Paul.

[Chris Patterson]

Are you running the latest MassTransit.RuntimeServices on the same machine listening to mt_subscriptions on that box? What you show should work, I'm not sure why it doesn't. Is there a log from the RuntimeServices that shows the SubscriptionRefresh message being sent back to the client?

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.
To post to this group, send email to masstrans...@googlegroups.com.
To unsubscribe from this group, send email to masstransit-dis...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/masstransit-discuss?hl=en.

[Paul Cox]

Yes, I'm running the latest MassTransit.RuntimeServices on the same
machine.

Here is the output from the MT.Messages appender for the application
service.

INFO 2011-11-16 10:15:10 SEND:msmq://lanaryu/mt_subscriptions?
tx=false:AddSubscriptionClient:

Here is the output from the MT.Messages appender for the
RuntimeServices.

INFO 10:15:12 RECV:msmq://lanaryu/mt_subscriptions?
tx=false:MassTransit.Services.Subscriptions.Messages.AddSubscriptionClient,
MassTransit:6a4f0476-2699-4e16-8068-93616047ce8b\145409
INFO 10:15:12 SEND:msmq://lanaryu/mt_subscriptions?
tx=false:SubscriptionClientAdded:
INFO 10:15:13 RECV:msmq://lanaryu/mt_subscriptions?
tx=false:MassTransit.Services.Subscriptions.Server.Messages.SubscriptionClientAdded,
MassTransit:6a4f0476-2699-4e16-8068-93616047ce8b\145410
INFO 10:15:15 SEND:msmq://lanaryu/icompile_server?
tx=false:SubscriptionRefresh:

The SubscriptionRefresh message is sitting in the icompile_server
queue unprocessed.

<?xml version="1.0"?>
<envelope><destinationAddress>msmq://lanaryu/icompile_server?tx=false</
destinationAddress><headers/><message/
><messageType>urn:message:MassTransit.Services.Subscriptions.Messages:SubscriptionRefresh</
messageType><retryCount>0</retryCount><sourceAddress>msmq://lanaryu/
mt_subscriptions?tx=false</sourceAddress></envelope>

While the system was in this state, I then started a new instance of
the Web app which produced the following log followed by a timeout
exception.

DEBUG 2011-11-16 10:27:35 Application_Start
INFO 2011-11-16 10:27:48 MassTransit v2.0.1.0, .NET Framework
v4.0.30319.239
DEBUG 2011-11-16 10:27:49 A queue was created: msmq://lanaryu/icompile_web?tx=false
DEBUG 2011-11-16 10:27:49 A queue was created: msmq://lanaryu/icompile_web_error?tx=false
DEBUG 2011-11-16 10:27:54 Starting SubscriptionClient using
msmq://localhost/mt_subscriptions
DEBUG 2011-11-16 10:27:55 Creating MessageQueue:

FormatName:DIRECT=OS:localhost\private$\mt_subscriptions

DEBUG 2011-11-16 10:27:55 SEND:FormatName:DIRECT=OS:localhost\private$
\mt_subscriptions:MassTransit.Services.Subscriptions.Messages.AddSubscriptionClient,
MassTransit:6a4f0476-2699-4e16-8068-93616047ce8b\145412
INFO 2011-11-16 10:27:55 SEND:msmq://lanaryu/mt_subscriptions?
tx=false:AddSubscriptionClient:
DEBUG 2011-11-16 10:27:55 Waiting for response from the subscription
service

The icompile_web queue had 2 unprocessed messages in it, a
SubscriptionRefresh and an AddSubscriptionClient.

It seems like the problem is that I'm just not configuring whatever
starts receiving the messages in my processes. Any ideas?

[Paul Cox]

Hi Chris,

After stepping through the code and searching the discussion forum it
seems like there is a bug in the new configuration API when using the
subscription service without a control bus as seen in this thread.

http://groups.google.com/group/masstransit-discuss/browse_thread/thread/387409611703ef32

If you do not use a separate control bus then the subscription client
attempts to wait for the bus to process the messages returned from
RuntimeServices but it hasn't yet been started.

If you do use a separate control bus then that is started before the
the actual bus you are creating sets up the subscription client and so
can process the messages received from the RuntimeServices.

If there wasn't this bug, is it normally preferable to use a control
bus anyway?

Also, out of interest are you still using MassTransit 2 with MSMQ and
RuntimeServices in any of your deployments or have you moved to using
RabbitMQ exclusively for new development?

Paul.

On Nov 16, 8:36 pm, Chris Patterson <ch...@phatboyg.com> wrote:

[Chris Patterson]

Oh, right, this one. Yeah, it's highly recommended to use a control bus with the subscription services, but we should fix it so that it works either way.

And we are not abandoning MSMQ on all platforms, just using the right transport for the right context. Having support for RabbitMQ makes for a rather nice system-to-system story using SSL (w/RabbitMQ).