Disposing and recreating bus on runtime vs. waiting inside consumer

83 views
Skip to first unread message

Jeremias Kauppinen

unread,
Jun 1, 2015, 9:44:31 AM6/1/15
to masstrans...@googlegroups.com
When our service X starts we need to do two things

1.) Get initial state through HTTP API
2.) Listen to updates on the data through MT 2.x on top of RabbitMQ

We want to listen to the updates even during when we get the initial state, and apply possible updates to the state afterwards.
Some ideas:

- Dispose bus on runtime and recreate it after the inital state has been retrieved. Is this "recommended"? In Bus.Shutdown you just call .Dispose() on bus instance so I assume this will not cause problems on non-singleton implementations?
- Thread.Sleep() or some other means of sleeping inside consumer until the inital state has been retrieved
- Putting messages to some other storage/queue to wait for processment after initial state has been retrieved

Do you have have some other clever ideas? How does MT behave with RabbitMQ if sending ack takes lets say 1min? We don't have high traffic, but messages could be big.

Thanks in advance

Travis Smith

unread,
Jun 1, 2015, 10:53:22 AM6/1/15
to masstrans...@googlegroups.com
Is there a reason why you can't just get state and then start up your Bus instance afterwards? If the exchange-queue binding is already there, the messages will hang out, waiting for you. 

Having a long ACK shouldn't be an issue for RMQ. There can be a timeout for ACKs, but I believe by default the timeout is forever. If the message will be in an un-acked state until it's either ack'ed or RMQ decides the connection is dead and returns your messages to the queue. 

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-dis...@googlegroups.com.
To post to this group, send email to masstrans...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/masstransit-discuss/1355c923-708d-4b5c-aa51-7ad311fe9b2e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Jeremias Kauppinen

unread,
Jun 2, 2015, 1:51:47 AM6/2/15
to masstrans...@googlegroups.com
Good point. Although we don't want to create exchange-queue bindings manually for each message, so we would need some "pre-warming" with masstransit. Basically quickly creating and disposing the bus on runtime so that MT will make the bindings. Then we would get the state and start bus normally.

What do you think about this?


maanantai 1. kesäkuuta 2015 17.53.22 UTC+3 Travis Smith kirjoitti:
Is there a reason why you can't just get state and then start up your Bus instance afterwards? If the exchange-queue binding is already there, the messages will hang out, waiting for you. 

Having a long ACK shouldn't be an issue for RMQ. There can be a timeout for ACKs, but I believe by default the timeout is forever. If the message will be in an un-acked state until it's either ack'ed or RMQ decides the connection is dead and returns your messages to the queue. 

On Mon, Jun 1, 2015 at 9:44 AM Jeremias Kauppinen <jeremias....@gmail.com> wrote:
When our service X starts we need to do two things

1.) Get initial state through HTTP API
2.) Listen to updates on the data through MT 2.x on top of RabbitMQ

We want to listen to the updates even during when we get the initial state, and apply possible updates to the state afterwards.
Some ideas:

- Dispose bus on runtime and recreate it after the inital state has been retrieved. Is this "recommended"? In Bus.Shutdown you just call .Dispose() on bus instance so I assume this will not cause problems on non-singleton implementations?
- Thread.Sleep() or some other means of sleeping inside consumer until the inital state has been retrieved
- Putting messages to some other storage/queue to wait for processment after initial state has been retrieved

Do you have have some other clever ideas? How does MT behave with RabbitMQ if sending ack takes lets say 1min? We don't have high traffic, but messages could be big.

Thanks in advance

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-discuss+unsub...@googlegroups.com.

Travis Smith

unread,
Jun 2, 2015, 6:57:18 AM6/2/15
to masstrans...@googlegroups.com
Chris, could you set the concurrent consumer limit to zero and do that? Once the bus is up and running it will try to consume messages, what you're trying to avoid.

I wonder if we could have a CreateTopology() method that would create all the stuff inside RMQ but not fully initialize the bus. Controlling the topology is a real use case.
On Tue, Jun 2, 2015 at 1:51 AM Jeremias Kauppinen <jeremias....@gmail.com> wrote:
Good point. Although we don't want to create exchange-queue bindings manually for each message, so we would need some "pre-warming" with masstransit. Basically quickly creating and disposing the bus on runtime so that MT will make the bindings. Then we would get the state and start bus normally.

What do you think about this?


maanantai 1. kesäkuuta 2015 17.53.22 UTC+3 Travis Smith kirjoitti:
Is there a reason why you can't just get state and then start up your Bus instance afterwards? If the exchange-queue binding is already there, the messages will hang out, waiting for you. 

Having a long ACK shouldn't be an issue for RMQ. There can be a timeout for ACKs, but I believe by default the timeout is forever. If the message will be in an un-acked state until it's either ack'ed or RMQ decides the connection is dead and returns your messages to the queue. 

On Mon, Jun 1, 2015 at 9:44 AM Jeremias Kauppinen <jeremias....@gmail.com> wrote:
When our service X starts we need to do two things

1.) Get initial state through HTTP API
2.) Listen to updates on the data through MT 2.x on top of RabbitMQ

We want to listen to the updates even during when we get the initial state, and apply possible updates to the state afterwards.
Some ideas:

- Dispose bus on runtime and recreate it after the inital state has been retrieved. Is this "recommended"? In Bus.Shutdown you just call .Dispose() on bus instance so I assume this will not cause problems on non-singleton implementations?
- Thread.Sleep() or some other means of sleeping inside consumer until the inital state has been retrieved
- Putting messages to some other storage/queue to wait for processment after initial state has been retrieved

Do you have have some other clever ideas? How does MT behave with RabbitMQ if sending ack takes lets say 1min? We don't have high traffic, but messages could be big.

Thanks in advance

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-dis...@googlegroups.com.
To post to this group, send email to masstrans...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-dis...@googlegroups.com.

To post to this group, send email to masstrans...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/masstransit-discuss/89e69ed0-92f7-4fd4-b712-fd1cc581989c%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.
--
-Travis

Jeremias Kauppinen

unread,
Jun 2, 2015, 7:37:34 AM6/2/15
to masstrans...@googlegroups.com
CreateTopology() would be very useful for us. It would make passing around instance of IServiceBus nice in situations where you don't want to start the bus right away.


tiistai 2. kesäkuuta 2015 13.57.18 UTC+3 Travis Smith kirjoitti:
Chris, could you set the concurrent consumer limit to zero and do that? Once the bus is up and running it will try to consume messages, what you're trying to avoid.

I wonder if we could have a CreateTopology() method that would create all the stuff inside RMQ but not fully initialize the bus. Controlling the topology is a real use case.
On Tue, Jun 2, 2015 at 1:51 AM Jeremias Kauppinen <jeremias....@gmail.com> wrote:
Good point. Although we don't want to create exchange-queue bindings manually for each message, so we would need some "pre-warming" with masstransit. Basically quickly creating and disposing the bus on runtime so that MT will make the bindings. Then we would get the state and start bus normally.

What do you think about this?


maanantai 1. kesäkuuta 2015 17.53.22 UTC+3 Travis Smith kirjoitti:
Is there a reason why you can't just get state and then start up your Bus instance afterwards? If the exchange-queue binding is already there, the messages will hang out, waiting for you. 

Having a long ACK shouldn't be an issue for RMQ. There can be a timeout for ACKs, but I believe by default the timeout is forever. If the message will be in an un-acked state until it's either ack'ed or RMQ decides the connection is dead and returns your messages to the queue. 

On Mon, Jun 1, 2015 at 9:44 AM Jeremias Kauppinen <jeremias....@gmail.com> wrote:
When our service X starts we need to do two things

1.) Get initial state through HTTP API
2.) Listen to updates on the data through MT 2.x on top of RabbitMQ

We want to listen to the updates even during when we get the initial state, and apply possible updates to the state afterwards.
Some ideas:

- Dispose bus on runtime and recreate it after the inital state has been retrieved. Is this "recommended"? In Bus.Shutdown you just call .Dispose() on bus instance so I assume this will not cause problems on non-singleton implementations?
- Thread.Sleep() or some other means of sleeping inside consumer until the inital state has been retrieved
- Putting messages to some other storage/queue to wait for processment after initial state has been retrieved

Do you have have some other clever ideas? How does MT behave with RabbitMQ if sending ack takes lets say 1min? We don't have high traffic, but messages could be big.

Thanks in advance

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-discuss+unsub...@googlegroups.com.
To post to this group, send email to masstrans...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-discuss+unsub...@googlegroups.com.
--
-Travis

Chris Patterson

unread,
Jun 2, 2015, 10:17:46 AM6/2/15
to masstrans...@googlegroups.com
That's actually an awesome idea, and if we could make it callable from the Topshelf service install, that would be double-awesome. I'll add that as an issue for MT3.


On Tue, Jun 2, 2015 at 4:37 AM, Jeremias Kauppinen <jeremias....@gmail.com> wrote:
CreateTopology() would be very useful for us. It would make passing around instance of IServiceBus nice in situations where you don't want to start the bus right away.


tiistai 2. kesäkuuta 2015 13.57.18 UTC+3 Travis Smith kirjoitti:
Chris, could you set the concurrent consumer limit to zero and do that? Once the bus is up and running it will try to consume messages, what you're trying to avoid.

I wonder if we could have a CreateTopology() method that would create all the stuff inside RMQ but not fully initialize the bus. Controlling the topology is a real use case.
On Tue, Jun 2, 2015 at 1:51 AM Jeremias Kauppinen <jeremias....@gmail.com> wrote:
Good point. Although we don't want to create exchange-queue bindings manually for each message, so we would need some "pre-warming" with masstransit. Basically quickly creating and disposing the bus on runtime so that MT will make the bindings. Then we would get the state and start bus normally.

What do you think about this?


maanantai 1. kesäkuuta 2015 17.53.22 UTC+3 Travis Smith kirjoitti:
Is there a reason why you can't just get state and then start up your Bus instance afterwards? If the exchange-queue binding is already there, the messages will hang out, waiting for you. 

Having a long ACK shouldn't be an issue for RMQ. There can be a timeout for ACKs, but I believe by default the timeout is forever. If the message will be in an un-acked state until it's either ack'ed or RMQ decides the connection is dead and returns your messages to the queue. 

On Mon, Jun 1, 2015 at 9:44 AM Jeremias Kauppinen <jeremias....@gmail.com> wrote:
When our service X starts we need to do two things

1.) Get initial state through HTTP API
2.) Listen to updates on the data through MT 2.x on top of RabbitMQ

We want to listen to the updates even during when we get the initial state, and apply possible updates to the state afterwards.
Some ideas:

- Dispose bus on runtime and recreate it after the inital state has been retrieved. Is this "recommended"? In Bus.Shutdown you just call .Dispose() on bus instance so I assume this will not cause problems on non-singleton implementations?
- Thread.Sleep() or some other means of sleeping inside consumer until the inital state has been retrieved
- Putting messages to some other storage/queue to wait for processment after initial state has been retrieved

Do you have have some other clever ideas? How does MT behave with RabbitMQ if sending ack takes lets say 1min? We don't have high traffic, but messages could be big.

Thanks in advance

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-dis...@googlegroups.com.
To post to this group, send email to masstrans...@googlegroups.com.

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-dis...@googlegroups.com.

To post to this group, send email to masstrans...@googlegroups.com.
--
-Travis

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-dis...@googlegroups.com.

To post to this group, send email to masstrans...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/masstransit-discuss/8c0c7a12-f74d-4c41-99bd-1b8b8be83b37%40googlegroups.com.
Reply all
Reply to author
Forward
0 new messages