MT, Quartz, and Job Dependency Injection
248 views
Skip to first unread message
shadow...@gmail.com
Nov 29, 2016, 7:19:04 PM11/29/16
to masstransit-discuss
Greetings masstransit-discuss!
I'm working on a new windows service and I need some best practice advice. The service's job is to periodically scan a directory full of files, move them, and then submit a message to the queue for a consumer to process each individual file. (One message per file)
There will be multiple producers and multiple consumers, though, the consumers will compete for messages rather than receive them all. The producers will be distributed but only one will be allowed to run at any given time.
The IJob instances need to know about the bus or at least something which allows it to publish a message.
I thought about having the IJob create its own instance of the IBusControl, but I don't think that is recommended. (Also its not unit testable)
I thought about creating a sort of BusRepo and using a DI container to send a singleton instance into the job but not sure if that's the right way to go either.
The other solution I came up with involves sending a callback (Action<T>) down into a custom IJobFactory in the Quartz job scheduler, and just pushing the callback around till it reaches the IJob. It keeps things 'testable' since an Action<T> is sort of loosely coupled, and it keeps to a single instance of an IBusControl, but I'm hesitant on this mostly because I don't want to write activator calls or hard-code any specific IJob types. That feels like something a DI container should be doing, but I don't know how MT would take to being injected by a container.
Mostly what I'm wondering about is whats the best practice for dealing with IBusControl instances in an environment like this?
I'm working on a new windows service and I need some best practice advice. The service's job is to periodically scan a directory full of files, move them, and then submit a message to the queue for a consumer to process each individual file. (One message per file)
There will be multiple producers and multiple consumers, though, the consumers will compete for messages rather than receive them all. The producers will be distributed but only one will be allowed to run at any given time.
The IJob instances need to know about the bus or at least something which allows it to publish a message.
I thought about having the IJob create its own instance of the IBusControl, but I don't think that is recommended. (Also its not unit testable)
I thought about creating a sort of BusRepo and using a DI container to send a singleton instance into the job but not sure if that's the right way to go either.
The other solution I came up with involves sending a callback (Action<T>) down into a custom IJobFactory in the Quartz job scheduler, and just pushing the callback around till it reaches the IJob. It keeps things 'testable' since an Action<T> is sort of loosely coupled, and it keeps to a single instance of an IBusControl, but I'm hesitant on this mostly because I don't want to write activator calls or hard-code any specific IJob types. That feels like something a DI container should be doing, but I don't know how MT would take to being injected by a container.
Mostly what I'm wondering about is whats the best practice for dealing with IBusControl instances in an environment like this?
Shadow Fox
Dec 1, 2016, 10:16:42 AM12/1/16
to masstransit-discuss
The quartz scheduler is initialized after the mass transit bus, so I just decided to pass the buscontrol (IPublishEndpoint) down into the scheduler. From there it creates a job factory which just checks the type of the job requested and passes the publish endpoint into the job. Everything seems to be working; and not a whole lot of code.
Chris Patterson
Dec 1, 2016, 10:29:08 AM12/1/16
to masstrans...@googlegroups.com
That's the simplest way, and simple is always good. Are you using Quartz as a message scheduler for MassTransit, or just using Quartz to schedule work and publishing events from those Quartz jobs? If only the latter, your solution is perfect.
On Thu, Dec 1, 2016 at 9:16 AM, Shadow Fox <shadow...@gmail.com> wrote:
The quartz scheduler is initialized after the mass transit bus, so I just decided to pass the buscontrol (IPublishEndpoint) down into the scheduler. From there it creates a job factory which just checks the type of the job requested and passes the publish endpoint into the job. Everything seems to be working; and not a whole lot of code.
--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-discuss+unsub...@googlegroups.com.
To post to this group, send email to masstransit-discuss@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/masstransit-discuss/73236d6c-f9a7-427e-a879-2b42418c994f%40googlegroups.com.
Shadow Fox
Dec 1, 2016, 10:38:13 AM12/1/16
to masstransit-discuss
Thanks for the feedback Chris!
Quartz (for now) is just for doing scheduled work, not necessarily for scheduling messages. We might do scheduled messages a bit later. Perhaps for doing cleanup or health checks or something.
This way does feel a lot cleaner than passing around function pointers basically. I'm not writing javascript! lol
Quartz (for now) is just for doing scheduled work, not necessarily for scheduling messages. We might do scheduled messages a bit later. Perhaps for doing cleanup or health checks or something.
This way does feel a lot cleaner than passing around function pointers basically. I'm not writing javascript! lol
Mark Gillen
Dec 8, 2016, 1:37:48 PM12/8/16
to masstransit-discuss
Quick question regarding your service topology:
Are you running your Quartz.Net job as a separate service from Mass Transit?
I run Quartz.net now as it's own standalone service for a variety of jobs and was going to use Mass Transit as an orchestration layer to send messages to initiate jobs in my Quartz.net instance and receive messages from those job instances. I was curious if you wrapped Mass Transit and Quatz.net together as two separate services under within on App or was just using the Quartz scheduler instantiated within Mass Transit.
Thanks! Sorry if this was a bit vague.
Mark G.
Are you running your Quartz.Net job as a separate service from Mass Transit?
I run Quartz.net now as it's own standalone service for a variety of jobs and was going to use Mass Transit as an orchestration layer to send messages to initiate jobs in my Quartz.net instance and receive messages from those job instances. I was curious if you wrapped Mass Transit and Quatz.net together as two separate services under within on App or was just using the Quartz scheduler instantiated within Mass Transit.
Thanks! Sorry if this was a bit vague.
Mark G.
Shadow Fox
Dec 8, 2016, 3:55:52 PM12/8/16
to masstransit-discuss
They're running in the same process. The jobs could probably be replaced with a simple .NET timer, but Quartz provides a clustering feature which can prevent two instances of the app from running the job at the same time. The integration between the two I chose was to instantiate the MassTransit bus on service start and pass the specific interface I needed from mass transit down into the object which configures quartz. (I wanted to inject the bus into the job so the job could submit messages for processing.)
The part which initializes the bus and the part which initializes quartz are at the same level of hierarchy except that the quartz stuff depends on the instance of the bus, so its started second.
Does that make sense?
The part which initializes the bus and the part which initializes quartz are at the same level of hierarchy except that the quartz stuff depends on the instance of the bus, so its started second.
Does that make sense?
Mark Gillen
Dec 8, 2016, 4:04:10 PM12/8/16
to masstrans...@googlegroups.com
Yep, thanks! I'm running my Quartz and MT in two separate service processes. Quartz, which is the basis of my service app that I dub "The Orbiter", does all the heavy lifting and I was afraid that sharing an process with MT might be vulnerability since some jobs can be resource intensive.
--
You received this message because you are subscribed to a topic in the Google Groups "masstransit-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/masstransit-discuss/B47zw4eWjYI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to masstransit-discuss+unsub...@googlegroups.com.
To post to this group, send email to masstransit-discuss@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/masstransit-discuss/0134360a-9573-4219-8c5b-b7cef0ad6c2a%40googlegroups.com.
Alexey Zimarev
Dec 9, 2016, 5:09:28 AM12/9/16
to masstransit-discuss
MassTransit Quartz host project has an example how these two work together.
https://github.com/MassTransit/MassTransit/tree/develop/src/MassTransit.Host.Quartz
In addition, I would really switch everything to messages, including scheduling, why not. I then can run one dedicated MT Quartz host and other services will get reliable scheduling from it.
Mark Gillen
Dec 9, 2016, 8:39:39 AM12/9/16
to masstrans...@googlegroups.com
Oh I agree. That's why I have them running as separate processes. I want all system to system dependencies to be coupled thru MT via RabbitMQ. The jobs remotely connect via MT service endpoints (MT supports different hosts as well) and nothing else. I've been building out the system utilizing Akka.net and the actor programming model. Now Akka.net provides a remoting and clustering capability as well but that strictly between actor systems. MT is the orchestration controller for all this activity.
--
You received this message because you are subscribed to a topic in the Google Groups "masstransit-discuss" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/masstransit-discuss/B47zw4eWjYI/unsubscribe.
To unsubscribe from this group and all its topics, send an email to masstransit-discuss+unsub...@googlegroups.com.
To post to this group, send email to masstransit-discuss@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/masstransit-discuss/31d4eb6d-08f3-4165-bee7-fc605d6c5232%40googlegroups.com.
Reply all
Reply to author
Forward
0 new messages