|Iron Foundry Background Tasks, Thor Project, etc.||Adron Hall||10/23/12 2:07 PM|
Recently there has been conversation around adding background services and moving forward with adding these to the Iron Foundry DEA Code Base with Brian M. Andy Piper, Tyler Power and a few others. I wanted to get a conversations going around a few key questions:
|Re: Iron Foundry Background Tasks, Thor Project, etc.||tyler||10/23/12 2:36 PM|
Thanks for starting the discussion!
For our situation we have a number of .NET console apps, some of which are waiting on RabbitMQ to give them work, some are waiting for scheduled times and some are polling the database periodically looking for work.
So for us at least, the ability to run console apps would be ideal. We have no desire to run background tasks as Windows services or as Cloud Foundry services.
In terms of status we would simply want to make sure each console app is running, so for each instance of the app to make sure there is a process running on a DEA somewhere. If the process crashes/exits then the app would be offline. It should be the developers responsibility (for now at least) to keep their process alive e.g. no need for built in scheduling etc.
In terms of deployment if we could push, configure, scale and bind services to these background apps, just like any other current Cloud Foundry app we would be happy.
|Re: Iron Foundry Background Tasks, Thor Project, etc.||Luke Bakken||10/23/12 2:45 PM|
Tyler has the right direction for this. I was planning on implementing this using console apps that run as a non-privileged user. Unfortunately there is no open source equivalent to chroot or jail in Windows, so this user will be able to access the entire filesystem - just not mess with it.
The DEA would be responsible for starting the app and monitoring it to ensure that it's running.
As of now the Iron Foundry forks of various Cloud Foundry repositories have support for the "clr20" and "clr40" runtime. You would push this type of app and vmc would detect it as standalone (probably by looking for a *.exe.config file) and you would specify a clr runtime. The DEA would receive this info and know that the droplet is a standalone .NET app.
|Re: [ironfoundry] Re: Iron Foundry Background Tasks, Thor Project, etc.||Adron Hall||10/23/12 2:48 PM|
Just to add my 2 cents, I'd love to keep encouraging good practice and just disallow by default any access to the filesystem. :) I don't want to encourage people to build anymore Wordpress style messes that heavily rely on the file system for things.
Adron B Hall
|Re: [ironfoundry] Iron Foundry Background Tasks, Thor Project, etc.||Shantu||10/23/12 2:54 PM|
Would it make sense to have a status queue (using RabbitMQ) that each console app subscribes to, regardless of its work load and the console app posts periodic messages in this queue? This queue in turn can be polled externally to get status of the console apps, where that is the DEA or another webapp. So essentially each console app would need to understand RabbitMQ and also implement some kind of .net scheduler.
|Re: Iron Foundry Background Tasks, Thor Project, etc.||Mark Kropf||10/23/12 3:57 PM|
I'd also like to see this run as a background console app. I can imagine an environment where these nodes are taking work off RabbitMQ, but wouldn't go as far as limiting the implementation to that type of setup. Ideally I'd like to deploy one more more applications to a IIS-less windows environment that upon connecting vmc to the node determines what types of apps I can push to it.
|Re: [ironfoundry] Re: Iron Foundry Background Tasks, Thor Project, etc.||tyler||10/23/12 4:09 PM|
We need temporary access to write files to disk, some of our background workers need to dowload files, extract them and manipulate them before cleaning them up from disk.
So having write access to the 'app' directory is essential for us :)
|Re: [ironfoundry] Iron Foundry Background Tasks, Thor Project, etc.||tyler||10/23/12 4:11 PM|
I think for now we should be careful and make this very general purpose, rather than imposing any restrictions or pre-req's that may involve having to modify existing apps.
Not all of our background apps use RabbitMQ.
|Re: Iron Foundry Background Tasks, Thor Project, etc.||tyler||10/23/12 4:12 PM|
Luke, you say the Iron Foundry forks have support for the clr runtimes, what else is left to be done? I assume the .NET DEA needs to accept/understand these run times and it's done?
|Re: [ironfoundry] Re: Iron Foundry Background Tasks, Thor Project, etc.||Adron Hall||10/23/12 9:15 PM|
Yikes. Well, at least I can "encourage" people to not use file systems. That's why I was saying a "Default" vs. a hard limit.
I agree on leaving RabbitMQ out of the picture for now. The idea Shantu had would however be a pretty cool feature. Prospectively even give it some resiliency in ways that wouldn't be available otherwise.
I'll come up with a battle plan from that perspective and get some stories added on Github. Maybe we can attack the feature set from that angle?
To view this discussion on the web visit https://groups.google.com/d/msg/ironfoundry/-/ytt0-6eUBugJ.
|Re: [ironfoundry] Re: Iron Foundry Background Tasks, Thor Project, etc.||Brian McClain||10/23/12 9:56 PM|
I think Tyler hit it dead-on in his first post. I've seen console application support requested the most. I assumed it would be implemented much like Standalone apps are on CF today, however if anyone has a better implementation idea, I'm not picky.
But as Tyler said, ideally I'd like standalone apps/background tasks to operate just like any other CF application. As far as Rabbit integration and such, I think it's important to support apps "as-is", as in no changes to their apps. If you're implying a wrapper around the application, I'd assume that the DEA would monitor the app rather than the app report it's status to the DEA, which would then toss status back to the NATS bus.
|Re: [ironfoundry] Re: Iron Foundry Background Tasks, Thor Project, etc.||Adron Hall||10/25/12 2:04 PM|
Added the issue. Now to allocate how we can work on this puppy. Anyone want to take a first stab?
Luke -> you're most familiar with the code base, any suggestion or advice on where to start adding this functionality within the solution?
To view this discussion on the web visit https://groups.google.com/d/msg/ironfoundry/-/91iEr4nfHmIJ.