Iron Foundry Background Tasks, Thor Project, etc.

Showing 1-12 of 12 messages
Iron Foundry Background Tasks, Thor Project, etc. Adron Hall 10/23/12 2:07 PM
Hello All,

Recently there has been conversation around adding background services and moving forward with adding these to the Iron Foundry DEA Code Base with Brian M. Andy Piper, Tyler Power and a few others. I wanted to get a conversations going around a few key questions:
  • What is an acceptable way for the background services to run for your particular scenarios with an Iron Foundry DEA?  Is a service running in IIS ok, do you need a Windows Service level background service or other criteria?
  • For a background service, what would be the ideal deployment path? Deploy as an application or as a service within Cloud Foundry? Do you think it would be possible to have it deployed as one vs. the other?
  • What type of status information should be available about the service as it is running? What is the minimal amount of information you need to make a background service usable in your environment?
Thanks all,
Adron
Re: Iron Foundry Background Tasks, Thor Project, etc. tyler 10/23/12 2:36 PM
Hi Adron,

Thanks for starting the discussion!

For our situation we have a number of .NET console apps, some of which are waiting on RabbitMQ to give them work, some are waiting for scheduled times and some are polling the database periodically looking for work.

So for us at least, the ability to run console apps would be ideal. We have no desire to run background tasks as Windows services or as Cloud Foundry services.

In terms of status we would simply want to make sure each console app is running, so for each instance of the app to make sure there is a process running on a DEA somewhere. If the process crashes/exits then the app would be offline. It should be the developers responsibility (for now at least) to keep their process alive e.g. no need for built in scheduling etc.

In terms of deployment if we could push, configure, scale and bind services to these background apps, just like any other current Cloud Foundry app we would be happy.

Thanks,

Tyler
Re: Iron Foundry Background Tasks, Thor Project, etc. Luke Bakken 10/23/12 2:45 PM
Hey everyone,

Tyler has the right direction for this. I was planning on implementing this using console apps that run as a non-privileged user. Unfortunately there is no open source equivalent to chroot or jail in Windows, so this user will be able to access the entire filesystem - just not mess with it.

The DEA would be responsible for starting the app and monitoring it to ensure that it's running.

As of now the Iron Foundry forks of various Cloud Foundry repositories have support for the "clr20" and "clr40" runtime. You would push this type of app and vmc would detect it as standalone (probably by looking for a *.exe.config file) and you would specify a clr runtime. The DEA would receive this info and know that the droplet is a standalone .NET app.

Luke
Re: [ironfoundry] Re: Iron Foundry Background Tasks, Thor Project, etc. Adron Hall 10/23/12 2:48 PM
Just to add my 2 cents, I'd love to keep encouraging good practice and just disallow by default any access to the filesystem. :)  I don't want to encourage people to build anymore Wordpress style messes that heavily rely on the file system for things.


Adron B Hall
Iron Foundry Projecthttp://www.ironfoundry.org



--
You received this message because you are subscribed to the Google Groups "Iron Foundry" group.
To post to this group, send email to ironf...@googlegroups.com.
To unsubscribe from this group, send email to ironfoundry...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msg/ironfoundry/-/In2IE5hDQQUJ.

For more options, visit https://groups.google.com/groups/opt_out.
 
 

Re: [ironfoundry] Iron Foundry Background Tasks, Thor Project, etc. Shantu 10/23/12 2:54 PM
Luke,
Would it make sense to have a status queue (using RabbitMQ) that each console app subscribes to, regardless of its work load and the console app posts periodic messages in this queue?  This queue in turn can be polled externally to get status of the console apps, where that is the DEA or another webapp.  So essentially each console app would need to understand RabbitMQ and also implement some kind of .net scheduler.

Thanks
Shantu
--
You received this message because you are subscribed to the Google Groups "Iron Foundry" group.
To post to this group, send email to ironf...@googlegroups.com.
To unsubscribe from this group, send email to ironfoundry...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msg/ironfoundry/-/In2IE5hDQQUJ.
For more options, visit https://groups.google.com/groups/opt_out.
 
 

Re: Iron Foundry Background Tasks, Thor Project, etc. Mark Kropf 10/23/12 3:57 PM
I'd also like to see this run as a background console app. I can imagine an environment where these nodes are taking work off RabbitMQ, but wouldn't go as far as limiting the implementation to that type of setup. Ideally I'd like to deploy one more more applications to a IIS-less windows environment that upon connecting vmc to the node determines what types of apps I can push to it. 

Mark
Re: [ironfoundry] Re: Iron Foundry Background Tasks, Thor Project, etc. tyler 10/23/12 4:09 PM
We need temporary access to write files to disk, some of our background workers need to dowload files, extract them and manipulate them before cleaning them up from disk.

So having write access to the 'app' directory is essential for us :)
Re: [ironfoundry] Iron Foundry Background Tasks, Thor Project, etc. tyler 10/23/12 4:11 PM
I think for now we should be careful and make this very general purpose, rather than imposing any restrictions or pre-req's that may involve having to modify existing apps.

Not all of our background apps use RabbitMQ.
Re: Iron Foundry Background Tasks, Thor Project, etc. tyler 10/23/12 4:12 PM
Luke, you say the Iron Foundry forks have support for the clr runtimes, what else is left to be done? I assume the .NET DEA needs to accept/understand these run times and it's done?
Re: [ironfoundry] Re: Iron Foundry Background Tasks, Thor Project, etc. Adron Hall 10/23/12 9:15 PM
Yikes. Well, at least I can "encourage" people to not use file systems. That's why I was saying a "Default" vs. a hard limit.

I agree on leaving RabbitMQ out of the picture for now. The idea Shantu had would however be a pretty cool feature. Prospectively even give it some resiliency in ways that wouldn't be available otherwise.

I'll come up with a battle plan from that perspective and get some stories added on Github. Maybe we can attack the feature set from that angle?

Adron B Hall
Iron Foundry Projecthttp://www.ironfoundry.org



To view this discussion on the web visit https://groups.google.com/d/msg/ironfoundry/-/ytt0-6eUBugJ.

For more options, visit https://groups.google.com/groups/opt_out.
 
 

Re: [ironfoundry] Re: Iron Foundry Background Tasks, Thor Project, etc. Brian McClain 10/23/12 9:56 PM
I think Tyler hit it dead-on in his first post. I've seen console application support requested the most. I assumed it would be implemented much like Standalone apps are on CF today, however if anyone has a better implementation idea, I'm not picky.

But as Tyler said, ideally I'd like standalone apps/background tasks to operate just like any other CF application. As far as Rabbit integration and such, I think it's important to support apps "as-is", as in no changes to their apps. If you're implying a wrapper around the application, I'd assume that the DEA would monitor the app rather than the app report it's status to the DEA, which would then toss status back to the NATS bus.
Re: [ironfoundry] Re: Iron Foundry Background Tasks, Thor Project, etc. Adron Hall 10/25/12 2:04 PM
https://github.com/IronFoundry/ironfoundry/issues/24

Added the issue. Now to allocate how we can work on this puppy. Anyone want to take a first stab?

Luke -> you're most familiar with the code base, any suggestion or advice on where to start adding this functionality within the solution?

Adron B Hall
Iron Foundry Projecthttp://www.ironfoundry.org



To view this discussion on the web visit https://groups.google.com/d/msg/ironfoundry/-/91iEr4nfHmIJ.

For more options, visit https://groups.google.com/groups/opt_out.