Iron Foundry Background Tasks, Thor Project, etc.

126 views
Skip to first unread message

Adron Hall

unread,
Oct 23, 2012, 5:07:02 PM10/23/12
to ironf...@googlegroups.com
Hello All,

Recently there has been conversation around adding background services and moving forward with adding these to the Iron Foundry DEA Code Base with Brian M. Andy Piper, Tyler Power and a few others. I wanted to get a conversations going around a few key questions:
  • What is an acceptable way for the background services to run for your particular scenarios with an Iron Foundry DEA?  Is a service running in IIS ok, do you need a Windows Service level background service or other criteria?
  • For a background service, what would be the ideal deployment path? Deploy as an application or as a service within Cloud Foundry? Do you think it would be possible to have it deployed as one vs. the other?
  • What type of status information should be available about the service as it is running? What is the minimal amount of information you need to make a background service usable in your environment?
Thanks all,
Adron

tyler

unread,
Oct 23, 2012, 5:36:57 PM10/23/12
to ironf...@googlegroups.com
Hi Adron,

Thanks for starting the discussion!

For our situation we have a number of .NET console apps, some of which are waiting on RabbitMQ to give them work, some are waiting for scheduled times and some are polling the database periodically looking for work.

So for us at least, the ability to run console apps would be ideal. We have no desire to run background tasks as Windows services or as Cloud Foundry services.

In terms of status we would simply want to make sure each console app is running, so for each instance of the app to make sure there is a process running on a DEA somewhere. If the process crashes/exits then the app would be offline. It should be the developers responsibility (for now at least) to keep their process alive e.g. no need for built in scheduling etc.

In terms of deployment if we could push, configure, scale and bind services to these background apps, just like any other current Cloud Foundry app we would be happy.

Thanks,

Tyler

Luke Bakken

unread,
Oct 23, 2012, 5:45:51 PM10/23/12
to ironf...@googlegroups.com
Hey everyone,

Tyler has the right direction for this. I was planning on implementing this using console apps that run as a non-privileged user. Unfortunately there is no open source equivalent to chroot or jail in Windows, so this user will be able to access the entire filesystem - just not mess with it.

The DEA would be responsible for starting the app and monitoring it to ensure that it's running.

As of now the Iron Foundry forks of various Cloud Foundry repositories have support for the "clr20" and "clr40" runtime. You would push this type of app and vmc would detect it as standalone (probably by looking for a *.exe.config file) and you would specify a clr runtime. The DEA would receive this info and know that the droplet is a standalone .NET app.

Luke

Adron Hall

unread,
Oct 23, 2012, 5:48:07 PM10/23/12
to ironf...@googlegroups.com
Just to add my 2 cents, I'd love to keep encouraging good practice and just disallow by default any access to the filesystem. :)  I don't want to encourage people to build anymore Wordpress style messes that heavily rely on the file system for things.


Adron B Hall
Iron Foundry Projecthttp://www.ironfoundry.org



--
You received this message because you are subscribed to the Google Groups "Iron Foundry" group.
To post to this group, send email to ironf...@googlegroups.com.
To unsubscribe from this group, send email to ironfoundry...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msg/ironfoundry/-/In2IE5hDQQUJ.

For more options, visit https://groups.google.com/groups/opt_out.
 
 

Shantu Roy

unread,
Oct 23, 2012, 5:54:28 PM10/23/12
to ironf...@googlegroups.com
Luke,
Would it make sense to have a status queue (using RabbitMQ) that each console app subscribes to, regardless of its work load and the console app posts periodic messages in this queue?  This queue in turn can be polled externally to get status of the console apps, where that is the DEA or another webapp.  So essentially each console app would need to understand RabbitMQ and also implement some kind of .net scheduler.

Thanks
Shantu

Mark Kropf

unread,
Oct 23, 2012, 6:57:02 PM10/23/12
to ironf...@googlegroups.com
I'd also like to see this run as a background console app. I can imagine an environment where these nodes are taking work off RabbitMQ, but wouldn't go as far as limiting the implementation to that type of setup. Ideally I'd like to deploy one more more applications to a IIS-less windows environment that upon connecting vmc to the node determines what types of apps I can push to it. 

Mark

tyler

unread,
Oct 23, 2012, 7:09:15 PM10/23/12
to ironf...@googlegroups.com
We need temporary access to write files to disk, some of our background workers need to dowload files, extract them and manipulate them before cleaning them up from disk.

So having write access to the 'app' directory is essential for us :)

tyler

unread,
Oct 23, 2012, 7:11:01 PM10/23/12
to ironf...@googlegroups.com
I think for now we should be careful and make this very general purpose, rather than imposing any restrictions or pre-req's that may involve having to modify existing apps.

Not all of our background apps use RabbitMQ.

tyler

unread,
Oct 23, 2012, 7:12:41 PM10/23/12
to ironf...@googlegroups.com
Luke, you say the Iron Foundry forks have support for the clr runtimes, what else is left to be done? I assume the .NET DEA needs to accept/understand these run times and it's done?

Adron Hall

unread,
Oct 24, 2012, 12:15:56 AM10/24/12
to ironf...@googlegroups.com
Yikes. Well, at least I can "encourage" people to not use file systems. That's why I was saying a "Default" vs. a hard limit.

I agree on leaving RabbitMQ out of the picture for now. The idea Shantu had would however be a pretty cool feature. Prospectively even give it some resiliency in ways that wouldn't be available otherwise.

I'll come up with a battle plan from that perspective and get some stories added on Github. Maybe we can attack the feature set from that angle?

Adron B Hall
Iron Foundry Projecthttp://www.ironfoundry.org



To view this discussion on the web visit https://groups.google.com/d/msg/ironfoundry/-/ytt0-6eUBugJ.

Brian McClain

unread,
Oct 24, 2012, 12:56:03 AM10/24/12
to ironf...@googlegroups.com
I think Tyler hit it dead-on in his first post. I've seen console application support requested the most. I assumed it would be implemented much like Standalone apps are on CF today, however if anyone has a better implementation idea, I'm not picky.

But as Tyler said, ideally I'd like standalone apps/background tasks to operate just like any other CF application. As far as Rabbit integration and such, I think it's important to support apps "as-is", as in no changes to their apps. If you're implying a wrapper around the application, I'd assume that the DEA would monitor the app rather than the app report it's status to the DEA, which would then toss status back to the NATS bus.

Adron Hall

unread,
Oct 25, 2012, 5:04:01 PM10/25/12
to ironf...@googlegroups.com
https://github.com/IronFoundry/ironfoundry/issues/24

Added the issue. Now to allocate how we can work on this puppy. Anyone want to take a first stab?

Luke -> you're most familiar with the code base, any suggestion or advice on where to start adding this functionality within the solution?

Adron B Hall
Iron Foundry Projecthttp://www.ironfoundry.org



To view this discussion on the web visit https://groups.google.com/d/msg/ironfoundry/-/91iEr4nfHmIJ.
Reply all
Reply to author
Forward
0 new messages