[Announcement] cloudfoundry-community/container-info-buildpack - a poor man's remote shell for CF

[da...@castlelaing.com]

All,

https://github.com/cloudfoundry-community/container-info-buildpack is a hacky attempt to simulate being able to SSH into a CF app warden container to poke around.

When deploying apps and especially when creating new buildpacks I've been getting frustrated because I have to guess what my apps warden runtime container environment is like.

This buildpack allows you to deploy an app with a web ui to run remote commands (eg ohai) and browse around the container filesystem.

Its pretty rough around the edges; PRs for improvements will be welcomed.

Hope someone finds this useful.

;D

[Dr Nic Williams]

Awesome Dave.

James/Scott (?) - what is the status of the "One Off Task" work - I don't remember it being announced as done and searching for "task" doesn't match to related items in http://cftracker.cfapps.io/runtime

Nic

To unsubscribe from this group and stop receiving emails from it, send an email to vcap-dev+u...@cloudfoundry.org.

--

Dr Nic Williams

Stark & Wayne LLC - consultancy for Cloud Foundry users

http://drnicwilliams.com

http://starkandwayne.com

cell +1 (415) 860-2185

twitter @drnic

[James Bayer]

david, this is so cool! i'd been thinking about something like this for awhile. it's so so useful! scott frederick and others will thank you profusely i'm sure.

have you thought about stacking the buildpacks together with multi-buildpack so you could use this buildpack with an app such that an app's actual buildpack gets run through it's compile phase too, but then this buildpack gets wrapped around it, it could start the app in a different process (perhaps with the web-ui) / perhaps proxies traffic to it locally in the container (since you only get one port) or maybe uses virtual hosts or something? worst case i suppose you can use curl from the web-ui to send it requests. very glad you did this!

nic, mark kropf is the pm for runtime and has a currently de-prioritized workstream around something the team has been calling 'taskmaster' that would enable one-off commands from an existing droplet like migrations, interactive commands like a terminal session, and eventually scheduled commands. 

this work is something i would like to see done soon, but it's not as high priority as:

- fixing very important bugs, found some issues in the dea's that can cause instability and unbalanced app/staging behavior that strains some deas in larger deployments

- getting our build pipeline clean, you may have noticed that we didn't do a release this week, which is extremely important that we be able to release with confidence whenever we want to, and we couldn't do that this week 

- improving the reliability and our confidence in health manager (HM9000 work)

- working on admin buildpacks to support things like ibm liberty buildpack

- garbage collection of no longer used blob store entries that have broken user deployments

i'm also very concerned about the mental model of how this all works given the existing simple construct of apps and services. i really like the heroku process model [1] and i'm hesitant to introduce separate constructs. mark has a different view that tasks are a separate concept from the main app process we have today.

i view the app as a codebase, and the named process types as different commands you can execute on that app/codebase. today we only have a single app process / command, it's 1:1 to the codebase. but we need to plan for how we think about multiple types of processes using the same codebase, and how that is conceptually represented to users and exposed via the api.

- is a Heroku inspired process model something Cloud Foundry should adopt?

- should short-lived tasks / app processes simply be just an attribute of a process type such that the system doesn't restart them when they exit?

- should tasks be conceptually separate from a long-running web app process?

- how many processes should be routable? just the web process or any app process that has routes associated?

we have lots of conceptual issues to resolve before this taskmaster work will be completed. in the meantime, i'm thrilled we have this new buildpack contribution! thanks david!

[1] https://devcenter.heroku.com/articles/process-model

Thank you,

James Bayer

[Scott Frederick]

This is very cool indeed! Anything that can give more visibility into what's going on inside a Warden container during and after staging will be welcomed by anyone working on buildpacks. 

Scott

[David Laing]

All,

Thanks for the positive reception; glad that scratching my own itch is useful to others :).

Some A&Qs:

1.  Scott - could you knock up a spec / script showing the kind of info you'd like to see during staging - https://github.com/cloudfoundry-community/container-info-buildpack/issues/1

2.  James - Could you add some pointers on how buildpack stacking could work - https://github.com/cloudfoundry-community/container-info-buildpack/issues/2

3.  James - is a Heroku inspired process model something Cloud Foundry should adopt?

Yes!  Please just adopt the whole Heroku Buildpack spec; Foreman makes Procfiles useful during development; and using the same mechanism for composing the runtime environment would be really useful.

Otherwise you're just going to get PRs from me like https://github.com/cloudfoundry/java-buildpack/pull/15 ;)

4.  James - should short-lived tasks / app processes simply be just an attribute of a process type such that the system doesn't restart them when they exit?

No.  I think conceptually processes are long lived, and I'd expect the DEA to keep them running.  I'd expect tasks to be initiated by something external to the DEA runtime.  Eg cf ssh 'bash command here'

5.  James - should tasks be conceptually separate from a long-running web app process?

Yes.  But don't force me to deploy a whole new app instance to run a single task.  Its slow, adds to my runtime costs, and useless when there are a series of exploratory tasks I want to run against an existing instance.

6. James - how many processes should be routable? just the web process or any app process that has routes associated?

I don't think there needs to be a tight coupling between a process and a route.  To me a route directs traffic into a specific port in the warden container.  I hook which ever process I want to receive that traffic up to the port indicated by the relevant env variable (eg $HTTP_PORT).  If traffic is being routed into another port (indicated by another env variable, eg $DEBUG_PORT), then I can hook up another (or the same) process to that.  

Let the app author decide how to link processes to routes (via the relevant PORT env var), rather than tightly coupling a specific process to a specific route.

--
David Laing
Open source @ City Index - github.com/cityindex
http://davidlaing.com
Twitter: @davidlaing