Auto-scaling agent pool: getting a trigger

[Max Rothman]

I'm looking to have my agent pool auto-scale, but I'm not finding a way to trigger that scaling.

Is there a way to do any of the following?

• query the number of available agents
• check how many pipelines are waiting for agents, and if so, how long
• get any information at all that indicates agent usage

Thanks in advance!

[srinivas upadhya]

On Tuesday, September 23, 2014 2:03:12 AM UTC+5:30, Max Rothman wrote:

I'm looking to have my agent pool auto-scale, but I'm not finding a way to trigger that scaling.

Is there a way to do any of the following?

• query the number of available agents

Agents API has a "List Agents", "Enable Agent", "Disable Agent" & "Delete Agent" APIs. They were specifically built for this purpose.

• check how many pipelines are waiting for agents, and if so, how long

Scheduled Jobs API can be used.

• get any information at all that indicates agent usage

This is a little tricky. Next release has this PR which will allow you to get "Agent Job Run History" which you can use to decide if the agent should be brought down i guess.

Thanks in advance!

A few of our users have "elastic" agents.

They use Scheduled Jobs API and if "new agent" is required & H/W is available do the following:

  > spin up agent

  > enable agent

Periodically detect agent idleness & if an agent can be brought down:

  > disable agent

  > bring it down

  > delete agent 

Hope that helps.

[Max Rothman]

Thank you! Guess I should read the docs more carefully next time.

[josh.k...@gmail.com]

We are also creating an elastic scale environment.  Is there a way to progammatically interact with the "Environments"?

In particular, we will have a few different agent types (e.g.: Build, SmokeTest, PerfTests) which will require different resources and machine sizes, etc.  We also want to have a NewAgent environment where agents which come online will go through a quick validation process before we promote them to the bigger pool to identify agent setup issues early.  This seems easy enough to do, but we haven't yet found a good way to move the agents from one environment to another.  The only options we see at this point are 

1) Modify the server config manually.

    This is tricky because we will have multiple agents coming online and there would be a race condition we would have to manage externally to prevent them from mangling the config.xml.  This seems like a really bad idea unless there is some protective call we can make to ensure consistency.

2) Contributing back to Go a means of doing this.

    This is obviously a nice thing to do, and we might do it in the future (and I would post to the developer discussion forum), but we want to experiment with our environment manipulation first to see if it's worth it.  Having a simple (if hacky) way of doing things initially will allow us to hone our process before contributing back.  Also, if someone else has already done the work, that would be nice (and preferred!) : )

jd

[srinivas upadhya]

Ok. I assume you want to edit "resources" & "environments" of an Agent through API. If thats correct then you are lucky! We have an undocumented API (its not easy to use. we want to re-write it) for it:

Resource:

curl -u admin:badger --data "operation=Apply_Resource&selected[]=f6a76eaa-96ac-43a2-8255-7c898236ba22&selections[resource_name_one]=remove&selections[resource_name_three]=add" http://localhost:8153/go/api/agents/edit_agents

Environment:

curl -u admin:badger --data "operation=Apply_Environment&selected[]=f6a76eaa-96ac-43a2-8255-7c898236ba22&selections[environment_name_one]=remove&selections[environment_name_three]=add" http://localhost:8153/go/api/agents/edit_agents

[David Leung]

Hi, Srinivas. Thanks for the update.

For the environment example:
> _Environment:_

> curl -u admin:badger --data
>
"operation=Apply_Environment&selected[]=f6a76eaa-96ac-43a2-8255-7c898236ba22&selections[environment_name_one]=remove&selections[environment_name_three]=add"

> http://localhost:8153/go/api/agents/edit_agents

In the selected[]=f6a7..., that looks like a uuid. What does that
correspond to, and where do I get it?

dave

On 10/29/2014 04:46 AM, srinivas upadhya wrote:
> Ok. I assume you want to edit "resources" & "environments" of an Agent
> through API. If thats correct then you are lucky! We have an
> undocumented API (its not easy to use. we want to re-write it) for it:
>

> _Resource:_

> curl -u admin:badger --data
> "operation=Apply_Resource&selected[]=f6a76eaa-96ac-43a2-8255-7c898236ba22&selections[resource_name_one]=remove&selections[resource_name_three]=add"
> http://localhost:8153/go/api/agents/edit_agents
>

>

> --
> You received this message because you are subscribed to the Google
> Groups "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to go-cd+un...@googlegroups.com
> <mailto:go-cd+un...@googlegroups.com>.
> For more options, visit https://groups.google.com/d/optout.

--
David Leung
Senior Software Engineer
david...@vecna.com
http://www.vecna.org

Cambridge Campus
Vecna Technologies, Inc.
36 Cambridge Park Drive
Cambridge, MA 02140
Phone: (617) 864-0636
Fax: (617) 864-0638

Better Technology, Better World (TM)

The contents of this message may be privileged and confidential.
Therefore, if this message has been received in error,
please delete it without reading it. Your receipt of this message is
not intended to waive any applicable privilege.
Please do not disseminate this message without the permission of the author.

[srinivas upadhya]

Sorry missed that! You will find agent uuids in Agent Listing API.

[josh.k...@gmail.com]

Is there some keyword to select all the existing environments in the 'environment_name_one' section?

I'm hoping to do something like "set_environments(uuid, environments[])" and the result will be that it will remove all existing environments and set them only equal to the ones passed in.

[josh.k...@gmail.com]

Also, i don't appear to be able to specify multiple environments.  I tried comma delimited with no success.  I'm going to experiment with other delimiters (and maybe go read some code), but some more guidance would be appreciated.