What algorithm does Go use to assign a job to an agent?

[glu...@thoughtworks.com]

Hello GoCd folks, 

We have 4 Agents, 12 resources, and about 12 pipelines currently setup. During the last 3 days of iteration, we have even more pipelines running (because our release branch is also being built in parallel with master branch). 

1) How does GO decide, which agent to allocate a job to.. especially when the resource needed by that job, can be serviced by two agents? Does GO wait for an agent to be free before it assigns a job to it, or does Go predict when an agent might get free, and therefore assign a job to it, even before it finishes? Can you tell me which code file contains the algorithm to decide which agent to pick? 

2) Are there any pointers on what to consider when assigning resources to an agent. Because multiple pipelines usually have a mix of fast and slow jobs, and some stages depend on previous stages to complete (and hence a job can become a bottleneck, even if multiple agents are available)? A blog pointer maybe on this by someone?

3) Is there some way that GO can tell us that a particular agent is not being "optimally" used, versus another agent is always "loaded", versus a particular "job" is being starved most of the time, or a particular resource on an agent is not being requested for, and hence might be unnecessary, etc?

4) Why is the "duration" field value in Go-Agent Job Run History table different from "duration" value when you click on that particular job run. One seems to be (Scheduled to End), and other seems to be (Started to Finish). It seems weird that both are called "Duration"?  

We are on Go Server v13.3.

Thank you for your help : ) 

Regards

Gurpreet 

[Zabil C M]

On Thu, Apr 9, 2015 at 12:42 PM, <glu...@thoughtworks.com> wrote:

1) How does GO decide, which agent to allocate a job to.. especially when the resource needed by that job, can be serviced by two agents? Does GO wait for an agent to be free before it assigns a job to it, or does Go predict when an agent might get free, and therefore assign a job to it, even before it finishes? Can you tell me which code file contains the algorithm to decide which agent to pick? 

To figure this out you can start here

The agent polls the server for work

[Gurpreet Luthra]

Thanks for the details Zabil. 

I will look into the code. 

I understand that GO Agent polls the server for work, but in some cases the work seems to be "scheduled" for an Agent much before an Agent is free. How does that work? I guess, my question is -- how can something be scheduled for an agent, if the agent wasn't free anyways? Is it only in case that is the ONLY agent that has the resource needed by a job. In which case, Go Server decides, that hey -- anyways this is the only agent that can execute it.. so let me schedule it on it. 

Because, there seem to be times when one of our jobs runs for really long time, and we find that another job gets scheduled on it, even before the previous job finished. Is this possible only in case of a single valid agent for that job?

Regards

Gurpreet

[Aravind SV]

A job goes through these states: Scheduled, Assigned, Preparing, Building, Completing, Completed (there's rescheduled as well, but these are the main ones).

Scheduled: Server has decided that a job should be created, due to a change in some material.

Assigned: An agent pings for work, and only then does the state of a job change to "Assigned". Not before. And not because this can be the only agent that has the resources. A new agent can always come up. So, that doesn't happen.

Preparing: Agent picked up the job, and has started checking out materials, cleaning up previous build state, etc.

Building: Building

Completing: Finished building. Is uploading artifacts and finishing up.

Completed: Done.

To answer your final question: You said: "we find that another job gets scheduled on it". Scheduling doesn't happen for an agent. Scheduling is for a job. Assignment is where the agent comes into the picture. Are you saying you've seen the state as "scheduled" in the job status bar (the one that goes green and red) and an agent is decided? That seems wrong. I haven't ever seen that. Maybe I haven't noticed. But, I'd be surprised. Maybe the job is getting rescheduled, because the other agent died?

Cheers,

Aravind

--
You received this message because you are subscribed to the Google Groups "go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email to go-cd+un...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Gurpreet Luthra]

Thanks for detailing out the Job States. This helps a lot. 

>> To answer your final question: You said: "we find that another job gets scheduled on it". Scheduling doesn't happen for an agent. Scheduling is for a job. Assignment is where the agent comes into the picture. Are you saying you've seen the state as "scheduled" in the job status bar (the one that goes green and red) and an agent is decided? That seems wrong. I haven't ever seen that. Maybe I haven't noticed. But, I'd be surprised. Maybe the job is getting rescheduled, because the other agent died?

I meant in the "Job Run History" Tab of an agent. If you see below, the pdp-build duration says  1hour:7mins. Where-as if one clicks on that build, the duration says: 1min:31seconds. So, it got scheduled long time back (while the previous build -- acceptance test was still running). 

I was wondering, why in the Job Run History, the duration shows in (Completed-Scheduled). It should be (Completed-Assigned). The current duration column is confusing. Isn't it? 

Also -- regarding my other question, do you have any insights:

>> 2) Are there any pointers on what to consider when assigning resources to an agent. Because multiple pipelines usually have a mix of fast and slow jobs, and some stages depend on previous stages to complete (and hence a job can become a bottleneck, even if multiple agents are available)? A blog pointer maybe on this by someone?

>> 3) Is there some way that GO can tell us that a particular agent is not being "optimally" used, versus another agent is always "loaded", versus a particular "job" is being starved most of the time, or a particular resource on an agent is not being requested for, and hence might be unnecessary, etc?

Thanks for your help. 

Regards

Gurpreet