Does Jenkins have a restriction (limit) with the number of concurrent connections to a Git repo (via Bitbucket) ?

[Tony C]

We've got a Jenkins server that can only run 5 jobs max, if each of those jobs needs to pull code from a git repo, which is hosted in our corporate Bitbucket.

If more than 5 jobs are pulling from Bitbucketat the same time, the jobs crash. I'll have to dig up the specific error message tomorrow.

We've been struggling to find out where this restriction is, but haven't been able to.

The IT admin who maintains all of our Atlassian tools says there's nothing he can find in any of the Bitbucket configurations that is a smoking gun.

We're all fairly sure that it's not Linux that is restricted. 

That leaves us with Jenkins and its plugins.

Have any of you had a similar issue?

Thanks

[Jeff]

We regularly run dozens of jobs.  It might need more resources (CPU/Memory) or consider adding slaves (unless you are already doing that).  We limit the jobs running on the master server, and instead spin up slaves.  I run VM's in our ESX environment and configure the slaves as a general rule with 2GB and 1 CPU for every slave process.  Most of my slave VMs are running 8 slave processes (8 CPU/16GB Ram).  I have 5 slaves total.  I can't recall right off what resources are configured on the master but I'm using the generic (WAR) distribution on tomcat 8 on Ubuntu 18.

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-use...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/73cc333f-9820-4483-8930-01efbb6f1207%40googlegroups.com.

[Tony C]

Another IT group maintains the VM where the server is hosted.

however, they are generally receptive if we ask for more,

>>We limit the jobs running on the master server, and instead spin up slaves.

In this context, I'm not sure what a slave is, from your point of view.

We have 1 Jenkins server running on a VM. We have 20+ test nodes that get the test files deployed to, in order to run tests.

Not all 20 test machines are running at the same time, but easily could be. We will have more than 30 firmware engineers wanting to run tests on their firmware. 

If the test nodes are not available, we expect Jenkins to queue up those jobs, until a node becomes available.

In order to run 1 test, the Jenkins job needs to:

Pull the FW repo from git,

compile the firmware,

pull the test files

copy the FW & test files to the test node,

Download the firmware to the device being tested,

Run the test if the FW download was successful.

It could be we are resource-limited. we would need the IT group that maintains the Jenkins server VM to use some instrumentation to monitor this.

On Tuesday, August 13, 2019 at 9:41:33 PM UTC-7, Jeff wrote:

We regularly run dozens of jobs.  It might need more resources (CPU/Memory) or consider adding slaves (unless you are already doing that).  We limit the jobs running on the master server, and instead spin up slaves.  I run VM's in our ESX environment and configure the slaves as a general rule with 2GB and 1 CPU for every slave process.  Most of my slave VMs are running 8 slave processes (8 CPU/16GB Ram).  I have 5 slaves total.  I can't recall right off what resources are configured on the master but I'm using the generic (WAR) distribution on tomcat 8 on Ubuntu 18.

On Tue, Aug 13, 2019 at 10:31 PM Tony C <capp...@gmail.com> wrote:

We've got a Jenkins server that can only run 5 jobs max, if each of those jobs needs to pull code from a git repo, which is hosted in our corporate Bitbucket.

If more than 5 jobs are pulling from Bitbucketat the same time, the jobs crash. I'll have to dig up the specific error message tomorrow.

We've been struggling to find out where this restriction is, but haven't been able to.

The IT admin who maintains all of our Atlassian tools says there's nothing he can find in any of the Bitbucket configurations that is a smoking gun.

We're all fairly sure that it's not Linux that is restricted. 

That leaves us with Jenkins and its plugins.

Have any of you had a similar issue?

Thanks

--
You received this message because you are subscribed to the Google Groups "Jenkins Users" group.

To unsubscribe from this group and stop receiving emails from it, send an email to jenkins...@googlegroups.com.

[Gareth Bowles]

BitBucket server and other Git servers do have a limit on the maximum number of concurrent clones, which is a factor of the resources you have on the BitBucket server machine. See https://confluence.atlassian.com/bitbucketserver062/scaling-bitbucket-server-969537534.html

[Tony Cappellini]

Oh, that is very helpful!

I had searched with so many terms, EXCEPT for scaling. Duh.

Thank you

On Wed, Aug 14, 2019 at 10:07 AM Gareth Bowles <gar...@optimalops.net> wrote:
>
> BitBucket server and other Git servers do have a limit on the maximum number of concurrent clones, which is a factor of the resources you have on the BitBucket server machine. See https://confluence.atlassian.com/bitbucketserver062/scaling-bitbucket-server-969537534.html
>

> --
> You received this message because you are subscribed to a topic in the Google Groups "Jenkins Users" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/topic/jenkinsci-users/JzXBUT8rGjw/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to jenkinsci-use...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/6cd7e11d-1a31-4a8c-ac50-673c6684d472%40googlegroups.com.