[JIRA] (JENKINS-59824) Ignore timeout while provisioning

7 views
Skip to first unread message

crazydfls@gmail.com (JIRA)

unread,
Oct 17, 2019, 9:14:03 AM10/17/19
to jenkinsc...@googlegroups.com
Mike K created an issue
 
Jenkins / Improvement JENKINS-59824
Ignore timeout while provisioning
Issue Type: Improvement Improvement
Assignee: Azure DevOps
Components: azure-vm-agents-plugin
Created: 2019-10-17 13:13
Priority: Major Major
Reporter: Mike K

We've set up a new custom image which takes a lot of time provisioning (40+ minutes).

I've set the deployment timeout to 5000 seconds but after around 40 minutes the VM will get deleted cause of this exception:

 

AzureVMCloud: createProvisionedAgent: Deployment win10pp16-1017121639027 not yet finished (Running): Microsoft.Compute/virtualMachines:win10pp-725b70 - waited 2,310 seconds
Oct 17, 2019 12:56:06 PM SEVERE com.microsoft.azure.vmagent.AzureVMCloud$2 callFailure creating provisioned agent 'win10pp-725b70'
com.microsoft.azure.vmagent.exceptions.AzureCloudException: AzureVMCloud: createProvisionedAgent: Deployment Failed: Microsoft.Compute/virtualMachines:win10pp-725b70 - Conflict - {status=Failed, error={code=ResourceDeploymentFailure, message=The resource operation completed with terminal provisioning state 'Failed'., details=[{code=OSProvisioningTimedOut, message=OS Provisioning for VM 'win10pp-725b70' did not finish in the allotted time. The VM may still finish provisioning successfully. Please check provisioning state later. Also, make sure the image has been properly prepared (generalized).
 * Instructions for Windows: https://azure.microsoft.com/documentation/articles/virtual-machines-windows-upload-image/ 
 * Instructions for Linux: https://azure.microsoft.com/documentation/articles/virtual-machines-linux-capture-image/ }]}}
	at com.microsoft.azure.vmagent.exceptions.AzureCloudException.create(AzureCloudException.java:37)
	at com.microsoft.azure.vmagent.AzureVMCloud.createProvisionedAgent(AzureVMCloud.java:584)
	at com.microsoft.azure.vmagent.AzureVMCloud$2.call(AzureVMCloud.java:830)
	at com.microsoft.azure.vmagent.AzureVMCloud$2.call(AzureVMCloud.java:808)
	at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
	at jenkins.security.ImpersonatingExecutorService$2.call(ImpersonatingExecutorService.java:71)
	at java.util.concurrent.FutureTask.run(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.lang.Thread.run(Unknown Source)


Is there a way to ignore this exception? The VM actually works after waiting another 10-20 minutes (it then switches from status creating to running).

Right now it will delete the VM and start creating a new one which also runs into the timout.

Add Comment Add Comment
 
This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f)
Atlassian logo

crazydfls@gmail.com (JIRA)

unread,
Oct 17, 2019, 9:15:02 AM10/17/19
to jenkinsc...@googlegroups.com
Mike K updated an issue
 
Jenkins / Bug JENKINS-59824
Ignore timeout while provisioning
Change By: Mike K
Issue Type: Improvement Bug

jieshe@microsoft.com (JIRA)

unread,
Oct 17, 2019, 9:51:02 PM10/17/19
to jenkinsc...@googlegroups.com
Jie Shen assigned an issue to Jie Shen
Change By: Jie Shen
Assignee: Azure DevOps Jie Shen

jieshe@microsoft.com (JIRA)

unread,
Oct 17, 2019, 9:53:02 PM10/17/19
to jenkinsc...@googlegroups.com
Jie Shen commented on Bug JENKINS-59824
 
Re: Ignore timeout while provisioning

According to discussion at https://github.com/Azure/azure-sdk-for-node/issues/1491, it seems to be a service side issue. I also have not figured out how to change it in the java sdk. So it is possible to improve the image itself or try to deploy to different locations?

crazydfls@gmail.com (JIRA)

unread,
Oct 18, 2019, 5:51:06 AM10/18/19
to jenkinsc...@googlegroups.com
Mike K commented on Bug JENKINS-59824

I was already troubleshooting the used image with the MS Azure support.
I'm using a custom generalized Win10 image with software and frameworks preinstalled. The reinstallation of that apps just takes a lot of time and sadly can't be speed up.

The VM is actually provisioning fine it just exceeds the 40 minutes after which Azure seems to throw the timeout exception.

As the support told me there is no way to increase the timeout on Azure I would expect the vm agent plugin to respect my set deploy timeout value.
The thrown exception can be clearly identified as timeout exception (by it's title and message. I can provide a log entry if needed) so I'd suggest to not abort the wait process if the timeout exception is thrown before the plugins deploy timeout is exceeded.

Reply all
Reply to author
Forward
0 new messages