[JIRA] (JENKINS-59633) EC2 Plguin: Windows EC2 instances are not launching

40 views
Skip to first unread message

tapvirvirk@gmail.com (JIRA)

unread,
Oct 2, 2019, 4:47:01 PM10/2/19
to jenkinsc...@googlegroups.com
tapvir virk created an issue
 
Jenkins / Bug JENKINS-59633
EC2 Plguin: Windows EC2 instances are not launching
Issue Type: Bug Bug
Assignee: Francis Upton
Components: ec2-plugin
Created: 2019-10-02 20:46
Priority: Critical Critical
Reporter: tapvir virk

Hi,

Getting below exception on windows slave. It is working fine in 1.43 version

Unexpected Exception: com.amazonaws.services.ec2.model.AmazonEC2Exception: You are not authorized to perform this operation. (Service: AmazonEC2; Status Code: 403; Error Code: UnauthorizedOperation; Request ID: 071f3254-51f1-4577-810f-132237add295)

 
Add Comment Add Comment
 
This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f)
Atlassian logo

raihaan.shouhell@autodesk.com (JIRA)

unread,
Oct 2, 2019, 10:45:02 PM10/2/19
to jenkinsc...@googlegroups.com
Raihaan Shouhell commented on Bug JENKINS-59633
 
Re: EC2 Plguin: Windows EC2 instances are not launching

What version are you on and can you show a screenshot of your settings? This might be because you used auto retrieve password from amazon which requires a new permission.

tapvirvirk@gmail.com (JIRA)

unread,
Oct 3, 2019, 9:41:02 AM10/3/19
to jenkinsc...@googlegroups.com
tapvir virk updated an issue
 
Change By: tapvir virk
Attachment: access_denied_windows_ec2_plugin.gif

tapvirvirk@gmail.com (JIRA)

unread,
Oct 3, 2019, 9:42:04 AM10/3/19
to jenkinsc...@googlegroups.com
tapvir virk commented on Bug JENKINS-59633
 
Re: EC2 Plguin: Windows EC2 instances are not launching

Attaching the screenshot of the ec2 slave setting. Do you know which permission is required? 

tapvirvirk@gmail.com (JIRA)

unread,
Oct 3, 2019, 9:46:01 AM10/3/19
to jenkinsc...@googlegroups.com
tapvir virk edited a comment on Bug JENKINS-59633
Attaching the screenshot of the ec2 slave setting. Do you know which permission is required? 


Though I can see the instance getting launched on aws account.

raihaan.shouhell@autodesk.com (JIRA)

unread,
Oct 8, 2019, 11:14:02 AM10/8/19
to jenkinsc...@googlegroups.com

You need to check specify password and specify your admin password or give getpassworddata to your appropriate role

tapvirvirk@gmail.com (JIRA)

unread,
Oct 9, 2019, 9:56:03 AM10/9/19
to jenkinsc...@googlegroups.com

I'm running Jenkins 2.476.3 version. Using latest ec2 plugin.

 

Do you know which permission needs to be added?

 

tapvirvirk@gmail.com (JIRA)

unread,
Oct 9, 2019, 1:40:02 PM10/9/19
to jenkinsc...@googlegroups.com

I have given the permission to retrieve the password. Now, I'm not getting the above exception. But instance/slave node is getting terminated right after 4-5 mins

tapvirvirk@gmail.com (JIRA)

unread,
Oct 9, 2019, 1:41:03 PM10/9/19
to jenkinsc...@googlegroups.com
tapvir virk assigned an issue to Raihaan Shouhell
 
Change By: tapvir virk
Assignee: Francis Upton Raihaan Shouhell

tapvirvirk@gmail.com (JIRA)

unread,
Oct 9, 2019, 1:45:02 PM10/9/19
to jenkinsc...@googlegroups.com
tapvir virk edited a comment on Bug JENKINS-59633
I have given the permission to retrieve the password. Now, I'm not getting the above exception. But instance/slave node is getting terminated right after 4-5 mins with the below exception
ERROR: Unable to decode password:
java.lang.IllegalArgumentException: Illegal base64 character d
com.amazonaws.AmazonClientException: Unable to decode password:
java.lang.IllegalArgumentException: Illegal base64 character d
at hudson.plugins.ec2.EC2PrivateKey.decryptWindowsPassword(EC2PrivateKey.java:141)
at hudson.plugins.ec2.win.EC2WindowsLauncher.connectToWinRM(EC2WindowsLauncher.java:156)
at hudson.plugins.ec2.win.EC2WindowsLauncher.launchScript(EC2WindowsLauncher.java:39)
at hudson.plugins.ec2.EC2ComputerLauncher.launch(EC2ComputerLauncher.java:48)
at hudson.slaves.SlaveComputer$1.call(SlaveComputer.java:294)
at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46)
at jenkins.security.ImpersonatingExecutorService$2.call(ImpersonatingExecutorService.java:71)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

tapvirvirk@gmail.com (JIRA)

unread,
Oct 9, 2019, 3:37:03 PM10/9/19
to jenkinsc...@googlegroups.com

I think it is related to newline characters returned in the PasswordData property from "getPasswordData" api call.
I tried to replicate on my local machine with standalone application. With newline characters, it is giving me below exception,

Exception in thread "main" org.bouncycastle.util.encoders.DecoderException: unable to decode base64 string: invalid characters encountered in base64 dataException in thread "main" org.bouncycastle.util.encoders.DecoderException: unable to decode base64 string: invalid characters encountered in base64 data at org.bouncycastle.util.encoders.Base64.decode(Unknown Source) at com.telushealth.thcp.pipeline.gradle.aws.ec2.AwsEC2GetPasswordTest.decryptWindowsPassword(AwsEC2GetPasswordTest.java:51) at com.telushealth.thcp.pipeline.gradle.aws.ec2.AwsEC2GetPasswordTest.main(AwsEC2GetPasswordTest.java:19)Caused by: java.io.IOException: invalid characters encountered in base64 data at org.bouncycastle.util.encoders.Base64Encoder.decode(Unknown Source) ... 3 more

tapvirvirk@gmail.com (JIRA)

unread,
Oct 10, 2019, 11:29:02 AM10/10/19
to jenkinsc...@googlegroups.com

Hi Raihaan Shouhell
As per your comment in JENKINS-58193 ,

I can see the getPasswordData cli returning \r\n prefixing the content.

tapvirvirk@gmail.com (JIRA)

unread,
Oct 10, 2019, 11:35:02 AM10/10/19
to jenkinsc...@googlegroups.com
tapvir virk edited a comment on Bug JENKINS-59633
Hi [~raihaan]

As per your comment in JENKINS-58193 ,

I can see the getPasswordData cli returning \r\n prefixing the content.


I haven't set any password in the ami. I'm just running below script in the userdata,

<powershell>

# Disable Credssp for Jenkins master to login via WinRM.
winrm set winrm/config/service/auth '@\{CredSSP="false"}'

$osString = wmic os get caption | Out-String
echo "Windows OS: $osString"
</powershell>

tapvirvirk@gmail.com (JIRA)

unread,
Oct 10, 2019, 11:51:01 AM10/10/19
to jenkinsc...@googlegroups.com

Below is the cli request and the response

aws ec2 get-password-data --instance-id i-xxxxxx --region ca-central-1

{ "InstanceId": "i-xxxxx", "Timestamp": "2019-10-10T15:48:21.000Z", "PasswordData": "\r\nPdbgYc5e0hJZAxqgrcGSPdEGmrno+6kCXej8hApQC2f05pfjCnUZVb/vG/57K42oN8skN6OLE5WGGBaKsYzjAl1qoXq0pw1WBANbsqJgeZWJLKG6O4B9XKog2YPHtb4+aaxnbWK+gU84cBGbGmrEbs5XSnpt54/WA5MOqi0iNV4LTnYc3DfEpGHR4Ir6Zjl0qo9FFOUD5KbA0BMiiRXD9/JguIKGHlES12G/0ko8pJhD3g7GMykWWjt2AAL52UlvUd/pxqzQZHbszrQ1xqKD12XDVs1ewJcl7pzCPKbRYg0hOrAeih4gVCcWNneTf27VObj0v4PSOQAFv3lfTB2jMA==\r\n" }

tapvirvirk@gmail.com (JIRA)

unread,
Oct 29, 2019, 8:59:03 PM10/29/19
to jenkinsc...@googlegroups.com
tapvir virk updated an issue
Change By: tapvir virk
Priority: Critical Blocker

tapvirvirk@gmail.com (JIRA)

unread,
Oct 29, 2019, 9:43:02 PM10/29/19
to jenkinsc...@googlegroups.com
 
Re: EC2 Plguin: Windows EC2 instances are not launching

Raihaan Shouhell - Did you get a chance to look at the above issue?

raihaan.shouhell@autodesk.com (JIRA)

unread,
Oct 29, 2019, 10:34:03 PM10/29/19
to jenkinsc...@googlegroups.com

tapvir virk I couldn't reproduce this issue. What ami are you using, if its public?

 

jason.boyd.ce@gmail.com (JIRA)

unread,
Nov 1, 2019, 5:44:03 PM11/1/19
to jenkinsc...@googlegroups.com

I am experiencing the same issue using plugin version 1.46.1. AMI is ami-0a6b38f2d62c0cc94: Microsoft Windows Server 2019 Base with Containers.

aws ec2 get-password-data --instance-id i-044a30c5ada9d3eb6 --region us-west-2

{ "InstanceId": "i-044a30c5ada9d3eb6", "PasswordData": "\r\nmG0KvA0Ll65mN0PiKbA6DKVwWD8mUcGRgfIqaCeXOh/evAcEpPY2DRTPYBF8IdVJeTfsZWM5bC6a1rgca9006gl6GKEKAIScfYcBxzdtZtSUVN1/xSBLdxuQSd6BZoqG9lAnhdZL6sWs8AWFgZle+7/1G8/k2GZx7R1lz59oMcOeHao+ggcofF8pJ+cyt2ytHcOYLhdNRX23xIM1OfaAjRtrkOYxV4r+c5en+Bisns79iC9LC700nbkSiGPHVdM0eLor78wiAwjmFVj3ukMVylEOMsTcDIIPjnF0Cq7eyssgblZd1v/rVlHS13iMMvYuoFs+ALBMmuOQlzeuYxScnw==\r\n", "Timestamp": "2019-11-01T21:36:18.000Z" }

raphael.pionke@t-systems.com (JIRA)

unread,
Dec 3, 2019, 11:12:03 AM12/3/19
to jenkinsc...@googlegroups.com

i submitted a pull request, which fixes the problem: https://github.com/jenkinsci/ec2-plugin/pull/418

tapvirvirk@gmail.com (JIRA)

unread,
Dec 3, 2019, 11:23:03 AM12/3/19
to jenkinsc...@googlegroups.com

Raphael Pionke - Can you give me the hpi file to test on my jenkins instance?

raphael.pionke@t-systems.com (JIRA)

unread,
Dec 3, 2019, 11:32:05 AM12/3/19
to jenkinsc...@googlegroups.com

raphael.pionke@t-systems.com (JIRA)

unread,
Dec 3, 2019, 11:32:06 AM12/3/19
to jenkinsc...@googlegroups.com
Raphael Pionke started work on Bug JENKINS-59633
 
Change By: Raphael Pionke
Status: Open In Progress

tapvirvirk@gmail.com (JIRA)

unread,
Dec 3, 2019, 11:50:03 AM12/3/19
to jenkinsc...@googlegroups.com

Raphael Pionke Fix is working for me.
Thanks

raihaan.shouhell@autodesk.com (JIRA)

unread,
Dec 3, 2019, 8:43:03 PM12/3/19
to jenkinsc...@googlegroups.com
Change By: Raihaan Shouhell
Status: In Progress Review

raihaan.shouhell@autodesk.com (JIRA)

unread,
Dec 3, 2019, 8:43:03 PM12/3/19
to jenkinsc...@googlegroups.com
Reply all
Reply to author
Forward
0 new messages