[JIRA] (JENKINS-59543) Unable to provision slaves due to network interface and subnet ID both being set

22 views
Skip to first unread message

pierre-yves.bigourdan@bbc.co.uk (JIRA)

unread,
Sep 26, 2019, 8:37:03 AM9/26/19
to jenkinsc...@googlegroups.com
Pierre-Yves Bigourdan created an issue
 
Jenkins / Bug JENKINS-59543
Unable to provision slaves due to network interface and subnet ID both being set
Issue Type: Bug Bug
Assignee: FABRIZIO MANFREDI
Components: ec2-plugin
Created: 2019-09-26 12:36
Priority: Major Major
Reporter: Pierre-Yves Bigourdan

We just tried upgrading from version 1.45 to version 1.46 of the plugin. Unfortunately, we are no longer able to provision slaves with the new version. Here is the relevant stack trace:

com.amazonaws.services.ec2.model.AmazonEC2Exception: Network interfaces and an instance-level subnet ID may not be specified on the same request (Service: AmazonEC2; Status Code: 400; Error Code: InvalidParameterCombination; Request ID: faa94738-4b8b-48e0-9600-11353ef9b452)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1712)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1367)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1113)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:770)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:744)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:726)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:686)
	at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:668)
	at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:532)
	at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:512)
	at com.amazonaws.services.ec2.AmazonEC2Client.doInvoke(AmazonEC2Client.java:22420)
	at com.amazonaws.services.ec2.AmazonEC2Client.invoke(AmazonEC2Client.java:22387)
	at com.amazonaws.services.ec2.AmazonEC2Client.invoke(AmazonEC2Client.java:22376)
	at com.amazonaws.services.ec2.AmazonEC2Client.executeRunInstances(AmazonEC2Client.java:21472)
	at com.amazonaws.services.ec2.AmazonEC2Client.runInstances(AmazonEC2Client.java:21444)
	at hudson.plugins.ec2.SlaveTemplate.provisionOndemand(SlaveTemplate.java:771)
	at hudson.plugins.ec2.SlaveTemplate.provisionOndemand(SlaveTemplate.java:620)
	at hudson.plugins.ec2.SlaveTemplate.provision(SlaveTemplate.java:575)
	at hudson.plugins.ec2.EC2Cloud.getNewOrExistingAvailableSlave(EC2Cloud.java:589)
	at hudson.plugins.ec2.EC2Cloud.doProvision(EC2Cloud.java:354)
	at java.lang.invoke.MethodHandle.invokeWithArguments(MethodHandle.java:627)
	at org.kohsuke.stapler.Function$MethodFunction.invoke(Function.java:396)
	at org.kohsuke.stapler.Function$InstanceFunction.invoke(Function.java:408)
	at org.kohsuke.stapler.Function.bindAndInvoke(Function.java:212)
	at org.kohsuke.stapler.Function.bindAndInvokeAndServeResponse(Function.java:145)
	at org.kohsuke.stapler.MetaClass$11.doDispatch(MetaClass.java:535)
	at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:58)
	at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:747)
	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:878)
	at org.kohsuke.stapler.MetaClass$4.doDispatch(MetaClass.java:280)
	at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:58)
	at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:747)
	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:878)
	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:676)
	at org.kohsuke.stapler.Stapler.service(Stapler.java:238)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
	at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:873)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1623)
	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:154)
	at com.smartcodeltd.jenkinsci.plugin.assetbundler.filters.LessCSS.doFilter(LessCSS.java:47)
	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
	at hudson.plugins.greenballs.GreenBallFilter.doFilter(GreenBallFilter.java:59)
	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
	at jenkins.metrics.impl.MetricsFilter.doFilter(MetricsFilter.java:125)
	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
	at hudson.plugins.scm_sync_configuration.extensions.ScmSyncConfigurationFilter$1.call(ScmSyncConfigurationFilter.java:49)
	at hudson.plugins.scm_sync_configuration.extensions.ScmSyncConfigurationFilter$1.call(ScmSyncConfigurationFilter.java:44)
	at hudson.plugins.scm_sync_configuration.ScmSyncConfigurationDataProvider.provideRequestDuring(ScmSyncConfigurationDataProvider.java:106)
	at hudson.plugins.scm_sync_configuration.extensions.ScmSyncConfigurationFilter.doFilter(ScmSyncConfigurationFilter.java:44)
	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
	at jenkins.telemetry.impl.UserLanguages$AcceptLanguageFilter.doFilter(UserLanguages.java:128)
	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:151)
	at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:157)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
	at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:99)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84)
	at org.jenkinsci.plugins.reverse_proxy_auth.ReverseProxySecurityRealm$1.doFilter(ReverseProxySecurityRealm.java:559)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84)
	at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at jenkins.security.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:117)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:135)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at jenkins.security.BasicHeaderProcessor.doFilter(BasicHeaderProcessor.java:93)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249)
	at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:67)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:90)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:90)
	at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:171)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
	at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:49)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
	at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:82)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
	at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1610)
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:540)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:146)
	at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:257)
	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1700)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1345)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:480)
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1667)
	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1247)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
	at org.eclipse.jetty.server.Server.handle(Server.java:505)
	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:370)
	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:267)
	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305)
	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
	at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:117)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
	at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:698)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:804)
	at java.lang.Thread.run(Thread.java:748)

I noticed that code related to network interfaces had been changed as part of JENKINS-58578. When getAssociatePublicIp() returns true, the code now sets both the network interface and the subnet ID in the RunInstancesRequest, which seems to be the cause of the above stack trace.

Add Comment Add Comment
 
This message was sent by Atlassian Jira (v7.13.6#713006-sha1:cc4451f)
Atlassian logo

aaronpwalker@gmail.com (JIRA)

unread,
Sep 26, 2019, 8:40:02 AM9/26/19
to jenkinsc...@googlegroups.com
Aaron Walker commented on Bug JENKINS-59543
 
Re: Unable to provision slaves due to network interface and subnet ID both being set

Can confirm getting the same error with 1.46 .. downgraded to 1.45 and everything is working

derek@nuna.com (JIRA)

unread,
Sep 26, 2019, 10:30:03 PM9/26/19
to jenkinsc...@googlegroups.com

Encountered same issue, confirming plugin downgrade from 1.46 to 1.45 was able to unblock EC2 plugin usage.  

arkantos798@gmail.com (JIRA)

unread,
Sep 27, 2019, 7:19:02 AM9/27/19
to jenkinsc...@googlegroups.com

Confirming 1.46 breaks scaling up nodes. 1.45 works fine

Our jenkins is in version 2.197

mattvonrocketstein@gmail.com (JIRA)

unread,
Sep 27, 2019, 11:54:04 AM9/27/19
to jenkinsc...@googlegroups.com

What's the fix? with a new . jenkins instance I have no option on the UI to downgrade. 

I tried manually installing but it does not work:

```

$java -jar /tmp/jenkins-cli.jar -s http://127.0.0.1:8080/ install-plugin https://updates.jenkins-ci.org/download/plugins/ec2/1.45/ec2.hpi

Sep 27, 2019 3:51:31 PM org.apache.sshd.common.util.security.AbstractSecurityProviderRegistrar getOrCreateProvider
INFO: getOrCreateProvider(EdDSA) created instance of net.i2p.crypto.eddsa.EdDSASecurityProvider
Installing a plugin from https://updates.jenkins-ci.org/download/plugins/ec2/1.45/ec2.hpi

ERROR: Unexpected exception occurred while performing install-plugin command.
java.util.zip.ZipException: error in opening zip file

```

derek@nuna.com (JIRA)

unread,
Sep 27, 2019, 3:50:02 PM9/27/19
to jenkinsc...@googlegroups.com

matt matthews

I downloaded the prior version of the plugin from the download site here: https://updates.jenkins-ci.org/download/plugins/ec2/

Them I uninstalled the EC2 plugin, restarted Jenkins, and then manually installed the plugin via Plugin UI (see upload plugin in the advanced tab). 

 

gawbul@gmail.com (JIRA)

unread,
Sep 30, 2019, 8:52:03 AM9/30/19
to jenkinsc...@googlegroups.com

I hit this issue too. Downgrading to 1.45 resolved. Running Jenkins 2.190.1.

isuftin@usgs.gov (JIRA)

unread,
Sep 30, 2019, 10:52:03 AM9/30/19
to jenkinsc...@googlegroups.com

gawbul@gmail.com (JIRA)

unread,
Oct 1, 2019, 5:31:02 AM10/1/19
to jenkinsc...@googlegroups.com

raihaan.shouhell@autodesk.com (JIRA)

unread,
Oct 1, 2019, 6:27:03 AM10/1/19
to jenkinsc...@googlegroups.com
Change By: Raihaan Shouhell
Status: Open Fixed but Unreleased
Assignee: FABRIZIO MANFREDI Raihaan Shouhell
Resolution: Fixed

raihaan.shouhell@autodesk.com (JIRA)

unread,
Oct 1, 2019, 6:29:04 AM10/1/19
to jenkinsc...@googlegroups.com
 

This should be resolved in the latest release

Change By: Raihaan Shouhell
Status: Fixed but Unreleased Resolved
Released As: 1.46.1

phil.whelan@bench.co (JIRA)

unread,
Oct 1, 2019, 1:32:03 PM10/1/19
to jenkinsc...@googlegroups.com

lg7983@yahoo.fr (JIRA)

unread,
Oct 7, 2019, 4:12:02 PM10/7/19
to jenkinsc...@googlegroups.com

Thank you for the fix i just encounter the same issue in production.

BTW, i can we introduce automatic test for this kind of issue in my pipeline ?

Do you have a good framework ?

Reply all
Reply to author
Forward
0 new messages