[JIRA] (JENKINS-38829) "Kubernetes server certificate key" not working

2 views
Skip to first unread message

akostadinov@java.net (JIRA)

unread,
Oct 7, 2016, 4:19:05 PM10/7/16
to jenkinsc...@googlegroups.com
akostadinov created an issue
 
Jenkins / Bug JENKINS-38829
"Kubernetes server certificate key" not working
Issue Type: Bug Bug
Assignee: Carlos Sanchez
Components: kubernetes-plugin
Created: 2016/Oct/07 8:18 PM
Priority: Critical Critical
Reporter: akostadinov

In JENKINS-29213 a parameter has been added "Kubernetes server certificate key".

It has no help but looking at the relevant commit I see it should read PEM formatted server certificate. This doesn't appear to work for me. Jenkins server doesn't use a self-signed certificate, it uses a corporate internal CA thus I put into the field the server certificate and it's CAs. This doesn't work. I tried changing order of certificates to no avail.

I think help needs to be added about format and order of certificates so that they can work.

Full backtrace:

javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
	at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
	at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1884)
	at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:276)
	at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:270)
	at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1341)
	at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:153)
	at sun.security.ssl.Handshaker.processLoop(Handshaker.java:868)
	at sun.security.ssl.Handshaker.process_record(Handshaker.java:804)
	at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1016)
	at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1312)
	at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1339)
	at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1323)
	at org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:394)
	at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:353)
	at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:134)
	at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:353)
	at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:380)
	at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236)
	at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
	at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)
	at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
	at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
	at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:107)
	at org.csanchez.jenkins.plugins.kubernetes.OpenShiftBearerTokenCredentialImpl.refreshToken(OpenShiftBearerTokenCredentialImpl.java:122)
	at org.csanchez.jenkins.plugins.kubernetes.OpenShiftBearerTokenCredentialImpl.getToken(OpenShiftBearerTokenCredentialImpl.java:67)
	at org.csanchez.jenkins.plugins.kubernetes.KubernetesFactoryAdapter.createClient(KubernetesFactoryAdapter.java:64)
	at org.csanchez.jenkins.plugins.kubernetes.KubernetesCloud$DescriptorImpl.doTestConnection(KubernetesCloud.java:588)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.kohsuke.stapler.Function$InstanceFunction.invoke(Function.java:298)
	at org.kohsuke.stapler.Function.bindAndInvoke(Function.java:161)
	at org.kohsuke.stapler.Function.bindAndInvokeAndServeResponse(Function.java:96)
	at org.kohsuke.stapler.MetaClass$1.doDispatch(MetaClass.java:121)
	at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:53)
	at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:746)
	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:876)
	at org.kohsuke.stapler.MetaClass$6.doDispatch(MetaClass.java:249)
	at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:53)
	at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:746)
	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:876)
	at org.kohsuke.stapler.Stapler.invoke(Stapler.java:649)
	at org.kohsuke.stapler.Stapler.service(Stapler.java:238)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
	at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:686)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1494)
	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:123)
	at com.sonymobile.jenkins.plugins.kerberossso.KerberosSSOFilter.doFilter(KerberosSSOFilter.java:235)
	at hudson.util.PluginServletFilter$1.doFilter(PluginServletFilter.java:120)
	at hudson.util.PluginServletFilter.doFilter(PluginServletFilter.java:114)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1482)
	at hudson.security.csrf.CrumbFilter.doFilter(CrumbFilter.java:85)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1482)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:84)
	at hudson.security.UnwrapSecurityExceptionFilter.doFilter(UnwrapSecurityExceptionFilter.java:51)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at jenkins.security.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:117)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at org.acegisecurity.providers.anonymous.AnonymousProcessingFilter.doFilter(AnonymousProcessingFilter.java:125)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at org.acegisecurity.ui.rememberme.RememberMeProcessingFilter.doFilter(RememberMeProcessingFilter.java:142)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at org.acegisecurity.ui.AbstractProcessingFilter.doFilter(AbstractProcessingFilter.java:271)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at jenkins.security.BasicHeaderProcessor.doFilter(BasicHeaderProcessor.java:93)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at org.acegisecurity.context.HttpSessionContextIntegrationFilter.doFilter(HttpSessionContextIntegrationFilter.java:249)
	at hudson.security.HttpSessionContextIntegrationFilter2.doFilter(HttpSessionContextIntegrationFilter2.java:67)
	at hudson.security.ChainedServletFilter$1.doFilter(ChainedServletFilter.java:87)
	at hudson.security.ChainedServletFilter.doFilter(ChainedServletFilter.java:76)
	at hudson.security.HudsonFilter.doFilter(HudsonFilter.java:168)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1482)
	at org.kohsuke.stapler.compression.CompressionFilter.doFilter(CompressionFilter.java:49)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1482)
	at hudson.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:81)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1482)
	at org.kohsuke.stapler.DiagnosticThreadNameFilter.doFilter(DiagnosticThreadNameFilter.java:30)
	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1474)
	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:499)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137)
	at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:533)
	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231)
	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086)
	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:428)
	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193)
	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020)
	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135)
	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116)
	at org.eclipse.jetty.server.Server.handle(Server.java:370)
	at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489)
	at org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:960)
	at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1021)
	at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:865)
	at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:240)
	at org.eclipse.jetty.server.AsyncHttpConnection.handle(AsyncHttpConnection.java:82)
	at org.eclipse.jetty.io.nio.SelectChannelEndPoint.handle(SelectChannelEndPoint.java:668)
	at org.eclipse.jetty.io.nio.SelectChannelEndPoint$1.run(SelectChannelEndPoint.java:52)
	at winstone.BoundedExecutorService$1.run(BoundedExecutorService.java:77)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
	at java.lang.Thread.run(Thread.java:745)
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
	at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:385)
	at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:292)
	at sun.security.validator.Validator.validate(Validator.java:260)
	at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:326)
	at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:231)
	at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:126)
	at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1323)
	... 98 more
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
	at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:196)
	at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:268)
	at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:380)
	... 104 more
Add Comment Add Comment
 
This message was sent by Atlassian JIRA (v7.1.7#71011-sha1:2526d7c)
Atlassian logo

jenkins-ci@carlossanchez.eu (JIRA)

unread,
Oct 12, 2016, 5:15:02 PM10/12/16
to jenkinsc...@googlegroups.com

akostadinov@java.net (JIRA)

unread,
Oct 13, 2016, 2:34:02 AM10/13/16
to jenkinsc...@googlegroups.com

Hello again. Looking at your test cases I understood and tested what the issue is. The certificate you test with is actually a PEM encoded cert but then base64 encoded. I think that it makes more sense to ask from user simply a PEM cert and base64 encode it upon passing to the kubernetes API. Also your new help text also indicates normal PEM encoding is needed while I could make it work only after additionally base64 encoding it.

i.e. I'd expect that I need to put this in the field this:

-----BEGIN CERTIFICATE-----
MIIC6jCCAdKgAwIBAgIBATANBgkqhkiG9w0BAQsFADAmMSQwIgYDVQQDDBtvcGVu
c2hpZnQtc2lnbmVyQDE0NzU4NzkyMDYwHhcNMTYxMDA3MjIyNjQ2WhcNMjExMDA2
MjIyNjQ3WjAmMSQwIgYDVQQDDBtvcGVuc2hpZnQtc2lnbmVyQDE0NzU4NzkyMDYw
ggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQC8e9H5mVf6T0VYzG3cUkXk
USArFStbZs1XQ0mOqM25DK/wPJ464NHcYBRlgnONqdvlwu/sMHTw9cIelWi9peAg
K5EcoDrp5spXwY/NSjmBtEV2+w+3FvvjZQhGhde79vafTiRAfCBJVvhZC6DwGzH6
c7axTNjF82WjQ1G5lv4pXpanj4zKpX4DCchuE1zet2UmDkDtl1vcho/eiTc737Bg
rXOCau3LbTtFlw2Cg1gzk5YlWTzB1DaprP/55+Ks6dUKKX7WsSALErv6Sj0RW1//
CQ6OqtACznyYjXRHH9zLBys7YV27gfvyqujdV8Obk66sI+kT9Pf8jW02GVUt//9R
AgMBAAGjIzAhMA4GA1UdDwEB/wQEAwICpDAPBgNVHRMBAf8EBTADAQH/MA0GCSqG
SIb3DQEBCwUAA4IBAQC7OhnypzgmpBN62GgDbFZTsFSmvxJHNjKeZfDJr1i/r2oZ
a/0SZ5Q2N7B5wJegA6mcjYD0p7i9++mRL5niwX2yIWSpMMcB1plDXMwraZyn0VqY
uXlOYYEQNsOOXcmGnIkRxMwEnqDVS6WSMDh0kRcLrWjockpErGlwSr60WLLUf2rD
rW3MNI6cMMY//XQyHjGl0s7YWj1hjprsQWUGcCz7GGBxYYGbE4KGUWSx2gnCloxw
C7ZOtxzSdKyNiY+uI9qxWiIxtezilH7CHlr3z/nWR+IKr+MB2ZGFbsZ//igS1nno
kPW40Ipo23cchopITlDVVusdlHVUELTeNn9Svvzk
-----END CERTIFICATE-----

But I had to put this instead:

 
                                                            

LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUM2akNDQWRLZ0F3SUJBZ0lCQVRBTkJna3Foa2lHOXcwQkFRc0ZBREFtTVNRd0lnWURWUVFEREJ0dmNHVnUKYzJocFpuUXRjMmxuYm1WeVFERTBOelU0TnpreU1EWXdIaGNOTVRZeE1EQTNNakl5TmpRMldoY05NakV4TURBMgpNakl5TmpRM1dqQW1NU1F3SWdZRFZRUUREQnR2Y0dWdWMyaHBablF0YzJsbmJtVnlRREUwTnpVNE56a3lNRFl3CmdnRWlNQTBHQ1NxR1NJYjNEUUVCQVFVQUE0SUJEd0F3Z2dFS0FvSUJBUUM4ZTlINW1WZjZUMFZZekczY1VrWGsKVVNBckZTdGJaczFYUTBtT3FNMjVESy93UEo0NjROSGNZQlJsZ25PTnFkdmx3dS9zTUhUdzljSWVsV2k5cGVBZwpLNUVjb0RycDVzcFh3WS9OU2ptQnRFVjIrdyszRnZ2alpRaEdoZGU3OXZhZlRpUkFmQ0JKVnZoWkM2RHdHekg2CmM3YXhUTmpGODJXalExRzVsdjRwWHBhbmo0ektwWDREQ2NodUUxemV0MlVtRGtEdGwxdmNoby9laVRjNzM3QmcKclhPQ2F1M0xiVHRGbHcyQ2cxZ3prNVlsV1R6QjFEYXByUC81NStLczZkVUtLWDdXc1NBTEVydjZTajBSVzEvLwpDUTZPcXRBQ3pueVlqWFJISDl6TEJ5czdZVjI3Z2Z2eXF1amRWOE9iazY2c0kra1Q5UGY4alcwMkdWVXQvLzlSCkFnTUJBQUdqSXpBaE1BNEdBMVVkRHdFQi93UUVBd0lDcERBUEJnTlZIUk1CQWY4RUJUQURBUUgvTUEwR0NTcUcKU0liM0RRRUJDd1VBQTRJQkFRQzdPaG55cHpnbXBCTjYyR2dEYkZaVHNGU212eEpITmpLZVpmREpyMWkvcjJvWgphLzBTWjVRMk43QjV3SmVnQTZtY2pZRDBwN2k5KyttUkw1bml3WDJ5SVdTcE1NY0IxcGxEWE13cmFaeW4wVnFZCnVYbE9ZWUVRTnNPT1hjbUduSWtSeE13RW5xRFZTNldTTURoMGtSY0xyV2pvY2twRXJHbHdTcjYwV0xMVWYyckQKclczTU5JNmNNTVkvL1hReUhqR2wwczdZV2oxaGpwcnNRV1VHY0N6N0dHQnhZWUdiRTRLR1VXU3gyZ25DbG94dwpDN1pPdHh6U2RLeU5pWSt1STlxeFdpSXh0ZXppbEg3Q0hscjN6L25XUitJS3IrTUIyWkdGYnNaLy9pZ1Mxbm5vCmtQVzQwSXBvMjNjY2hvcElUbERWVnVzZGxIVlVFTFRlTm45U3Z2emsKLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=

 
                                                            

akostadinov@java.net (JIRA)

unread,
Oct 13, 2016, 2:35:02 AM10/13/16
to jenkinsc...@googlegroups.com
akostadinov edited a comment on Bug JENKINS-38829
Hello again. Looking at your test cases I understood and tested what the issue is. The certificate you test with is actually a PEM encoded cert but then base64 encoded. I think that it makes more sense to ask from user simply a PEM cert and base64 encode it upon passing to the kubernetes API. Also your new help text also indicates normal PEM encoding is needed while I could make it work only after additionally base64 encoding it.

i.e. I'd expect that I need to put this in the field this:
{CODE}

-----BEGIN CERTIFICATE-----
MIIC6jCCAdKgAwIBAgIBATANBgkqhkiG9w0BAQsFADAmMSQwIgYDVQQDDBtvcGVu
c2hpZnQtc2lnbmVyQDE0NzU4NzkyMDYwHhcNMTYxMDA3MjIyNjQ2WhcNMjExMDA2
MjIyNjQ3WjAmMSQwIgYDVQQDDBtvcGVuc2hpZnQtc2lnbmVyQDE0NzU4NzkyMDYw
ggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQC8e9H5mVf6T0VYzG3cUkXk
USArFStbZs1XQ0mOqM25DK/wPJ464NHcYBRlgnONqdvlwu/sMHTw9cIelWi9peAg
K5EcoDrp5spXwY/NSjmBtEV2+w+3FvvjZQhGhde79vafTiRAfCBJVvhZC6DwGzH6
c7axTNjF82WjQ1G5lv4pXpanj4zKpX4DCchuE1zet2UmDkDtl1vcho/eiTc737Bg
rXOCau3LbTtFlw2Cg1gzk5YlWTzB1DaprP/55+Ks6dUKKX7WsSALErv6Sj0RW1//
CQ6OqtACznyYjXRHH9zLBys7YV27gfvyqujdV8Obk66sI+kT9Pf8jW02GVUt//9R
AgMBAAGjIzAhMA4GA1UdDwEB/wQEAwICpDAPBgNVHRMBAf8EBTADAQH/MA0GCSqG
SIb3DQEBCwUAA4IBAQC7OhnypzgmpBN62GgDbFZTsFSmvxJHNjKeZfDJr1i/r2oZ
a/0SZ5Q2N7B5wJegA6mcjYD0p7i9++mRL5niwX2yIWSpMMcB1plDXMwraZyn0VqY
uXlOYYEQNsOOXcmGnIkRxMwEnqDVS6WSMDh0kRcLrWjockpErGlwSr60WLLUf2rD
rW3MNI6cMMY//XQyHjGl0s7YWj1hjprsQWUGcCz7GGBxYYGbE4KGUWSx2gnCloxw
C7ZOtxzSdKyNiY+uI9qxWiIxtezilH7CHlr3z/nWR+IKr+MB2ZGFbsZ//igS1nno
kPW40Ipo23cchopITlDVVusdlHVUELTeNn9Svvzk
-----END CERTIFICATE-----
{CODE}


But I had to put this instead:
{ CODE code }
LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUM2akNDQWRLZ0F3SUJBZ0lCQVRBTkJna3Foa2lHOXcwQkFRc0ZBREFtTVNRd0lnWURWUVFEREJ0dmNHVnUKYzJocFpuUXRjMmxuYm1WeVFERTBOelU0TnpreU1EWXdIaGNOTVRZeE1EQTNNakl5TmpRMldoY05NakV4TURBMgpNakl5TmpRM1dqQW1NU1F3SWdZRFZRUUREQnR2Y0dWdWMyaHBablF0YzJsbmJtVnlRREUwTnpVNE56a3lNRFl3CmdnRWlNQTBHQ1NxR1NJYjNEUUVCQVFVQUE0SUJEd0F3Z2dFS0FvSUJBUUM4ZTlINW1WZjZUMFZZekczY1VrWGsKVVNBckZTdGJaczFYUTBtT3FNMjVESy93UEo0NjROSGNZQlJsZ25PTnFkdmx3dS9zTUhUdzljSWVsV2k5cGVBZwpLNUVjb0RycDVzcFh3WS9OU2ptQnRFVjIrdyszRnZ2alpRaEdoZGU3OXZhZlRpUkFmQ0JKVnZoWkM2RHdHekg2CmM3YXhUTmpGODJXalExRzVsdjRwWHBhbmo0ektwWDREQ2NodUUxemV0MlVtRGtEdGwxdmNoby9laVRjNzM3QmcKclhPQ2F1M0xiVHRGbHcyQ2cxZ3prNVlsV1R6QjFEYXByUC81NStLczZkVUtLWDdXc1NBTEVydjZTajBSVzEvLwpDUTZPcXRBQ3pueVlqWFJISDl6TEJ5czdZVjI3Z2Z2eXF1amRWOE9iazY2c0kra1Q5UGY4alcwMkdWVXQvLzlSCkFnTUJBQUdqSXpBaE1BNEdBMVVkRHdFQi93UUVBd0lDcERBUEJnTlZIUk1CQWY4RUJUQURBUUgvTUEwR0NTcUcKU0liM0RRRUJDd1VBQTRJQkFRQzdPaG55cHpnbXBCTjYyR2dEYkZaVHNGU212eEpITmpLZVpmREpyMWkvcjJvWgphLzBTWjVRMk43QjV3SmVnQTZtY2pZRDBwN2k5KyttUkw1bml3WDJ5SVdTcE1NY0IxcGxEWE13cmFaeW4wVnFZCnVYbE9ZWUVRTnNPT1hjbUduSWtSeE13RW5xRFZTNldTTURoMGtSY0xyV2pvY2twRXJHbHdTcjYwV0xMVWYyckQKclczTU5JNmNNTVkvL1hReUhqR2wwczdZV2oxaGpwcnNRV1VHY0N6N0dHQnhZWUdiRTRLR1VXU3gyZ25DbG94dwpDN1pPdHh6U2RLeU5pWSt1STlxeFdpSXh0ZXppbEg3Q0hscjN6L25XUitJS3IrTUIyWkdGYnNaLy9pZ1Mxbm5vCmtQVzQwSXBvMjNjY2hvcElUbERWVnVzZGxIVlVFTFRlTm45U3Z2emsKLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=
{code}
Reply all
Reply to author
Forward
0 new messages